HOWTO: OpenStack Deployment using TripleO and the Red Hat OpenStack Director

25 minute read



In this article we will look at how to deploy an OpenStack cloud using TripleO, the upstream project from the Red Hat OpenStack Director. Regardless of what OpenStack distribution you are using OpenStack is essentially OpenStack. Everyone has the same code-base to work with. The main differences between distributions are around what OpenStack projects are part of distribution, how it is supported and the deployment of the distribution. Every distribution has their own OpenStack deployment tool. Clearly deployments differ as they are based on support decisions each distribution makes. However many distributions have created their own proprietary installers. Shouldn't the OpenStack community unite around a common installer? What would be better than using OpenStack to deploy OpenStack? Why should OpenStack administrators have to learn separate proprietary tooling? Why should we be creating unnecessary vendor lock-in for OpenStack's deployment tooling? Installing OpenStack is one thing but what about upgrade and life-cycle management?

This is the promise of TripleO! The TripleO (OpenStack on OpenStack) project was started to solve these problems and bring unification around OpenStack deployment as well as eventually life-cycle management. This has taken quite some time and been a journey but finally the first distribution is using TripleO. Red Hat Enterprise OpenStack Platform 7 has shifted away from foreman/puppet and is now based largely on TripleO. Red Hat is bringing its expertise and learning over the past years around OpenStack deployments and contributing heavily to TripleO.

TripleO Concepts

Before getting into the weeds, we should understand some basic concepts. First TripleO uses OpenStack to deploy OpenStack. It mainly utilizes Ironic for provisioning and Heat for orchestration. Under the hood puppet is used for configuration management. TripleO first deploys an OpenStack cloud used to deploy other OpenStack clouds. This is referred to as the undercloud. The OpenStack cloud environment deployed from undercloud is known as overcloud. The networking requirement is that all systems share a non-routed provisioning network. TripleO also uses PXE to boot and install initial OS image (bootstrap). There are different types of nodes or roles a node can have. In addition to controller and compute you can have nodes for Cinder, CEPH or Swift storage. CEPH storage is also integrated and since most OpenStack deployments use CEPH this is an obvious advantage.


In this environment we have the KVM hypervisor host (Laptop), the undercloud (single VM) and overcloud (1 X compute, 1 Xcontroller). The undercloud and overcloud are all VMs running on the KVM hypervisor host (Laptop). The KVM hypervisor host is on the network and has IP of The undercloud runs on a single VM on the management network and (provisioning) netowrk. The undercloud has an IP address of (eth0). The overcloud is on the (provisioning) and (external) network. This is a very simple network configuration. In a real production environment there will be many more networks used in overcloud.


Deploying Undercloud

In this section we will configure the undercloud. Normally you would deploy OpenStack nodes on bare-metal but since this is designed to run on Laptop or in lab, we are using KVM virtualization. Before beginning install RHEL or CentOS 7.1 on your KVM hypervisor.

Disable NetworkManager.

undercloud# systemctl stop NetworkManager
undercloud# systemctl disable NetworkManager

Enable port forwarding.

undercloud# vi /etc/sysctl.conf
net.ipv4.ip_forward = 1
undercloud# sysctl -p /etc/sysctl.conf

Ensure hostname is static.

undercloud# hostnamectl set-hostname
undercloud# systemctl restart network

Register to subscription manager and enable appropriate repositories for RHEL.

undercloud# subscription-manager register
undercloud# subscription-manager list --available
undercloud# subscription-manager attach --pool=8a85f9814f2c669b014f3b872de132b5
undercloud# subscription-manager repos --disable=*
undercloud# subscription-manager repos --enable=rhel-7-server-rpms --enable=rhel-7-server-optional-rpms --enable=rhel-7-server-extras-rpms --enable=rhel-7-server-openstack-7.0-rpms --enable=rhel-7-server-openstack-7.0-director-rpms

Perform yum update and reboot system.

undercloud# yum update -y && reboot

Install facter and ensure hostname is set properly in /etc/hosts.

undercloud# yum install facter -y
undercloud# ipaddr=$(facter ipaddress_eth0)
undercloud# echo -e "$ipaddr\t\\tundercloud" >> /etc/hosts

Install TripleO packages.

undercloud# yum install python-rdomanager-oscplugin -y 

Create a stack user.

undercloud# useradd stack
undercloud# echo "redhat" | passwd stack --stdin
undercloud# echo "stack ALL=(root) NOPASSWD:ALL" | tee -a /etc/sudoers.d/stack undercloud# chmod 0440 /etc/sudoers.d/stack
undercloud# su - stack

Determine network settings for undercloud. At minimum you need two networks. One for provisioning and the other for the overcloud which should be external network. In this case we have two networks. The undercloud provisioning network and the overcloud external network

[stack@undercloud ~]$ cp /usr/share/instack-undercloud/undercloud.conf.sample ~/undercloud.conf
[stack@undercloud ~]$ vi ~/undercloud.conf
 local_ip =
 undercloud_public_vip =
 undercloud_admin_vip =
 local_interface = eth1
 masquerade_network =
 dhcp_start =
 dhcp_end =
 network_cidr =
 network_gateway =
 discovery_iprange =,

Install the undercloud.

[stack@undercloud ~]$ openstack undercloud install
instack-install-undercloud complete.
The file containing this installation's passwords is at /home/stack/undercloud-passwords.conf.
There is also a stackrc file at /home/stack/stackrc.
These files are needed to interact with the OpenStack services, and should be secured.

Verify undercloud.

 [stack@undercloud ~]$ source ~/stackrc
 [stack@undercloud ~]$ openstack catalog show nova
 | Field | Value |
 | endpoints | regionOne                                                                    |
 |           | publicURL:     |
 |           | internalURL:   |
 |           | adminURL:      |
 |           |                                                                              |
 | name      | nova                                                                         |
 | type      | compute                                                                      |

Deploying Overcloud

The overcloud is as mentioned a separate cloud from the undercloud. They are not sharing any resources, other than the provisioning network. Over and under sometimes confuse people into thinking the overcloud is sitting on top of undercloud, from networking perspective. This is of course not the case. In reality the clouds are sitting side-by-side from one another. The term over and under really refers to a logical relationship between both clouds. We will do a minimal deployment for the overcloud, 1 X controller and 1 X compute.

Create directory for storing undercloud images. These are the images used by Ironic to provision an OpenStack node.

[stack@undercloud]$ mkdir ~/images

Download images from and copy to ~/images.

[stack@undercloud images]$ ls -l
total 2307076
-rw-r-----. 1 stack stack 61419520 Oct 12 16:11 deploy-ramdisk-ironic-7.1.0-39.tar
-rw-r-----. 1 stack stack 155238400 Oct 12 16:11 discovery-ramdisk-7.1.0-39.tar
-rw-r-----. 1 stack stack 964567040 Oct 12 16:12 overcloud-full-7.1.0-39.tar

Extract image tarballs.

[stack@undercloud ~]$ cd ~/images
[stack@undercloud ~]$ for tarfile in *.tar; do tar -xf $tarfile; done

Upload images to Glance.

[stack@undercloud ~]$ openstack overcloud image upload --image-path /home/stack/images
[stack@undercloud ~]$ openstack image list
 | ID | Name |
 | 31c01b42-d164-4898-b615-4787c12d3a53 | bm-deploy-ramdisk |
 | e38057f6-24f2-42d1-afae-bb54dead864d | bm-deploy-kernel |
 | f1708a15-5b9b-41ac-8363-ffc9932534f3 | overcloud-full |
 | 318768c2-5300-43cb-939d-44fb7abca7de | overcloud-full-initrd |
 | 28422b76-c37f-4413-b885-cccb24a4611c | overcloud-full-vmlinuz |

Configure DNS for undercloud. The undercloud system is connected to a network that provides DNS.

[stack@undercloud]$ neutron subnet-list
| id | name | cidr | allocation_pools |
| 532f3344-57ed-4a2f-b438-67a5d60c71fc | | | {"start": "", "end": ""} |
[stack@undercloud ~]$ neutron subnet-update 532f3344-57ed-4a2f-b438-67a5d60c71fc --dns-nameserver

Since we are in nested virtual environment it is necessary to tweak timeouts.

undercloud# sudo su -
undercloud# openstack-config --set /etc/nova/nova.conf DEFAULT rpc_response_timeout 600
undercloud# openstack-config --set /etc/ironic/ironic.conf DEFAULT rpc_response_timeout 600
undercloud# openstack-service restart nova 
undercloud# openstack-service restart ironic
undercloud# exit

Create provisioning and external networks on KVM Hypervisor host. Ensure that NAT forwarding and DHCP is enabled on the external network. The provisioning network should be non-routable and DHCP disabled. The undercloud will handle DHCP services for the provisioning network.

[ktenzer@ktenzer ~]$ cat > /tmp/external.xml <<EOF
   <forward mode='nat'>
      <nat> <port start='1024' end='65535'/>
   <ip address='' netmask=''>
      <dhcp> <range start='' end=''/>
[ktenzer@ktenzer ~]$ virsh net-define /tmp/external.xml
[ktenzer@ktenzer ~]$ virsh net-autostart external
[ktenzer@ktenzer ~]$ virsh net-start external
[ktenzer@ktenzer ~]$ cat > /tmp/provisioning.xml <<EOF
   <ip address='' netmask=''>
[ktenzer@ktenzer ~]$ virsh net-define /tmp/provisioning.xml
[ktenzer@ktenzer ~]$ virsh net-autostart provisioning
[ktenzer@ktenzer ~]$ virsh net-start provisioning

Create VM hulls in KVM using virsh on hypervisor host. You will need to change the disk path to suit your needs.

ktenzer# cd /home/ktenzer/VirtualMachines
ktenzer# for i in {1..2}; do qemu-img create -f qcow2 -o preallocation=metadata overcloud-node$i.qcow2 60G; done
ktenzer# for i in {1..2}; do virt-install --ram 4096 --vcpus 4 --os-variant rhel7 --disk path=/home/ktenzer/VirtualMachines/overcloud-node$i.qcow2,device=disk,bus=virtio,format=qcow2 --noautoconsole --vnc --network network:provisioning --network network:overcloud --name overcloud-node$i --cpu SandyBridge,+vmx --dry-run --print-xml > /tmp/overcloud-node$i.xml; virsh define --file /tmp/overcloud-node$i.xml; done

Enable access on KVM hypervisor host so that Ironic can control VMs.

ktenzer# cat << EOF > /etc/polkit-1/localauthority/50-local.d/50-libvirt-user-stack.pkla
[libvirt Management Access]

Copy ssh key from undercloud system to KVM hypervisor host for stack user.

undercloud$ ssh-copy-id -i ~/.ssh/ stack@

Save the MAC addresses for the provisioning network on the VMs. Ironic needs to know what MAC addresses a node has associated for provisioning network.

[stack@undercloud ~]$ for i in {1..2}; do virsh -c qemu+ssh://stack@ domiflist overcloud-node$i | awk '$3 == "mgmt" {print $5};'; done > /tmp/nodes.txt
[stack@undercloud ~]$ cat /tmp/nodes.txt

Create JSON file for Ironic baremetal node configuration. In this case we are configuring two nodes which are of course the virtual machines we already created. The pm_addr IP is set to IP of the KVM hypervisor host.

[stack@undercloud ~]$ jq . << EOF > ~/instackenv.json
  "ssh-user": "stack",
  "ssh-key": "$(cat ~/.ssh/id_rsa)",
  "power_manager": "nova.virt.baremetal.virtual_power_driver.VirtualPowerManager",
  "host-ip": "",
  "arch": "x86_64",
  "nodes": [
      "pm_addr": "",
      "pm_password": "$(cat ~/.ssh/id_rsa)",
      "pm_type": "pxe_ssh",
      "mac": [
        "$(sed -n 1p /tmp/nodes.txt)"
      "cpu": "4",
      "memory": "4096",
      "disk": "60",
      "arch": "x86_64",
      "pm_user": "stack"
      "pm_addr": "",
      "pm_password": "$(cat ~/.ssh/id_rsa)",
      "pm_type": "pxe_ssh",
      "mac": [
        "$(sed -n 2p /tmp/nodes.txt)"
      "cpu": "4",
      "memory": "4096",
      "disk": "60",
      "arch": "x86_64",
      "pm_user": "stack"

Validate JSON file.

[stack@undercloud ~]$ curl -O
python -f instackenv.json
INFO:__main__:Checking node
DEBUG:__main__:Identified virtual node
INFO:__main__:Checking node
DEBUG:__main__:Identified virtual node
DEBUG:__main__:Baremetal IPs are all unique.
DEBUG:__main__:MAC addresses are all unique.

SUCCESS: instackenv validator found 0 errors

Add nodes to Ironic

[stack@undercloud ~]$ openstack baremetal import --json instackenv.json

List newly added baremetal nodes.

[stack@undercloud ~]$ openstack baremetal list
| UUID | Name | Instance UUID | Power State | Provision State | Maintenance |
| cd620ad0-4563-44a5-8078-531b7f906188 | None | None | power off | available | False |
| 44df8163-7381-46a7-b016-a0dd18bfee53 | None | None | power off | available | False |

Enable nodes for baremetal provisioning and inspect ram and kernel images.

[stack@undercloud ~]$ openstack baremetal configure boot
[stack@undercloud ~]$ ironic node-show cd620ad0-4563-44a5-8078-531b7f906188 | grep -A1 deploy

| driver_info | {u'ssh_username': u'stack', u'deploy_kernel': u'50125b15-9de3-4f03-bfbb- |
| | 76e740741b68', u'deploy_ramdisk': u'25b55027-ca57-4f15-babe- |
| | 6e14ba7d0b0c', u'ssh_key_contents': u'-----BEGIN RSA PRIVATE KEY----- |
[stack@undercloud ~]$ openstack image show 50125b15-9de3-4f03-bfbb-76e740741b68
| Field | Value |
| checksum | 061e63c269d9c5b9a48a23f118c865de |
| container_format | aki |
| created_at | 2015-10-12T10:22:38.000000 |
| deleted | False |
| disk_format | aki |
| id | 50125b15-9de3-4f03-bfbb-76e740741b68 |
| is_public | True |
| min_disk | 0 |
| min_ram | 0 |
| name | bm-deploy-kernel |
| owner | 2ad8c320cf7040ef9ec0440e94238f58 |
| properties | {} |
| protected | False |
| size | 5027584 |
| status | active |
| updated_at | 2015-10-12T10:22:38.000000 |
[stack@undercloud ~]$ openstack image show 25b55027-ca57-4f15-babe-6e14ba7d0b0c
| Field | Value |
| checksum | eafcb9601b03261a7c608bebcfdff41c |
| container_format | ari |
| created_at | 2015-10-12T10:22:38.000000 |
| deleted | False |
| disk_format | ari |
| id | 25b55027-ca57-4f15-babe-6e14ba7d0b0c |
| is_public | True |
| min_disk | 0 |
| min_ram | 0 |
| name | bm-deploy-ramdisk |
| owner | 2ad8c320cf7040ef9ec0440e94238f58 |
| properties | {} |
| protected | False |
| size | 56355601 |
| status | active |
| updated_at | 2015-10-12T10:22:40.000000 |
Ironic at this point only supports IPMI booting and since we are using VMs we need to use ssh_pxe. This is a workaround to allow that to work.
[stack@undercloud ~]$ sudo su -
undercloud# cat << EOF > /usr/bin/bootif-fix
#!/usr/bin/env bash

while true;
        do find /httpboot/ -type f ! -iname "kernel" ! -iname "ramdisk" ! -iname "*.kernel" ! -iname "*.ramdisk" -exec sed -i 's|{mac|{net0/mac|g' {} +;

undercloud# chmod a+x /usr/bin/bootif-fix
undercloud# cat << EOF > /usr/lib/systemd/system/bootif-fix.service
Description=Automated fix for incorrect iPXE BOOFIF



undercloud# systemctl daemon-reload
undercloud# systemctl enable bootif-fix
undercloud# systemctl start bootif-fix
undercloud# exit

Create new flavor for the baremetal nodes and set boot option to local.

undercloud$ openstack flavor create --id auto --ram 4096 --disk 58 --vcpus 4 baremetal
undercloud$ openstack flavor set --property "cpu_arch"="x86_64" --property "capabilities:boot_option"="local" baremetal

Perform introspection on baremetal nodes. This will discover hardware and configure node roles.

[stack@undercloud ~]$ openstack baremetal introspection bulk start
Setting available nodes to manageable...
Starting introspection of node: 79f2a51c-a0f0-436f-9e8a-c082ee61f938
Starting introspection of node: 8ba244fd-5362-45fe-bb6c-5f15f2949912
Waiting for discovery to finish...
Discovery for UUID 79f2a51c-a0f0-436f-9e8a-c082ee61f938 finished successfully.
Discovery for UUID 8ba244fd-5362-45fe-bb6c-5f15f2949912 finished successfully.
Setting manageable nodes to available...
Node 79f2a51c-a0f0-436f-9e8a-c082ee61f938 has been set to available.
Node 8ba244fd-5362-45fe-bb6c-5f15f2949912 has been set to available.

To check progress of introspection.

[stack@undercloud ~]$ sudo journalctl -f -l -u openstack-ironic-discoverd -u openstack-ironic-discoverd-dnsmasq -f


List the Ironic baremetal nodes. Nodes should be available if introspection worked.

[stack@undercloud ~]$ ironic node-list 
| UUID | Name | Instance UUID | Power State | Provision State | Maintenance |
| cd620ad0-4563-44a5-8078-531b7f906188 | None | None | power on | available | False |
| 44df8163-7381-46a7-b016-a0dd18bfee53 | None | None | power on | available | False |

Deploy overcloud.

[stack@undercloud ~]$ openstack overcloud deploy --templates --control-scale 1 --compute-scale 1 --neutron-tunnel-types vxlan --neutron-network-type vxlan
Overcloud Endpoint:
Overcloud Deployed

Check status of Heat resources to monitor status of overcloud deployment.

[stack@undercloud ~]$ heat resource-list -n 5 overcloud

Once the OS install is complete on the baremetal nodes you can follow progress of the OpenStack overcloud configuration.

[stack@undercloud ~]$ nova list
| ID                                   | Name                   | Status | Task State | Power State | Networks                |
| 507d1172-fc73-476b-960f-1d9bf7c1c270 | overcloud-compute-0    | ACTIVE | -          | Running     | ctlplane=|
| ff0e5e15-5bb8-4c77-81c3-651588802ebd | overcloud-controller-0 | ACTIVE | -          | Running     | ctlplane=|
[stack@undercloud ~]$ ssh heat-admin@
overcloud-controller-0$ sudo -i
overcloud-controller-0# journalctl -f -u os-collect-config

Deploying using the OpenStack Director UI

The overcloud deployment can be done using the UI. You can even do the preliminary configuration using the CLI and run deployment from UI.


We can see exactly what OpenStack services will be configured in the overcloud.


Deployment status is shown and using the UI it is also to see when baremetal nodes have been completely provisioned.


Deployment details are available in the deployment log.


Once deployment is complete using the UI, the overcloud must be initialized.


Upon completion the overcloud is available and can be accessed.



In this article we have discussed how OpenStack distributions have a proprietary mindset in regards to their deployment tools. We have discussed the need for a OpenStack community sponsored upstream project responsible for deployment and life-cycle management. That project is TripleO and Red Hat is the first distribution to ship its deployment tool based on TripleO. Using OpenStack to deploy OpenStack not only benefits entire community but also administrators and end-users. Finally we have seen how to deploy both the undercloud as well as overcloud using TripleO and the Red Hat OpenStack Director. Hopefully you found this article informative and useful. I would be very interested in hearing your feedback on this topic, so please share.

Happy OpenStacking!

(c) 2015 Keith Tenzer