Let’s say that you have set up an environment using tripleo-quickstart and you would like to add another virtual machine to the mix that has both “external” connectivity (“external” in quotes because I am using it in the same way as the quickstart does w/r/t the undercloud) and connectivity to the overcloud nodes. How would you go about setting that up?

For a concrete example, let’s presume you have deployed an environment using the default tripleo-quickstart configuration, which looks like this:

  - name: control_0
    flavor: control

  - name: compute_0
    flavor: compute

extra_args: >-
  --neutron-network-type vxlan
  --neutron-tunnel-types vxlan
  --ntp-server pool.ntp.org

network_isolation: true

That gets you one controller, one compute node, and enables network isolation. When your deployment is complete, networking from the perspective of the undercloud looks like this:

  • eth0 is connected to the host’s brext bridge and gives the undercloud NAT access to the outside world. The interface will have an address on the network.

  • eth1 is connected to the host’s brovc bridge, which is the internal network for the overcloud. The interface is attached to the OVS bridge br-ctlplane.

    The br-ctlplane bridge has the address

And your overcloud environment probably looks something like this:

[stack@undercloud ~]$ nova list
| ID    ...| Name                    | Status |...| Networks           |
| 32f6ec...| overcloud-controller-0  | ACTIVE |...| ctlplane= |
| d98474...| overcloud-novacompute-0 | ACTIVE |...| ctlplane= |

We want to set up a new machine that has the same connectivity as the undercloud.

Upload an image

Before we can boot a new vm we’ll need an image; let’s start with the standard CentOS 7 cloud image. First we’ll download it:

curl -O http://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2

Let’s add a root password to the image and disable cloud-init, since we’re not booting in a cloud environment:

virt-customize -a CentOS-7-x86_64-GenericCloud.qcow2 \
  --root-password password:changeme \
  --run-command "yum -y remove cloud-init"

Now let’s upload it to libvirt:

virsh vol-create-as oooq_pool centos-7-cloud.qcow2 8G \
  --format qcow2 \
  --allocation 0
virsh vol-upload --pool oooq_pool centos-7-cloud.qcow2 \

Boot the vm

I like to boot from a copy-on-write clone of the image, so that I can use the base image multiple times or quickly revert to a pristine state, so let’s first create that clone:

virsh vol-create-as oooq_pool myguest.qcow2 10G \
  --allocation 0 --format qcow2 \
  --backing-vol centos-7-cloud.qcow2 \
  --backing-vol-format qcow2

And then boot our vm:

virt-install --disk vol=oooq_pool/myguest.qcow2,bus=virtio \
  --import \
  -r 2048 -n myguest --cpu host \
  --os-variant rhel7 \
  -w bridge=brext,model=virtio \
  -w bridge=brovc,model=virtio \
  --serial pty \

The crucial parts of the above command are the two -w ... arguments, which create interfaces attached to the named bridges.

We can now connect to the console and log in as root:

$ virsh console myguest
localhost login: root

We’ll see that the system already has an ip address on the “external” network:

[root@localhost ~]# ip addr show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:7f:5c:5a brd ff:ff:ff:ff:ff:ff
    inet brd scope global dynamic eth0
       valid_lft 3517sec preferred_lft 3517sec
    inet6 fe80::5054:ff:fe7f:5c5a/64 scope link 
       valid_lft forever preferred_lft forever

And we have external connectivity:

[root@localhost ~]# ping -c1 google.com
PING google.com ( 56(84) bytes of data.
64 bytes from lga25s40-in-f14.1e100.net ( icmp_seq=1 ttl=56 time=20.6 ms

--- google.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 20.684/20.684/20.684/0.000 ms

Let’s give eth1 an address on the ctlplane network:

[root@localhost ~]# ip addr add dev eth1
[root@localhost ~]# ip link set eth1 up

Now we can access the undercloud:

[root@localhost ~]# ping -c1
PING ( 56(84) bytes of data.
64 bytes from icmp_seq=1 ttl=64 time=0.464 ms

--- ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.464/0.464/0.464/0.000 ms

As well as all of the overcloud hosts using their addresses on the same network:

[root@localhost ~]# ping -c1
PING ( 56(84) bytes of data.
64 bytes from icmp_seq=1 ttl=64 time=0.464 ms

--- ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.464/0.464/0.464/0.000 ms

Allocating an address using DHCP

In the above instructions we’ve manually assigned an ip address on the ctlplane network. This works fine for testing, but it could ultimately prove problematic if neutron were to allocate the same address to another overcloud host. We can use neutron to configure a static dhcp lease for our new host.

First, we need the MAC address of our guest:

virthost$ virsh dumpxml myguest |
  xmllint --xpath '//interface[source/@bridge="brovc"]' -
<interface type="bridge">
  <mac address="52:54:00:42:d6:c2"/>
  <source bridge="brovc"/>
  <target dev="tap9"/>
  <model type="virtio"/>
  <alias name="net1"/>
  <address type="pci" domain="0x0000" bus="0x00" slot="0x04" function="0x0"/>

And then on the undercloud we run neutron port-create to create a port and associate it with our MAC address:

[stack@undercloud]$ neutron port-create --mac-address 52:54:00:42:d6:c2 ctlplane

Now if we run dhclient on our guest, it will acquire a lease from the neutron-managed DHCP server:

[root@localhost]# dhclient -d eth1
Internet Systems Consortium DHCP Client 4.2.5
Copyright 2004-2013 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/

Listening on LPF/eth1/52:54:00:42:d6:c2
Sending on   LPF/eth1/52:54:00:42:d6:c2
Sending on   Socket/fallback
DHCPREQUEST on eth1 to port 67 (xid=0xc90c0ba)
DHCPACK from (xid=0xc90c0ba)
bound to -- renewal in 42069 seconds.

We can make this persistent by creating /etc/sysconfig/network-scripts/ifcfg-eth1:

[root@localhost]# cd /etc/sysconfig/network-scripts
[root@localhost]# sed s/eth0/eth1/g ifcfg-eth0 > ifcfg-eth1
[root@localhost]# ifup eth1
Determining IP information for eth1... done.