In Quantum in Too Much Detail, I discussed the architecture of a Neutron deployment in detail. Since that article was published, Neutron gained the ability to handle multiple external networks with a single L3 agent. While I wrote about that back in 2014, I covered the configuration side of it in much more detail than I discussed the underlying network architecture. This post addresses the architecture side.

The players

This document describes the architecture that results from a particular OpenStack configuration, specifically:

The current version of Docker has a very limited set of networking options:

• bridge – connect a container to the Docker bridge
• host – run the container in the global network namespace
• container:xxx – connect a container to the network namespace of another container
• none – do not configure any networking

If you need something more than that, you can use a tool like pipework to provision additional network interfaces inside the container, but this leads to a synchronization problem: pipework can only be used after your container is running. This means that when starting your container, you must have logic that will wait until the necessary networking is available before starting your service.

Update (2018-03-22) Since I wrote this document back in 2014, Docker has developed the macvlan network driver. That gives you a supported mechanism for direct connectivity to a local layer 2 network. I’ve written an article about working with the macvlan driver.

This article discusses four ways to make a Docker container appear on a local network. These are not suggested as practical solutions, but are meant to illustrate some of the underlying network technology available in Linux.

This article expands on my answer to Add multiple specific IPs to instance, a question posted to ask.openstack.org.

In order to serve out SSL services from an OpenStack instance, you will generally want one local ip address for each SSL virtual host you support. It is possible to create an instance with multiple fixed addresses, but there are a few complications to watch out for.

Assumptions

This article assumes that the following resources exist:

In the old days (so, like, last year), Neutron supported a single external network per L3 agent. You would run something like this…

$ neutron net-create external --router:external=true

…and neutron would map this to the bridge defined in external_network_bridge in /etc/neutron/l3_agent.ini. If you wanted to support more than a single external network, you would need to run multiple L3 agents, each with a unique value for external_network_bridge.

There is now a better option available.

I’ve just put a video on Youtube that looks at the steps required to set up the external bridge (br-ex) on a single-interface system:

Normally I like to post solutions, but today’s post is about a vexing problem to which I have not been able to find a solution.

This started as a simple attempt to set up external connectivity on an all-in-one Icehouse install deployed on an OpenStack instance. I wanted to add eth0 to br-ex in order to model a typical method for providing external connectivity, but I ran into a very odd problem: the system would boot and work fine for a few seconds, but would then promptly lose network connectivity.

In my previous post I discussed a problem I was having setting a persistent MAC address on an OVS bridge device. It looks like the short answer is, “don’t use ip link set ...” for this purpose.

You can set the bridge MAC address via ovs-vsctl like this:

ovs-vsctl set bridge br-ex other-config:hwaddr=$MACADDR

So I’ve updated my ifconfig-br-ex to look like this:

DEVICE=br-ex
DEVICETYPE=ovs
TYPE=OVSBridge
ONBOOT=yes
OVSBOOTPROTO=dhcp
OVSDHCPINTERFACES=eth0
MACADDR=fa:16:3e:ef:91:ec
OVS_EXTRA="set bridge br-ex other-config:hwaddr=$MACADDR"

The OVS_EXTRA parameter gets passed to the add-br call like this:

I run OpenStack on my laptop, and I’ve been chasing down a pernicious problem with OVS bridge interfaces under both F19 and F20. My OpenStack environment relies on an OVS bridge device named br-ex for external connectivity and for making services available to OpenStack instances, but after rebooting, br-ex was consistently unconfigured, which caused a variety of problems.

This is the network configuration file for br-ex on my system:

DEVICE=br-ex
DEVICETYPE=ovs
TYPE=OVSBridge
BOOTPROT=static
IPADDR=192.168.200.1
NETMASK=255.255.255.0
ONBOOT=yes
NM_CONTROLLED=no
ZONE=openstack

Running ifup br-ex would also fail to configure the interface, but running ifdown br-ex; ifup br-ex would configure things appropriately.

These are my notes on making OpenStack play well with firewalld and NetworkManager.

NetworkManager

By default, NetworkManager attempts to start a DHCP client on every new available interface. Since booting a single instance in OpenStack can result in the creation of several virtual interfaces, this results in a lot of:

May 19 11:58:24 pk115wp-lkellogg NetworkManager[1357]: <info>
  Activation (qvb512640bd-ee) starting connection 'Wired connection 2'

You can disable this behavior by adding the following to /etc/NetworkManager/NetworkManager.conf: