In Quantum in Too Much Detail, I discussed the architecture of a Neutron deployment in detail. Since that article was published, Neutron gained the ability to handle multiple external networks with a single L3 agent. While I wrote about that back in 2014, I covered the configuration side of it in much more detail than I discussed the underlying network architecture. This post addresses the architecture side.

The players

This document describes the architecture that results from a particular OpenStack configuration, specifically:

In an OpenStack environment, cloud-init generally fetches information from the metadata service provided by Nova. It also has support for reading this information from a configuration drive, which under OpenStack means a virtual CD-ROM device attached to your instance containing the same information that would normally be available via the metadata service.

It is possible to generate your network configuration from this configuration drive, rather than relying on the DHCP server provided by your OpenStack environment. In order to do this you will need to make the following changes to your Nova configuration:

Setting the stage

I ran into a problem earlier this week deploying RDO Icehouse under RHEL 6. My target systems were a set of libvirt guests deployed from the RHEL 6 KVM guest image, which includes cloud-init in order to support automatic configuration in cloud environments. I take advantage of this when using libvirt by attaching a configuration drive so that I can pass in ssh keys and a user-data script.

This article expands on my answer to Add multiple specific IPs to instance, a question posted to ask.openstack.org.

In order to serve out SSL services from an OpenStack instance, you will generally want one local ip address for each SSL virtual host you support. It is possible to create an instance with multiple fixed addresses, but there are a few complications to watch out for.

Assumptions

This article assumes that the following resources exist:

In the old days (so, like, last year), Neutron supported a single external network per L3 agent. You would run something like this…

$ neutron net-create external --router:external=true

…and neutron would map this to the bridge defined in external_network_bridge in /etc/neutron/l3_agent.ini. If you wanted to support more than a single external network, you would need to run multiple L3 agents, each with a unique value for external_network_bridge.

There is now a better option available.

Due to an unfortunate incident involving sleep mode and an overheated backpack I had the “opportunity” to rebuild my laptop. Since this meant reinstalling OpenStack I used this as an excuse to finally move to the ML2 network plugin for Neutron.

I was attempting to add an external network using the normal incantation:

neutron net-create external -- --router:external=true \
    --provider:network_type=flat \
    --provider:physical_network=physnet1

While this command completed successfully, I was left without any connectivity between br-int and br-ex, despite having in my /etc/neutron/plugins/ml2/ml2_conf.ini:

I’ve put together a few tools to help gather information about your Neutron and network configuration and visualize it in different ways. All of these tools are available as part of my neutron-diag repository on GitHub.

In this post I’m going to look at a tool that will help you visualize the connectivity of network devices on your system.

mk-network-dot

There are a lot of devices involved in your Neutron network configuration. Information originating in one of your instances has two traverse at least seven network devices before seeing the light of day. Understanding how everything connects is critical if you’re trying to debug problems in your envionment.

Heat is a template-based orchestration mechanism for use with OpenStack. With Heat, you can deploy collections of resources – networks, servers, storage, and more – all from a single, parameterized template.

In this article I will introduce Heat templates and the heat command line client.

Writing templates

Because Heat began life as an analog of AWS CloudFormation, it supports the template formats used by the CloudFormation (CFN) tools. It also supports its own native template format, called HOT (“Heat Orchestration Templates”). In this article I will be using the HOT template syntax, which is fully specified on the OpenStack website.

The documentation for configuring Neutron recommends that a network controller has three physical interfaces:

Before you start, set up a machine to be a dedicated network node. Dedicated network nodes should have the following NICs: the management NIC (called MGMT_INTERFACE), the data NIC (called DATA_INTERFACE), and the external NIC (called EXTERNAL_INTERFACE).

People occasionally ask, “why three interfaces? What if I only have two?”, so I wanted to provide an extended answer that might help people understand what the interfaces are for and what trade-offs are involved in using fewer interfaces.