I’ve put together a few tools to help gather information about your Neutron and network configuration and visualize it in different ways. All of these tools are available as part of my neutron-diag repository on GitHub.

In this post I’m going to look at a tool that will help you visualize the connectivity of network devices on your system.

mk-network-dot

There are a lot of devices involved in your Neutron network configuration. Information originating in one of your instances has two traverse at least seven network devices before seeing the light of day. Understanding how everything connects is critical if you’re trying to debug problems in your envionment.

I originally posted this article on the RDO website.

The players

This document describes the architecture that results from a particular OpenStack configuration, specifically:

• Quantum networking using GRE tunnels;
• A dedicated network controller;
• A single instance running on a compute host

Much of the document will be relevant to other configurations, but details will vary based on your choice of layer 2 connectivity, number of running instances, and so forth.

The examples in this document were generated on a system with Quantum networking but will generally match what you see under Neutron as well, if you replace quantum by neutron in names. The OVS flow rules under Neutron are somewhat more complex and I will cover those in another post.

The documentation for configuring Neutron recommends that a network controller has three physical interfaces:

Before you start, set up a machine to be a dedicated network node. Dedicated network nodes should have the following NICs: the management NIC (called MGMT_INTERFACE), the data NIC (called DATA_INTERFACE), and the external NIC (called EXTERNAL_INTERFACE).

People occasionally ask, “why three interfaces? What if I only have two?”, so I wanted to provide an extended answer that might help people understand what the interfaces are for and what trade-offs are involved in using fewer interfaces.

I’ve recently been exploring the world of Windows scripting, and I ran into a small problem: I was running a script at system startup, and the script was running before the network interface (which was using DHCP) was configured.

There are a number of common solutions proposed to this problem:

• Just wait for some period of time.

  This can work but it’s ugly, and because it doesn’t actually verify the network state it can result in things breaking if some problem prevents Windows from pulling a valid DHCP lease.

The original problem

I’ve recently spent some time working on an OpenStack deployment. I ran into a problem in which the compute service would frequently stop communicating with the AMQP message broker (qpidd).

In order to gather some data on the problem, I ran the following simple test:

• Wait n minutes
• Run nova boot ... to create an instance
• Wait a minute and see if the new instance becomes ACTIVE
• If it works, delete the instance, set n = 2n and repeat

This demonstrated that communication was failing after about an hour, which correlates rather nicely with the idle connection timeout on the firewall.

I use CrashPlan as a backup service. It works and is very simple to set up, but has limited options for controlling bandwidth. In fact, if you’re running it on a headless system (e.g., a fileserver of some sort), your options are effectively “too slow” and “CONSUME EVERYTHING”. There is an open request to add time-based limitations to the application itself, but for now I’ve solved this using a very simple traffic shaping configuration. Because the learning curve for “tc” and friends is surprisingly high, I’m putting my script here in the hopes that other people might find it useful, and so that I can find it when I need to do this again someday.

Like many other folks out there, I have several computers in my house connected to the outside world via a Linux box acting as a NAT gateway. I often want to use application such as BitTorrent and Freenet, which require that a number of ports be forwarded from my external connection to the particular computer on which I happen to be working. It turns out there’s a protocol for this, called UPnP. From Wikipedia: