Multiple external networks with a single L3 agent
In the old days (so, like, last year), Neutron supported a single external network per L3 agent. You would run something like this…
$ neutron net-create external --router:external=true
…and neutron would map this to the bridge defined in
external_network_bridge in /etc/neutron/l3_agent.ini. If you
wanted to support more than a single external network, you would need
to run multiple L3 agents, each with a unique value for
external_network_bridge.
There is now a better option available.
Assumptions⌗
In this post, I’m assuming:
- You’re using the ML2 plugin for Neutron.
- You’re using the Open vSwitch mechanism driver for the ML2 plugin
- You have
eth1andeth2connected directly to networks that you would like to make available as external networks in OpenStack.
Create your bridges⌗
For each external network you wish to support, create a new OVS
bridge. For example, assuming that we want to make a network attached
to eth1 and a network attached to eth2 available to tenants:
# ovs-vsctl add-br br-eth1
# ovs-vsctl add-port br-eth1 eth1
# ovs-vsctl add-br br-eth2
# ovs-vsctl add-port br-eth2 eth2
Realistically, you would accomplish this via your system’s native network configuration mechanism, but I’m going to gloss over that detail for now.
Configure the L3 Agent⌗
Start with the following comment in l3_agent.ini:
# When external_network_bridge is set, each L3 agent can be associated
# with no more than one external network. This value should be set to the UUID
# of that external network. To allow L3 agent support multiple external
# networks, both the external_network_bridge and gateway_external_network_id
# must be left empty.
Following those instructions, make sure that both
external_network_bridge and gateway_external_network_id are unset
in l3_agent.ini.
Configure the ML2 Plugin⌗
We are creating “flat” networks in this example, so we need to make
sure that we can create flat networks. Make sure that the
type_drivers parameter of the [ml2] section of your plugin
configuration includes flat:
[ml2]
type_drivers = local,flat,gre,vxlan
In the [ml2_type_flat] section, need to create a list of physical
network names that can be used to create flat networks. If you want
all physical networks to be available for flat networks, you can use
*:
[ml2_type_flat]
flat_networks = *
Both of these changes probably go in /etc/neutron/plugin.ini, but
may going elsewhere depending on how your system is configured.
Configure the Open vSwitch Agent⌗
For each bridge, you will need to add entries to both the
network_vlan_ranges and bridge_mappings parameters of the [ovs]
section of your plugin configuration. For the purposes of this post,
that means:
[ovs]
network_vlan_ranges = physnet1,physnet2
bridge_mappings = physnet1:br-eth1,physnet2:br-eth2
This will probably go in /etc/neutron/plugin.ini. Specifically, it
needs to go wherever your neutron-openvswitch-agent process is
looking for configuration information. So you if you see this:
$ ps -fe | grep openvswitch-agent
neutron 12529 1 0 09:50 ? 00:00:08 /usr/bin/python /usr/bin/neutron-openvswitch-agent --config-file /usr/share/neutron/neutron-dist.conf --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini --log-file /var/log/neutron/openvswitch-agent.log
…then you would make the changes to /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini.
Restart Neutron⌗
You will need to restart both the l3 agent and the openvswitch agent. If you’re on a recent Fedora/RHEL/CentOS, you can restart all Neutron services like this:
# openstack-service restart neutron
Inspect your Open vSwitch Configuration⌗
As root, run ovs-vsctl show. You should see something like this:
f4a4312b-307e-4c3c-b728-9434000a34ff
Bridge br-int
Port br-int
Interface br-int
type: internal
Port "int-br-eth2"
Interface "int-br-eth2"
Port int-br-ex
Interface int-br-ex
Port "int-br-eth1"
Interface "int-br-eth1"
Bridge "br-eth2"
Port "br-eth2"
Interface "br-eth2"
type: internal
Port "phy-br-eth2"
Interface "phy-br-eth2"
Port "eth2"
Interface "eth2"
Bridge "br-eth1"
Port "br-eth1"
Interface "br-eth1"
type: internal
Port "phy-br-eth1"
Interface "phy-br-eth1"
Port "eth1"
Interface "eth1"
ovs_version: "2.0.1"
Here you can see the OVS bridge br-eth1 and br-eth2, each with the
appropriate associated physical interface and links to the integration
bridge, br-int.
Create your external networks⌗
With admin credentials, use the net-create and subnet-create
commands to create the appropiate networks:
$ neutron net-create external1 -- --router:external=true \
--provider:network_type=flat \
--provider:physical_network=physnet1
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
| admin_state_up | True |
| id | 23f4b5f6-14fd-4bab-a8b0-445257bbc0d1 |
| name | external1 |
| provider:network_type | flat |
| provider:physical_network | physnet1 |
| provider:segmentation_id | |
| router:external | True |
| shared | False |
| status | ACTIVE |
| subnets | |
| tenant_id | 6f736b1361b74789a81d4d53d88be3c5 |
+---------------------------+--------------------------------------+
$ neutron subnet-create --disable-dhcp external1 10.1.0.0/24
+------------------+--------------------------------------------+
| Field | Value |
+------------------+--------------------------------------------+
| allocation_pools | {"start": "10.1.0.2", "end": "10.1.0.254"} |
| cidr | 10.1.0.0/24 |
| dns_nameservers | |
| enable_dhcp | False |
| gateway_ip | 10.1.0.1 |
| host_routes | |
| id | 363ba289-a989-4acb-ac3b-ffaeb90796fc |
| ip_version | 4 |
| name | |
| network_id | 23f4b5f6-14fd-4bab-a8b0-445257bbc0d1 |
| tenant_id | 6f736b1361b74789a81d4d53d88be3c5 |
+------------------+--------------------------------------------+
$ neutron net-create external2 -- --router:external=true \
--provider:network_type=flat \
--provider:physical_network=physnet2
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
| admin_state_up | True |
| id | 762be5de-31a2-46b8-925c-0967871f8181 |
| name | external2 |
| provider:network_type | flat |
| provider:physical_network | physnet2 |
| provider:segmentation_id | |
| router:external | True |
| shared | False |
| status | ACTIVE |
| subnets | |
| tenant_id | 6f736b1361b74789a81d4d53d88be3c5 |
+---------------------------+--------------------------------------+
$ neutron subnet-create --disable-dhcp external2 10.2.0.0/24
+------------------+--------------------------------------------+
| Field | Value |
+------------------+--------------------------------------------+
| allocation_pools | {"start": "10.2.0.2", "end": "10.2.0.254"} |
| cidr | 10.2.0.0/24 |
| dns_nameservers | |
| enable_dhcp | False |
| gateway_ip | 10.2.0.1 |
| host_routes | |
| id | edffc5c6-0e16-4da0-8eba-9d79ab9fd2fe |
| ip_version | 4 |
| name | |
| network_id | 762be5de-31a2-46b8-925c-0967871f8181 |
| tenant_id | 6f736b1361b74789a81d4d53d88be3c5 |
+------------------+--------------------------------------------+
This assumes that eth1 is connected to a network using
10.1.0.0/24 and eth2 is connected to a network using
10.2.0.0/24, and that each network has a gateway sitting at the
corresponding .1 address.
And you’re all set!