In some ways, the most exciting thing about Docker isn’t the ability to start containers. That’s been around for a long time in various forms, such as LXC or OpenVZ. What Docker brought to the party was a convenient method of building and distributing the filesystems necessary for running containers. Suddenly, it was easy to build a containerized service and to share it with other people.

I was taking a closer at the systemd-nspawn command, which it seems has been developing it’s own set of container-related superpowers recently, including a number of options for setting up the network environment of a container. Like Docker, systemd-nspawn needs a filesystem on which to operate, but unlike Docker, there is no convenient distribution mechanism and no ecosystem of existing images. In fact, the official documentation seems to assume that you’ll be building your own from scratch. Ain’t nobody got time for that…

This is a long-form response to this question, and describes how to get the nova-docker driver up running with devstack under Ubuntu 14.04 (Trusty). I wrote a similar post for Fedora 21, although that one was using the RDO Juno packages, while this one is using devstack and the upstream sources.

Getting started

We’ll be using the Ubuntu 14.04 cloud image (because my test environment runs on OpenStack).

First, let’s install a few prerequisites:

$ sudo apt-get update
$ sudo apt-get -y install git git-review python-pip python-dev

And generally make sure things are up-to-date:

I have recently started running some “real” services (that is, “services being consumed by someone other than myself”) on top of Kubernetes (running on bare metal), which means I suddenly had to confront the question of how to provide external access to Kubernetes hosted services. Kubernetes provides two solutions to this problem, neither of which is particularly attractive out of the box:

1. There is a field createExternalLoadBalancer that can be set in a service description. This is meant to integrate with load balancers provided by your local cloud environment, but at the moment there is only support for this when running under GCE.

This post comes about indirectly by a request on IRC in #rdo for help getting nova-docker installed on Fedora 21. I ran through the process from start to finish and decided to write everything down for posterity.

Getting started

I started with the Fedora 21 Cloud Image, because I’m installing onto OpenStack and the cloud images include some features that are useful in this environment.

We’ll be using OpenStack packages from the RDO Juno repository. Because there is often some skew between the RDO packages and the current Fedora selinux policy, we’re going to start by putting SELinux into permissive mode (sorry, Dan):

In this post, we’ll look at a method for building minimal Docker images for dynamically linked ELF binaries, and then at a tool for automating this process.

It is tempting, when creating a simple Docker image, to start with one of the images provided by the major distributions. For example, if you need an image that provides tcpdump for use on your Atomic host, you might do something like:

FROM fedora
RUN yum -y install tcpdump

And while this will work, you end up consuming 250MB for tcpdump. In theory, the layering mechanism that Docker uses to build images will reduce the practical impact of this (because other images based on the fedora image will share the common layers), but in practice the size is noticeable, especially if you often find yourself pulling this image into a fresh environment with no established cache.

While working with Docker the other day, I ran into an undesirable interaction between Docker and systemd services that utilize the PrivateTmp directive.

The PrivateTmp directive, if true, “sets up a new file system namespace for the executed processes and mounts private /tmp and /var/tmp directories inside it that is not shared by processes outside of the namespace”. This is a great idea from a security perspective, but can cause some unanticipated consequences.

The problem in a nutshell

1. Start a Docker container:

I regularly use OpenStack on my laptop with libvirt as my hypervisor. I was interested in experimenting with recent versions of the nova-docker driver, but I didn’t have a spare system available on which to run the driver, and I use my regular nova-compute service often enough that I didn’t want to simply disable it temporarily in favor of nova-docker.

NB As pointed out by gustavo in the comments, running two neutron-openvswitch-agents on the same host – as suggested in this article – is going to lead to nothing but sadness and doom. So kids, don’t try this at home. I’m leaving the article here because I think it still has some interesting bits.

I have recently been doing some work with Kubernetes, and wanted to put together a minimal image with which I could test service and pod deployment. Size in this case was critical: I wanted something that would download quickly when initially deployed, because I am often setting up and tearing down Kubernetes as part of my testing (and some of my test environments have poor external bandwidth).

Building thttpd

My go-to minimal webserver is thttpd. For the normal case, building the software is a simple matter of ./configure followed by make. This gets you a dynamically linked binary; using ldd you could build a Docker image containing only the necessary shared libraries:

I like Docker, but I’m not a huge fan of using shell scripts for complex system configuration…and Dockerfiles are basically giant shell scripts.

I was curious whether or not it would be possible to use Puppet during the docker build process. As a test case, I used the ssh module included in the openstack-puppet-modules package.

I started with a manifest like this (in puppet/node.pp):

class { 'ssh': }

And a Dockerfile like this:

The current version of Docker has a very limited set of networking options:

• bridge – connect a container to the Docker bridge
• host – run the container in the global network namespace
• container:xxx – connect a container to the network namespace of another container
• none – do not configure any networking

If you need something more than that, you can use a tool like pipework to provision additional network interfaces inside the container, but this leads to a synchronization problem: pipework can only be used after your container is running. This means that when starting your container, you must have logic that will wait until the necessary networking is available before starting your service.