I’ve been playing around with the LXC support in libvirt recently, and I’m trying to use a model where each LXC instance is backed by a dedicated LVM volume. This means that the process of starting an instance is:

  • mount the instance root filesystem if necessary
  • start the instance

It’s annoying to have to do this by hand. I could simply add all the LXC filesystems to /etc/fstab, but this would mean and extra step when creating and deleting each instance.

I thought about creating a wrapper script to handle the mounting (and unmounting, perhaps) for me, but this leads to another problem: I need a way to figure out which backing device corresponding to the virtual instance I’m trying to boot.

I could just adopt a strict naming scheme, so that for a given instance “foo” the backing store would be /dev/vg_something/domain-foo, but that’s too easy! In order to make life interesting:

  • I’ve added some namespace-prefixed metadata to the instance XML description, and
  • Written a shell script to extract this data (and more) using xmllint.

Metadata

The Libvirt domain XML format allows for a general metadata section:

The metadata node can be used by applications to store custom metadata in the form of XML nodes/trees. Applications must use custom namespaces on their XML nodes/trees, with only one top-level element per namespace (if the application needs structure, they should have sub-elements to their namespace element).

This means that if you want to add metadata to a domain description, you need to define a namespace and use a namespace prefix. For example, if I want to add a device element pointing at the backend storage device for my domain, it might look like this:

<metadata xmlns:ob="http://oddbit.com/ns/libvirt/1">
    <ob:device>/dev/vg0/instance</ob:device>
</metadata>

Extracting metadata with xmllint

The xmllint tool, part of libxml, can extract nodes from an XML document using xpath expressions. Unfortunately, while xmllint does have namespace support, it’s not particularly convenient. Using the --shell mode there’s a helpful setns command:

$ xmllint --shell domain.xml
/ > setns x=http://oddbit.com/ns/libvirt/1
/ > xpath //x:device
Object is a Node Set :
Set contains 1 nodes:
1  ELEMENT ob:device

But that’s not available from the command line. We can use the namespace-uri() and local-name() xpath functions to get to the same place, albeit more verbosely. An equivalent to the above shell session would be:

$ xmllint --xpath '//*[namespace-uri()="http://oddbit.com/ns/libvirt/1" and local-name()="device"]'
<ob:device>/dev/vg0/instance</ob:device>

Putting it all together

The following shell script looks through all inactive LXC domains and figures out:

  • If it should try to start them, using the <oddbit:autostart> element,

  • What the backend storage is, using the <oddbit:device> element, and

  • Where to mount it, by looking for the libvirt <filesystem> element with a target of /.

    #!/bin/sh
    
    tmpfile=$(mktemp)
    trap 'rm -f $tmpfile' EXIT
    
    virsh -c lxc:/// list --name --inactive | while read domain; do
      [ "$domain" ] || continue
    
      virsh dumpxml $domain > $tmpfile
    
      autostart=$(xmllint --xpath '//domain/metadata/*[namespace-uri()="http://oddbit.com/ns/libvirt/1" and local-name()="autostart"]/text()' $tmpfile)
    
      [ "$autostart" = True ] || continue
    
      device=$(xmllint --xpath '//domain/metadata/*[namespace-uri()="http://oddbit.com/ns/libvirt/1" and local-name()="device"]/text()' $tmpfile)
      mount=$(xmllint --xpath 'string(//filesystem/target[@dir = "/"]/../source/@dir)' $tmpfile)
    
      echo "$domain $autostart $device $mount"
    done
    

I’m not actually planning on using this in practice. I’ll probably go the naming scheme route. But this was fun to figure out.