I ran into an odd problem the other day: I was testing out some configuration changes for a web application by dropping files into /tmp and pointing the application configuration at the appropriate directory. Everything worked out great when testing it by hand…but when starting up the httpd service, the application behaved as if it was unable to find any of the files in /tmp.

My first assumption was that had simply missed something obvious like file permissions or that I had a typo in my configuration, but after repeated checks and lots of testing it was obvious that something else was going on.

This is the first of two articles in which I discuss my work in getting some Windows instances up and running in our OpenStack environment. This article is primarily about problems I encountered along the way.

Motivations

Like many organizations, we have a mix of Linux and Windows in our environment. Some folks in my group felt that it would be nice to let our Windows admins take advantage of OpenStack for prototyping and sandboxing in the same ways our Linux admins can use it.

I was looking for PowerShell solutions for generating a random password (in order to set the Administrator password on a Windows instance provisioned in OpenStack), and found several solutions using the GeneratePassword method of System.Web.Security.Membership (documentation here), along the lines of this:

Function New-RandomComplexPassword ($length=8)
{
    $Assembly = Add-Type -AssemblyName System.Web
    $password = [System.Web.Security.Membership]::GeneratePassword($length,2)
    return $password
}

While this works, I was unhappy with the generated passwords: they were difficult to type or transcribe because they make heavy use of punctuation. For example:

I’ve recently been exploring the world of Windows scripting, and I ran into a small problem: I was running a script at system startup, and the script was running before the network interface (which was using DHCP) was configured.

There are a number of common solutions proposed to this problem:

• Just wait for some period of time.

  This can work but it’s ugly, and because it doesn’t actually verify the network state it can result in things breaking if some problem prevents Windows from pulling a valid DHCP lease.

Occasionally we will deploy a virtual instance into our KVM infrastructure and realize after the fact that we need more local disk space available. This is the process we use to expand the disk image. This process assumes the following:

• You’re using legacy disk partitions. The process for LVM is similar and I will describe that in another post (it’s generally identical except for an additional pvresize thrown in and lvextend in place of resize2fs).
• The partition you need to resize is the last partition on the disk.

This process will work with either a qcow2 or raw disk image. For raw images you can also run fdisk on the host, potentially saving yourself a reboot, but that’s less convenient for qcow2 format images.

Recently, changes from the xmlgawk project have been integrated into GNU awk, and xmlgawk has been renamed to gawkextlib. With both a recent (post-4.0.70) gawk and gawkextlib built and installed correctly, you can write simple XML parsing scripts using gawk.

For example, let’s say you would like to generate a list of disk image files associated with a KVM virtual instance. You can use the virsh dumpxml command to get output like the following:

I really like Markdown, a minimal markup language designed to be readable as plain text that can be rendered into structurally valid HTML. Markdown is already used on sites such as GitHub and all the StackExchange sites.

I use Markdown often enough that it’s become ingrained in my fingers, to the point that I’ve started unconsciously using Markdown syntax in my email. This isn’t particularly useful by itself, although it means that I can take a message and render it to something pretty if I decide it needs to go somewhere other than my sent mail folder.

The original problem

I’ve recently spent some time working on an OpenStack deployment. I ran into a problem in which the compute service would frequently stop communicating with the AMQP message broker (qpidd).

In order to gather some data on the problem, I ran the following simple test:

• Wait n minutes
• Run nova boot ... to create an instance
• Wait a minute and see if the new instance becomes ACTIVE
• If it works, delete the instance, set n = 2n and repeat

This demonstrated that communication was failing after about an hour, which correlates rather nicely with the idle connection timeout on the firewall.

I’ve been playing around with Git, Puppet, and GPG verification of our Puppet configuration repository, and these are some random facts about Git that have come to light as part of the process.

If you want to pull both changes and new tags from a remote repository, you can do this:

$ git fetch
$ git fetch --tags

Or you can do this:

$ git fetch --tags
$ git fetch

What’s the difference? git fetch will leave FETCH_HEAD pointing at the remote HEAD, whereas git fetch --tags will leave FETCH_HEAD pointing at the most recent tag.

Pursuant to my last post, I’ve written a simple man-in-the-middle proxy to intercept communication between the Envoy and the Enphase servers. The code is available here.

What it does

As I detailed in my previous post, the Envoy sends data to Enphase via http POST requests. The proxy intercepts these requests, extracts the XML data from the request, and writes it to a local file (by default in /var/spool/envoy). It then forwards the request on to Enphase, and returns the reply to your Envoy.