Deploying Ubuntu 12.04 on XenServer Made Easy
To follow up on my previous post about disk errors with Ubuntu 12.04 on XenServer, I wanted to cover the process I've put in place for provisioning Ubuntu VMs.
With PaaS.io, I have a mixture between systems deployed on bare metal and virtualized. With the virtualized systems, I set out to make the provisioning as easy as if I was using an IaaS provider, while still giving me fine control over sizing and placement. Some of the decisions there are enough for another post.
Ubuntu 12.04 is the new hotness and I'd been anxiously awaiting it, with plans to progressively roll it out throughout PaaS.io.
With XenServer, it provides an easy template for provisioning Ubuntu Lucid 10.04 VMs and even for some newer releases. However, the way XenSever provisions Ubuntu is to essentially netboot it and install it from a remote source. Because of that, we can't simply use one of those templates buts install a newer release.
Fortunately, the template itself is very simple, and which release it installs is just a configuration parameter.
To create a new Ubuntu 12.04 template, simply log into the XenServer console and
runt he following commands. It will clone the Lucid template and then change
the parameter for the release to install from
Now you will find a "Ubuntu Precise (64-bit)" option in the template list of XenCenter.
Now, we can actually provision a new box. Next, one interesting discovery was that you can pass in a kickstart script in the boot parameters options for the new VM.
Kickstart allows you to do a scripted installation, automated pretty much every aspect of the system. Instead of picking a bunch of options, it allows for an easy, repeatable process that basically leaves the machine completely ready to go.
To use a kickstart script, make a script available over HTTP somewhere on your
LAN or on the public internet. By the time it is grabbed, the machine will have
an IP, DNS, and all. Then, simply add
ks=http://url/to/your/kickstart to the
beginning part of "advanced OS boot parameters" option when selecting the
Below is a cleaned version of the kickstart script I used on my Ubuntu VMs. The main things of note:
- Sets up my partition table
- Configures base system with ubuntu-minimal as well as some common packages like openssh-server, curl, wget, and screen.
- Disables the creation of the
ubuntuuser (I create a normal every day user through chef)
- Configures the fstab with
barrier=0as mentioned before
/bin/shfrom pointing to
- Updates apt sources
- Installs xenstore-utils
- Downloads some auto-configuration scripts
- Installs XenTools
- Installs the Ubuntu virtual kernel and removes the generic one
- Cleans up apt caches
- Shuts down the VM.
It is important that it shuts down at the end. My goal was to have it be all inclusive, and that means setting up the virtual kernel. However, it can't successfully reboot, because we need to update some PV boot options before it is ready to go.
Also, some packages must be installed in the post steps. The default install sources for Ubuntu don't always have all packages available, and I found it best to do the kernel last.
When you first bring up the new VM in XenServer, you may need to enter in a few details if you don't have DHCP running. It will self configure by default, but I normally opt to manually configure it to ensure the template gets a certain IP just to avoid any future possible collision.
After the bootstrapping is done and the VM is then off, log into console on the XenServer host itself and run the following snippet:
Before you run it, of course set the VMNAME to the name of your VM. You might
also want to check some of the values as you go (
echo $UUID). You can use the
xe-edit-bootloader command to view the grub configuration within the VM, but
need to set the PV settings outside the guest. By faking the
EDITOR to cat,
you can export the grub file to a local file, then use some grep+awk to get
the necessary pieces and finally set the PV settings correctly.
At this point, the machine is ready to be booted, converted into a template, or simply cloned. I personally like to leave my templates as never-been-used.
One of the main benefits of an IaaS setup like OpenStack or CloudStack is the self orientating of the VMs. Normally with a template, it keeps the template's setting for its hostname and network configuration. However, I found some example scripts on Github for passing network information into a VM through the VM's "xenstore" options. That way on boot, the VM can read in the settings it needs and update itself. I did some work to update the scripts to work with Precise, to more thoroughly update the hostname and DNS, as well as to regenerate the ssh host keys (since the kickstart did delete them).
As an example, this would be how to clone the template to a VM, set the params, and boot it:
That is cool and all, but I don't want to log into the XenServer console each
time I want to setup a VM. Luckily, found a plugin for Chef's
that adds XenServer provisioning. I
forked it and added setting xenstore network parameters
Now, provisioning a new VM, configuring it, and bootstrapping chef on it is just a matter of one call:
The command may be a little long, but it is all encapsulated in a single command. I simply have an internal README of sorts with the command pre-prepared for various roles.
If you have any questions, please leave a comment and ask. I'd be glad to help and always looking to improve my process as well.