In Valid Logic

S3 is a great place to host an APT repository... cheap storage, easily available, no server to run or anything. The annoying thing is that most ways to set it up involve using something like reprepro on a local box and using s3cmd to then sync the local files up to S3. For me, the annoyance is that I use throw away VMs. I don't want to store them, or I'll remove the VM without thinking, and then need to redownload everything to just add one one package.

I wanted just a simple tool that would upload the new package, generate the Package file adding in the new file, and update relevant hashes in the Release file. It seems simple, but could find something for the lazy people like me.

Finally decided to scratch my own itch. Over the weekend I wrote deb-s3. Have to upload a package? No problem.

$ deb-s3 upload my-deb-package-1.0.0_amd64.deb --bucket my-bucket
>> Examining package file my-deb-package-1.0.0_amd64.deb
>> Retrieving existing package manifest
>> Uploading package and new manifests to S3
   -- Transferring pool/m/my/my-deb-package-1.0.0_amd64.deb
   -- Transferring dists/stable/main/binary-amd64/Packages
   -- Transferring dists/stable/main/binary-amd64/Packages.gz
   -- Transferring dists/stable/Release
>> Update complete.

If it is your first package, it creates all the necessary files from scratch. On your server, you can then plug it in your /etc/apt/sources.list as:

deb https://my-bucket.s3.amazonaws.com stable main

It features:

• Specify different components or codenames as you wish. It'll maintain other components and architectures in the Release file.
• Options to specify the visibility of the file. Can set to private or authenticated, and then use apt-s3 to supported authenticated requests against S3.
• Ability to sign the Release file (GPG will prompt for the passphrase)

Check it out. Use it, love it, fork it, extend it, send a pull request.

If you simply want to use it, its available as a gem as well.

$ gem install deb-s3

And you're set.

Every language in existance has its own set of oddities. Since I've been working in Go full time for a couple of months now, I have run into some of its nuances and wanted to chronicle some of them.

While I am being critical in these kind of posts, the intent isn't to bash Go, more it is about educating others. Go isn't breaking existing convention (too much), however it is an emerging language and there isn't as much out there to familiarize a newbie with things to be aware of.

So to start off with, give you something simple but very annoying:

Strings cannot be null, only empty

In Go, null (or nil) isn't covered as heavily as I wish it was. Not every type is nilable, and this can lead to some annoyances.

All strings upon creation are simply an empty string (""). On the surface this doesn't sound bad, but it can cause a lot of other busy work when dealing with other things that allow string to be null, or where the difference between null and empty are very important.

Most databases have understood for a long time that a null string and an empty string are completely different. Another is with user supplied input. Go is excellent for writing servers and APIs, and a common case with an API is CRUD functionality. You might want to support a partial update, where omitted values (essentially null) or not altered while supplied values (which may be a blank string) are updated.

Take the case of user supplied input, such as over a JSON API:

// try at http://play.golang.org/p/1A7XZva4C1
package main

import (
  "encoding/json"
  "fmt"
)

type Person struct {
  Name string `json:"name"`
  Location string `json:"location"`
}

func main() {
  var p Person;
  json.Unmarshal([]byte(`{"name":"John"}`), &p)
  fmt.Printf("Name: %q\nLocation: %q\n", p.Name, p.Location)
}

Name: "John"
Location: ""

In this case I define a struct and unmarshal some JSON that only specified the name. But then you can see Location is set to "". If they already have Name and Location set, and are doing an update with only Name, I don't want to blank out Location. Now you got to do hoops.

Pick up that hula hoop... welcome casting

// try at http://play.golang.org/p/VlAJ4N9uGY
package main

import (
  "encoding/json"
  "fmt"
)

type Person struct {
  Name interface{} `json:"name"`
  Location interface{} `json:"location"`
}

func main() {
  var p Person;
  json.Unmarshal([]byte(`{"name":"John","location":"Gotham"}`), &p)

  fmt.Printf("Your zipcode is %d\n", lookupZipcode(p.Location.(string)))
}

func lookupZipcode(l string) int {
  switch l {
  case "Gotham": return 1
  case "Metropolis": return 2
  }
  return 3
}

In this mock example, passing the location to another function to look up the zip code, but it expects the location as a string, so now you need to cast it.

Uhh ohh, with interace{}, type enforcement isn't inherent

  json.Unmarshal([]byte(`{"name":"John","location":1234}`), &p)
  ...
  fmt.Printf("Your zipcode is %d\n", lookupZipcode(p.Location.(string)))

panic: interface conversion: interface is float64, not string

goroutine 1 [running]:
main.main()
  /tmpfs/gosandbox-9aac7f9a_0c33fe58_a998effb_2a4a973a_458fb2a3/prog.go:17 +0xcd

However, when using interface{} as our type we lose the inherit type handling within json.Unmarshal and this will result in a panic rather than returning an error. In this case, the panic is on the print line rather than when unmarshaling. So now we need to do our own type validation, which has annoyances of its own.

Type checking #1

  switch p.Location.(type) {
  case string:
    fmt.Printf("Your zipcode is %d\n", lookupZipcode(p.Location.(string)))
  default:
    fmt.Println("OMG you didn't enter the right value")
  }

Type checking #2

  if s, ok := p.Location.(string); ok {
    fmt.Printf("Your zipcode is %d\n", lookupZipcode(s))
  } else {
    fmt.Println("OMG you didn't enter the right value")
  }

For type checking you can either go the switch route or check the second parameter in the cast call. If you're morbid you could write your own typeof() using reflection perhaps.

For me, it is annoying because I need to actually do it and care. Perhaps I'm simply too spoiled by Ruby, however plenty of other languages support null strings as well. And so what if I am spoiled by Ruby... it focuses on developer happiness rather than hoops for performance. There are trade offs to all things, and I've accepted Go's empty strings. I just miss my null strings.

Earlier this week I had the pleasure of talking with Bryan and Nathan on the Food Fight Show, a devops focused podcast, talking about PaaS, Cloud Foundry, where PaaS is heading, and also talking about Go and the influence it will have on operations.

Be sure to check it out or look it up on iTunes.

It isn't really a secret that Apcera has been really looking at Go... just look follow me or Derek on Twitter. We think Go will be growing wildly in cloud services and high scale environments in the future. It is a very elegant language for server/systems development, has a solid base, growing ecosystem, and it will continue to grow and mature in the space. In some ways, I see it as solving the problems people were saying Node solved, but in a better way.

I'm starting to put together a few little mini posts about some of my adventures (or misadventures) with Go. I've been really starting to like it a lot, and there are some powerful things you can do with it that are encapsulated in a simple way. But it also has some idiosyncrasies, some of which are just ironing out as the language/ecosystem matures, and others are design decisions.

This morning I realized I never really updated my blog after changing jobs a few months ago. Back in June, I posted to the PaaS.io blog about joining Apcera, though never actually made a personal post about the change.

Earlier in the year, I had the opportunity to meet Derek Collison. Can read the PaaS.io blog post about why it made sense to merge the two. On a personal point of view, I was also drawn to the opportunity to get back to an early stage, highly motivated, build something fresh kind of startup.

My first profession gig right after college was at Telligent and I started when the company was just a month old. The early years at Telligent are amung the most memoriable of my career and a lot of that stemed from the energy at the company. A combination of the team transending to feeling like a family, building a great product out of nothing, and truly being energetic and dedicated to the goal. All of us ate, slept, and drank the product in those early years.

Since then, I've grown and learned a lot, however most of my work since has been on already established products and later-stage startups. Typically when you start needing Operations people, or people to do scaling/performance work, you already have something that is generating that need.

Working with Derek presented itself as a really unqiue chance. The team of people he was gathering together was truly awesome with strong backgrounds. It would be a tremendous chance ot learn from my peers. On top of that, he had the financial backing to be able to carry it through. He had plenty of experience from building Cloud Foundry and talking with clients, and his goals for Apcera's products were pushing the status quo and appealing as a challenge.

It was almost one of those "if I could do that again knowing what I know now" type chances. Do an early stage startup again, build something incredible from scratch. Two of the biggest motivators for me are fulfillment from building something with my own hands and tackling the kind of challenging problem that your first reaction is "yeah, I think I could do that" followed by you staring into space while starting to think about how.

It has been a littl over two and a half months at Apcera and it as been amazing so far. We're deep in code building what we've been envisioning. I am definitely eager to spill the beans, but until then I can only offer a slight preview.

To follow up on my previous post about disk errors with Ubuntu 12.04 on XenServer, I wanted to cover the process I've put in place for provisioning Ubuntu VMs.

With PaaS.io, I have a mixture between systems deployed on bare metal and virtualized. With the virtualized systems, I set out to make the provisioning as easy as if I was using an IaaS provider, while still giving me fine control over sizing and placement. Some of the decisions there are enough for another post.

Ubuntu 12.04 is the new hotness and I'd been anxiously awaiting it, with plans to progressively roll it out throughout PaaS.io.

With XenServer, it provides an easy template for provisioning Ubuntu Lucid 10.04 VMs and even for some newer releases. However, the way XenSever provisions Ubuntu is to essentially netboot it and install it from a remote source. Because of that, we can't simply use one of those templates buts install a newer release.

Fortunately, the template itself is very simple, and which release it installs is just a configuration parameter.

To create a new Ubuntu 12.04 template, simply log into the XenServer console and runt he following commands. It will clone the Lucid template and then change the parameter for the release to install from lucid to precise.

$ TEMPLATE_UUID=`xe template-list \
        name-label="Ubuntu Lucid Lynx 10.04 (64-bit)" params=uuid --minimal`
$ NEW_TEMPLATE_UUID=`xe vm-clone uuid=$TEMPLATE_UUID \
        new-name-label="Ubuntu Precise (64-bit)"`
$ xe template-param-set other-config:default_template=true \
        other-config:debian-release=precise uuid=$NEW_TEMPLATE_UUID

Now you will find a "Ubuntu Precise (64-bit)" option in the template list of XenCenter.

Now, we can actually provision a new box. Next, one interesting discovery was that you can pass in a kickstart script in the boot parameters options for the new VM.

Kickstart allows you to do a scripted installation, automated pretty much every aspect of the system. Instead of picking a bunch of options, it allows for an easy, repeatable process that basically leaves the machine completely ready to go.

To use a kickstart script, make a script available over HTTP somewhere on your LAN or on the public internet. By the time it is grabbed, the machine will have an IP, DNS, and all. Then, simply add ks=http://url/to/your/kickstart to the beginning part of "advanced OS boot parameters" option when selecting the installation media.

Below is a cleaned version of the kickstart script I used on my Ubuntu VMs. The main things of note:

• Sets up my partition table
• Configures base system with ubuntu-minimal as well as some common packages like openssh-server, curl, wget, and screen.
• Disables the creation of the ubuntu user (I create a normal every day user through chef)
• Configures the fstab with barrier=0 as mentioned before
• Disables /bin/sh from pointing to /bin/dash (personal preference)
• Updates apt sources
• Installs xenstore-utils
• Downloads some auto-configuration scripts
• Installs XenTools
• Installs the Ubuntu virtual kernel and removes the generic one
• Cleans up apt caches
• Shuts down the VM.

lang en_US
langsupport en_US
keyboard us
timezone America/Los_Angeles
text
install
skipx
halt
url --url http://us.archive.ubuntu.com/ubuntu

rootpw pa$$word   # you should replace, and use --iscrypted
auth --useshadow --enablemd5
user --disabled

bootloader --location=mbr
zerombr yes
clearpart --all --initlabel
part /boot --fstype=ext2 --size=64
part swap --size=1024
part / --fstype=ext4 --size=1 --grow

network --device=eth0 --bootproto=static --ip=10.0.0.50 --netmask=255.255.255.0 \
        --nameserver=10.0.0.1 --gateway=10.0.0.1
firewall --disabled

%packages
ubuntu-minimal
openssh-server
screen
curl
wget

%post

# update fstab for the root partition
perl -pi -e 's/(errors=remount-ro)/noatime,nodiratime,$1,barrier=0/' /etc/fstab

# point sh to bash instead of dash
rm /bin/sh
ln -s /bin/bash /bin/sh

# add normal apt source list
(
cat <<'EOP'
deb http://us.archive.ubuntu.com/ubuntu/ precise main restricted universe
deb http://us.archive.ubuntu.com/ubuntu/ precise-security main restricted universe
deb http://us.archive.ubuntu.com/ubuntu/ precise-updates main restricted universe
EOP
) > /etc/apt/sources.list
apt-get update
apt-get upgrade -y

# install some additional packages
apt-get install -y xenstore-utils

# set up xenserver automation scripts
AUTOMATER_REPO=https://raw.github.com/krobertson/xenserver-automater
curl $AUTOMATER_REPO/master/usr/sbin/xe-set-hostname > /usr/sbin/xe-set-hostname
curl $AUTOMATER_REPO/master/usr/sbin/xe-set-network > /usr/sbin/xe-set-network
curl $AUTOMATER_REPO/master/usr/sbin/generate-sshd-keys > /usr/sbin/generate-sshd-keys
curl $AUTOMATER_REPO/master/etc/init/xe-automator.conf > /etc/init/xe-automator.conf
chmod a+x /usr/sbin/xe-set-hostname
chmod a+x /usr/sbin/xe-set-network
chmod a+x /usr/sbin/generate-sshd-keys

# setup locales
locale-gen en_US.UTF-8
update-locale LANG="en_US.UTF-8"
echo 'LANG=en_US.UTF-8' >> /etc/environment
echo 'LC_ALL=en_US.UTF-8' >> /etc/environment

# install xe tools
cd /tmp
wget http://some/url/to/xe-guest-utilities_6.0.0-743_amd64.deb
dpkg -i xe-guest-utilities_6.0.0-743_amd64.deb

# install paravirt kernel image
apt-get install -f -y linux-virtual
dpkg -l | grep generic | grep linux | awk '{print $2}' | xargs apt-get remove -y

# clean up some stuff
rm -f /etc/ssh/ssh_host_*
rm -f /var/cache/apt/archives/*.deb
rm -f /var/cache/apt/*cache.bin
rm -f /var/lib/apt/lists/*_Packages

It is important that it shuts down at the end. My goal was to have it be all inclusive, and that means setting up the virtual kernel. However, it can't successfully reboot, because we need to update some PV boot options before it is ready to go.

Also, some packages must be installed in the post steps. The default install sources for Ubuntu don't always have all packages available, and I found it best to do the kernel last.

When you first bring up the new VM in XenServer, you may need to enter in a few details if you don't have DHCP running. It will self configure by default, but I normally opt to manually configure it to ensure the template gets a certain IP just to avoid any future possible collision.

After the bootstrapping is done and the VM is then off, log into console on the XenServer host itself and run the following snippet:

$ VMNAME=precise-20120501
$ UUID=`xe vm-list name-label="$VMNAME" params=uuid --minimal`
$ EDITOR=cat xe-edit-bootloader -n "$VMNAME" -p 1 \
        -f /grub/grub.cfg > /tmp/$VMNAME-grub
$ KERNEL=`grep vmlinuz /tmp/$VMNAME-grub | grep virtual |
        grep -v recovery | awk '{print $2}'`
$ ROOT=`grep vmlinuz /tmp/$VMNAME-grub | grep virtual |
        grep -v recovery | awk '{print $3}'`
$ RAMDISK=`grep initrd /tmp/$VMNAME-grub | head -1 | awk '{print $2}'`
$ xe vm-param-set uuid=$UUID PV-bootloader-args="--kernel=$KERNEL --ramdisk=$RAMDISK"
$ xe vm-param-set uuid=$UUID PV-args="$ROOT ro quiet console=hvc0"

Before you run it, of course set the VMNAME to the name of your VM. You might also want to check some of the values as you go (echo $UUID). You can use the xe-edit-bootloader command to view the grub configuration within the VM, but need to set the PV settings outside the guest. By faking the EDITOR to cat, you can export the grub file to a local file, then use some grep+awk to get the necessary pieces and finally set the PV settings correctly.

At this point, the machine is ready to be booted, converted into a template, or simply cloned. I personally like to leave my templates as never-been-used.

One of the main benefits of an IaaS setup like OpenStack or CloudStack is the self orientating of the VMs. Normally with a template, it keeps the template's setting for its hostname and network configuration. However, I found some example scripts on Github for passing network information into a VM through the VM's "xenstore" options. That way on boot, the VM can read in the settings it needs and update itself. I did some work to update the scripts to work with Precise, to more thoroughly update the hostname and DNS, as well as to regenerate the ssh host keys (since the kickstart did delete them).

As an example, this would be how to clone the template to a VM, set the params, and boot it:

$ UUID=`xe vm-install template=precise-20120501 new-name-label=builder`
$ xe vm-param-set uuid=$UUID xenstore-data:vm-data/ip=10.0.0.11
$ xe vm-param-set uuid=$UUID xenstore-data:vm-data/gw=10.0.0.1
$ xe vm-param-set uuid=$UUID xenstore-data:vm-data/nm=255.255.255.0
$ xe vm-param-set uuid=$UUID xenstore-data:vm-data/ns='10.0.0.1 10.0.0.2'
$ xe vm-start uuid=$UUID

That is cool and all, but I don't want to log into the XenServer console each time I want to setup a VM. Luckily, found a plugin for Chef's knife utility that adds XenServer provisioning. I forked it and added setting xenstore network parameters to it.

Now, provisioning a new VM, configuring it, and bootstrapping chef on it is just a matter of one call:

$ knife xenserver vm create --vm-template precise-20120501 \
                            -x root --keep-template-networks \
                            -r "role[foo]" -E production \
                            --vm-ip 10.0.0.12 --vm-name foobar

The command may be a little long, but it is all encapsulated in a single command. I simply have an internal README of sorts with the command pre-prepared for various roles.

If you have any questions, please leave a comment and ask. I'd be glad to help and always looking to improve my process as well.

Ubuntu 12.04 LTS Precise is hot off the press right now. Over the past few days, have been working on building new base images for PaaS.io, but was running into random issues where the root parition would encounter an error and freeze up. It normally happened just after it would finish booting and about 50% of the time.

A few times it happened after I already logged in, so was able to do basic read only operations. In dmesg, was able to see the following:

[    6.748868] blkfront: barrier: empty write xvda op failed
[    6.748876] blkfront: xvda: barrier or flush: disabled
[    6.748890] end_request: I/O error, dev xvda, sector 6584768
[    6.748908] end_request: I/O error, dev xvda, sector 6584768
[    6.748943] Aborting journal on device xvda6-8.
[    6.767022] EXT4-fs error (device xvda6): ext4_journal_start_sb:327: Detected aborted journal
[    6.767046] EXT4-fs (xvda6): Remounting filesystem read-only

Or if you weren't able to log in first, you might see this in the XenCenter console:

After some Googling, was able to track a similar error down by someone else. The fix was to update the mount options for the root partition. Mine are now:

noatime,nodiratime,errors=remount-ro,barrier=0

The key is the barrier=0. From some documentation, it is an option to help increase the integrity of writes by ensuring everything is flushed to disk be committing to the journal. However sometimes in a virtualized environment that is difficult to guarantee. In my case, have disk->RAID->dom0->LVM->domU.

Figure many other people will be diving into Precise this weekend, potentially running into this issue like me.

Soon, will post some additional details about how to easily get a nice Precise template in XenServer 6. I've been getting my setup nicely tuned using a kickstart script for the base system and leveraging xenstore data to dynamically setup the IP and hostname on boot.

While working on developing PaaS.io, one of my primary objectives is to ensure that everything is configured to be truly Highly Available (HA). This means everything has a secondary, replicated, and has automated failover. This way it is nicely resilent and fault tolerant. Core infrastructure changes can be made without any affect to running applications, servers go down without issues. Sweet, huh?

When it came to setting up MongoDB, the obvious option was to go to using Replica Sets. They're like master-slave replication in RDBMSes but the master node is essentially floating. Client applications connect to the whole cluster, identify the master, and then start to speak to it.

By default, Cloud Foundry provisions a dedicated MongoDB instance for you and provides you the IP, port, and other credentials. This doesn't include replication, and focuses on single IP connections rather than replica sets.

However when using Replica Sets, the connection pattern in your code is slightly different. You use a ReplSetConnection instead of a normal Connection, in ruby driver speak. So you need to know you're connecting to a Replica Set if the host you're connecting to isn't the master. If you only connect to the master though, you can use a traditional connection.

I was interested in maintainin full compatibility with Cloud Foundry's out-of-the-box experience, so first looked at using mongos in front of replicas. It is normally used to front a sharded setup, but using it without sharding is still a feature request.

So I set out looking at how to use a virtual IP have it track which node is the master. In this case, if one switches over, it will pick it up and move the virtual IP to follow. This would be mostly transparent to the application. The only handling in the app is the next query will fail, but quickly reconnect and its fine.

I'd already been using keepalived for HA within HAProxy, and one of the nice things it provides is the ability to define a script to run as a part of its local health checks. Using this, can have a script that just asks the local system "you the master?" and returns appropriately.

Below is the script:

#!/usr/bin/env ruby

require 'optparse'

options = { :hostname => '127.0.0.1',
            :port     => 27017,
            :username => nil,
            :password => nil }

optparser = OptionParser.new do |opt|
  opt.banner = "Usage: #{$0} [options]"
  opt.on('-H', '--hostname HOST', 'Hostname') { |o| options[:hostname] = o if o }
  opt.on('-P', '--port PORT', 'Hostname') { |o| options[:port] = o.to_i if o }
  opt.on('-u', '--username USER', 'Username') { |o| options[:username] = o if o }
  opt.on('-p', '--password PASSWORD', 'Password') {|o| options[:password] = o if o }
end

begin
  optparser.parse!
rescue => e
  $stderr.print e
  $stderr.print optparser
  exit 1
end

require 'rubygems'
require 'mongo'

conn = Mongo::Connection.new(options[:hostname], options[:port])
conn.db('admin').authenticate(options[:username], options[:password])
master_status = conn.db('admin').command({'isMaster'=>1})

if master_status['ismaster']
  exit 0
else
  exit 1
end

This won't return any output, but will exit with 0 if it is the master or 1 if it isn't.

Its requirements are simple... Ruby, RubyGems, mongo gem, and recommend the bson_ext gem too.

Then within our keepalived.conf file, it looks like this:

global_defs {
   notification_email {
     me@example.com
   }
   notification_email_from system@example.com
   smtp_server 192.168.1.1
   smtp_connect_timeout 30
}

# Define the script used to check if mongod is running
vrrp_script chk_mongod {
    script "killall -0 mongod"
    interval 2 # every two seconds
    weight 2
}

# Define the script to see if the local node is the primary
vrrp_script chk_mongo_primary {
    script "/usr/local/mongodb/bin/mongo_check_primary -u admin -p password"
    interval 2 # every two seconds
    weight 2
}

# Configuation for the virtual interface
vrrp_instance VI_1 {
    interface bond0
    state node MASTER        # SLAVE on the other nodes
    priority 101             # 100 on other nodes
    virtual_router_id 55

    authentication {
        auth_type AH
        auth_pass secret     # Set this to some secret phrase
    }

    # The virtual ip address shared between the two nodes
    virtual_ipaddress {
        192.168.1.222
    }

    # Use the script above to check if we should fail over
    track_script {
        chk_mongod
        chk_mongo_primary
    }
}

With this in place, if a server goes down, it'll switch. If mongod dies, it will switch over (since Mongo itself will recognize that), and if the current primary is switched, the mongo_check_primary will return an exit code of 1, causing it to switch over.

Another reason this might be useful in some cases is because some clients still don't support replica sets. With this method, they don't need to.

In addition to these steps, you will need to perform the normal steps to setup keepalived. These would include:

• Ensure mongod is bound to 0.0.0.0 so it'll accept any incoming connection.
• Set net.ipv4.ip_nonlocal_bind=1 in sysctl so you can use virtual IPs.

Try this out and let me know your experiences. If you're interested in this kind of seamless infrastructure automation, check out PaaS.io or drop me a line.

Recently I moved my blog over to the service I am currently working on building, PaaS.io.

Running Jekyll on PaaS.io isn't all that different from running it on other services, though I had a few other goals in mind. The way I wanted it set up was:

• Stop using rack-jekyll. Its a nice gem, however it is locked to an older version of Jekyll. And the current gem is on an even more outdated one. Currently, Cloud Foundry doesn't support pulling bundler sources from git too.
• Have a public folder for static content like CSS and images.
• Have the _site folder for generated content
• Don't have it copy the Gemfile and the config.ru into the _site folder (annoys me)
• Redirect www.invalidlogic.com to invalidlogic.com
• Low foot print

First, the Gemfile:

source :rubygems

gem 'rack-contrib', :require => 'rack/contrib/try_static'
gem 'rack-redirect'
gem 'thin'

group :development do
  gem 'jekyll'
  gem 'RedCloth'
  gem 'rdiscount'
end

The main gems being used are thin, rack-contrib (for TryStatic, note on that later), and rack-redirect (for www redirection). I also include some of the gems I use for Jekyll in the development group. That way they are available locally but not loaded when I deploy (lower footprint... and yes, it is minor).

Now, the config.ru:

require 'rubygems'
require 'bundler'
Bundler.require

use Rack::EY::Solo::DomainRedirect

use Rack::TryStatic,
    :root => "_site",
    :urls => %w[/],
    :try => ['.html', 'index.html', '/index.html']

use Rack::Static,
    :root => "public",
    :urls => %w[/]

run lambda { [404, {'Content-Type' => 'text/html'}, ['Not Found']]}

There are 4 rack components here. First, Rack::EY::Solo::DomainRedirect is the rack-redirect gem and handles the www redirection. Next, is Rack::TryStatic. It is used to access files from the _site generated content directory. It gives a couple different :try values for different ways to find the intended file. Then is the Rack::Static which gets static content from the public directory. No need to try different combinations. And last is a generic lambda that will return 404.

Next, want to avoid duplication. With things as they are, when I run jekyll it will copy the public and other items into the _site folder duplicating it. To resolve that, in our _config.yml, can add an exclude line:

exclude: [ 'public', 'Gemfile', 'Gemfile.lock', 'config.ru' ]

And with that, we are set! All of our goals are met. Commit and push to deploy! Currently I have Cloud Foundry set up with a Rack framework defined (will be sending a pull request with it soon) and also have my blog set to use Ruby 1.9.3 as well.

Soon I'll be providing some more details on PaaS.io, so stay tuned and click over to it and sign up to get access to the beta.