If you are in the Sacramento area next Thursday, come by the SacRuby Meetup! I will be presenting on using Chef for server and configuration management. I will be talking about using Chef on a large scale with a cluster of servers, all the way down to aiding management of a single server.

If you are in the area, please stop by! Pizza and refreshments are normally available.

Meetup details: http://www.meetup.com/sacruby/calendar/14122667/
Date: Thursday, August 19th
Time: 7:30pm til we go home
Location: 10911 White Rock Road, Rancho Cordova, CA 95670 (see details for number to call to get building access)

The main talk starts at 7:30, however we will also have a Ruby Clinic starting at 6:30. Bring your questions, problems, or other issues in and we’ll try to help you out.

One thing we often run into without deploys is “what server is that executing on?” After updating code, we actually a fairly elaborate process to remove servers from load balancing, restart, wait for load to drop off, then add it back in to load balancing. We also just use the AWS hostnames rather than our own since our server list pretty much changes weekly. Wouldn’t it be nice to have the ouptut show “app10″ instead of “ec2-123-123-123-123.compute-1.amazonaws.com”.

Luckily adding an alias to Capistrano is pretty easy!

class Capistrano::ServerDefinition
  def to_s
    @to_s ||= begin
      s = @options[:alias] || host
      s = "#{user}@#{s}" if user
      s = "#{s}:#{port}" if port && port != 22
      s
    end
  end
end

Simply add that to your deploy.rb, or another file that is included with your recipes for utility methods. It works simply by overriding the ServerDefinition.to_s method, which is used for the logger output. The only change to the default method is line 4.

Then for your role definitions:

role :app, 'ec2-123-123-123-123.compute-1.amazonaws.com', :alias => 'app10'

Then you see much better output:

 ** [out :: app10] Alive

Yet another New Relic goodie today.

One thing I was kind of surprised that New Relic didn’t already have instrumentation around out of the box was controller filters. We had noticed some occasional gaps after the main processing of an action, where all the templates would be rendered, but the execution still went on for a few hundred milliseconds (or sometimes more). Finally, it dawned that it is probably one of the after_filters.

So we aded tracing around filter execution, and lo and behold, we had one that from time to time would misbehave and execute slow.

To instrument the filters, the simplest way seemed to be by tacking on to the ‘call’ method of each filter type.

::ActionController::Filters::BeforeFilter.class_eval do
  add_method_tracer :call, 'Custom/BeforeFilter/#{method.to_s}'
end

::ActionController::Filters::AfterFilter.class_eval do
  add_method_tracer :call, 'Custom/AfterFilter/#{method.to_s}'
end

::ActionController::Filters::AroundFilter.class_eval do
  add_method_tracer :call, 'Custom/AroundFilter/#{method.to_s}'
end

And one more goodie while I’m looking at my list of New Relic tracers… ActionMailer. Have seen this time and time again, that sending email synchronously with the HTTP request can drastically slow down performance. SMTP server can be slow, can be mailing a number of recipients, submission fails, etc. For us, most email delivery is asynchronous in the background, but one or two are triggered in line.

To trace ActionMailer delivery:

::ActionMailer::Base.class_eval do
  add_method_tracer :deliver!, 'Custom/ActionMailer/deliver!'
end

Will maybe sometime look at email processing more, but email templates are often far simpler and their action methods do less loading.

At Involver, we’ve been starting to make heavy use of New Relic to start monitoring performance and track down bottlenecks. I’ve been working to add method tracers around components that utilize external APIs or services, and some of the major components we leverage to know how much time is spent in various areas.

We use Cache Money, a gem that helps transparently cache models in Memcached. Out of the box, New Relic supports tracing calls for Memcached, however on traces we’d see a long list of Memcached calls but no information as to which request they were associated with.

For a lot of our additional traces, we’ve added an rpm_extras.rb file. To include tracing for Cache Money, just need to define it as follows:

::ActiveRecord::Base.class_eval do
  class << self
    add_method_tracer :transaction, 'Custom/CacheMoney/transaction'
    add_method_tracer :find_every, 'Custom/CacheMoney/find_every'
    add_method_tracer :find_from_ids, 'Custom/CacheMoney/find_from_ids'
  end
end

New Relic itself uses add_method_tracer around transaction and find_by_sql, however Cache Money works by alias_method_chaining transaction, find_every, and find_from_ids.

One thing worth noting is that since both New Relic and Cache Money make use of transaction, order of loading could be important. In our environment.rb, we load our rpm_extras.rb in the after_initialize block. When RPM is loaded, it wraps the actual ActiveRecord#transaction call, then Cache Money chains it, and our own tracer wraps the chained call.

Initially, I tried putting the tracer around Cache Money’s own transaction_with_cache_transaction, however that didn’t show up in traces.

Last week, I gave a presentation at the SacRuby meetup on using Twilio with Sinatra.

Someone, I believe it was Jeremy Evans (author of the Sequel ORM) asked me why I was using exclusive returns at the end of my methods (code in question). It had never occurred to me and I had just answered with no reason.

While going over the video of the presentation, came across his question again and was thinking about it. I realized it is still a .NET-ism that I’m carrying. In Ruby, you don’t need to explicitly return at the end of a method. The result of the last command will automatically be returned. However in .NET, everything has to be an explicit return.

When working with multiple languages, it is interesting to see the idioms cross over. There have been multiple times I’ve been in C# and started typing “return … if …” or something similar. Much the same, I’m so used to explicit returns in C# that I’ve used them in Ruby where I didn’t really need to.

It is important to be aware of the “isms” that influence you. When moving into a new language, often times they are visible to others more familiar with the language, and more importantly, can shield you from other techniques the language offers. While many are innocent, awareness is important to help you continue to grow.

However when “isms” cross back to a language you already know, they offer a chance to gain new perspective on methodologies you already practice and a might offer an opportunity to pose the question “how can I make this better”.

I’ve talked about the Papercut tool I’ve written before. I initially wrote it to scratch an itch I had with previewing and testing emails during the development process.

Recently though while working on Trunks, I’ve been feeling the need for an email testing tool for staging environments. Since Trunks involves several moving components, I have a staging environment set up so I can do live testing on features. Often times that testing involves emails. Since I run the staging environment with a snapshot of the live database, I want the emails to be sent and viewed for testing, but don’t want them going out to live accounts. Would hate for a user to mistakenly receive an email from my staging environment!

In comes Papercut Web Edition.

Papercut Web Edition provides you with an SMTP account that you can connect to, authenticate with, and send emails two. It works just like the desktop Papercut where messages sent to it are only stored locally, not sent on to their destination. It then allows you to view the messages through the website.

With this, when you set up a staging environment, you simply point it to the SMTP server and off you go. In something like Telligent Community, just a matter of quickly checking the Email Settings, or with a Rails app, just need to edit your “staging” environment configuration.

The service is provided free of charge. It is simply ad supported, so while using it you will see an AdSense banner on the side. I’ve tried to make it unobtrusive while at the same time striving for the service to generate something in return. I may be changing it to use some different ads if it gets some sponsors, as I don’t expect AdSense to really draw in a whole lot.

Currently, it will delete messages after they are 2 hours old. It is intended for testing emails, not retaining them, so the idea is you trigger the email, view it online, and then are done with it… or simply proving a way for emails to be safely sent while you test other aspects of your app.

I have several more features that will be coming down the road. Right now, it is rather simplistic as I created it quickly to solve my problem. But it has some big things coming to help tackle some other emails testing annoyances I’ve felt. It’ll also have support for attachments, embedded images, forwarding messages onto a live account, and much much more.

So the most important piece? The URL!

dummysmtp.com

All good names with Papercut in them were taken… and as a site URL, “papercut” has more connotations with paper and pain than email. Dummysmtp is very descriptive of what it does… may the SEO gods look fairly upon it.

Sign up today and try it out! Go forth and test emails… safely!

Have been working on a new take on Papercut (details coming soon) when I noticed that there were several issues opened in Codeplex and a number of patches submitted.  I’d forgotten to enable email notifications!  So decided to sit down and felt some of them merited a new release.  Release notes below.

Download

This release includes a few bug fixes and updates, several of which were contributed patches (thanks!).

• Feature: Added support for embedded images (patch submitted by jasonpenny)
• Enhancement: Show an alert when trying to use port that is already in use (Issue #5405)
• Enhancement: Don’t show the options dialog in the taskbar (patch submitted by dukesb11)
• Fix: Updates to fix issues with multipart messages (Issue #4959, patched submitted by jasonpenny)
• Fix: Issue with async networking and not handling a disposed object (patch submitted by jasonpenny)
• Fix: Out dated URL references (Issue #5457, patch submitted by billrob458)

Tonight I realized that its now been two years since I first started working with Ruby. Back in February of 2008, I’d posted about my first experiments with Rails.  Lo and behold, its now been two years.  I do say two years with Ruby though, not Rails.  I’ve been using Merb for about the past year and a half.  Although I have started messing around with Rails 3 and really like what I see so far.

So in the past two years, what have I done?  Sadly, a lot of it is sorely behind and out dated, but take a bit of a trip down memory lane…

• The SkitchDav I originally blogged about, but now no longer use
• DM-DBSlayer. I was using it in a personal app, but the app didn’t need scale, so eventually removed it.
• DM-Paperclip.  I really need to dust this off, it needs some TLC.
• Some hacking on Warehouse, which eventually led to my ideas for Trunks.
• Branches, which was basically a ruby port of Gitosis.  I wrote it anticipating using it for Trunks so I could interact with the DB in Ruby.  I did in the first incarnation of trunks, but am now on “Branches 2.0″ which was done from scratch and super simplified so it is Trunks-only and structured to do both Git and Mercurial.  Would actually be nice to pull it out of Trunks and release it.  Damn, another project for my list.
• Tigger.  Really bad name.  When I first wrote Trunks but was unsure about launching it, tried to redo the interface as a better read-only web interface to git repos.  The app should still work… was on Merb 1.0.
• And of course, Trunks.  Merb 1.1-pre.

And I’ve also had several apps that haven’t seen the light of day and were more for my personal use or tinkering.

• For a while I was into EVE Online and wrote an app to parse and process item reports from Eve Central.  It allowed me to locate items that were selling under market rate quickly, compare prices across markets, and I could somewhat defer what items were selling most at different market hubs.  Really geeky.  Probably put several months into it.  Merb app.
• Also wrote an app to track my sales by location, by item, and comparing sales volumes based on price as I played with prices.  Merb app.
• And another for EVE Online… came up with the “Jita Mineral Index” which tracked mineral prices in the main market hub in the game.  I left it up for several months after I stopped playing, until I found that the app that processed the emails from Eve Central broke and had like 200k+ unprocessed messages in its inbox.  Merb app.
• Some time ago, was playing with different virtualization APIs and wrote an app that would provision and deploy virtual servers in KVM and in XenServer.  I’m probably too much of a nut about virtualization.  Mixture of stuff… Ruby, Nanite, Merb.
• A really basic Sinatra app to handle my domains that just have static pages.  Rather than having multiple sites, I have just one and it determines which template to render based on the host header.  Was my first Sinatra app.

And of course some other stuff on Github and Gist.

And I have some that will likely be coming soon:

• This, which I’ll likely blog about once I get a name for it and a domain.  And update the incorrect launch date in the News part, oops.
• Was actually thinking of dusting off my original SkitchDav as a Sinatra app.  What triggered me to write this.
• Also thinking of dusting off an app I prototyped a while ago involving Rack and Solr.
• Always got ideas brewing in my head… just a matter of parsing out which are really worth my free time.

All this Ruby has also helped me to grow so much more as a .NET developer.

• More languages you know the better.
• Exposure to new methodologies and mindsets, new more progressive community.
• Exposure to testing.  Testing is far easier in dynamic languages, and grasping the concepts in a dynamic language makes it easier to apply to a static language.
• MVC.  Thank god for MVC.  Unlearned all the webforms crap and re-learned plain and simple HTTP and a logical structure for your app.
• Making programming fun again.  Its nothing against .NET, but sometimes you need to break out from the familiar and learn something new to remind you why you love programming.  Before Ruby, I was stuck in what I knew and don’t really think I was coding for fun much.

I have some of my own datacenter space with a couple of servers that I use for a little bit of hosting that I do.  This weekend, I’m finishing a migration and having done this type of thing a couple of times, thought I’d share a few tips.

First of all, one thing that is very nice to do, if you have a presence in both places at the same time, is to configure IP forwarding from your old site to your new site.  While it is a best practice to set the TTL on your DNS records really low (mine are currently 60 seconds), ISPs don’t always honor the TTLs and some will be more aggressive with caching.  As a rule, its usually 8-24 hours before you can really count on old caches being expired.

To combat this, forwarding any traffic that might be going to the old addresses to the new ones can be very helpful, and fairly easy to do.

There can be several ways to do this, but can often times end up more complicated.  The pure and simple way to do it is to use an application called rinetd, which is basically an “internet redirection server”.  You simple edit its configuration file, specify the old IP address and port, and the new IP address and port.  Run it and you’re good!

I run ESXi on some of my systems, so I simply created a new virtual machine with a very basic configuration and installed Ubuntu on it.  I gave it a base IP that I wasn’t going to be forwarding, installed SSH on it (to remotely manage it), and then went to town setting up what I needed forwarded.  First, I added the IPs to be forwarded as aliases in /etc/network/interfaces:

auto eth0:0
iface eth0:0 inet static
     address 192.168.1.111
     netmask 255.255.255.0

Now, I need rinetd. On Ubuntu, it is readily available as a package, so just run sudo apt-get install rinetd and you’re set. Edit the /etc/rinetd.conf to list out how you want things forwarded:

# old ip         old port    new ip          new port
192.168.1.111    80          172.16.1.222    80

Afterwards, all you need to do is restart networking and rinetd. So run sudo /etc/init.d/networking restart and sudo /etc/init.d/rinetd restart.

Few things worth noting. rinetd binds to the address/port you specify, so if you are trying to forward port 80, you can’t have something else bound to port 80 on that IP, such as a webserver. So if you need to forward something that you might have in use, you may have to tweak things. If you need to forward SSH, it might be best to configure SSH to use another port for a little while. Second, I’d sometimes just reboot the virtual machine instead of restarting networking. I’d migrate some stuff one night and have it forward for 24 hours, then next night migrate something else and have it forwarded. It’d sometimes get a little hairy adding/removing/reconfiguring aliases a lot, so I’d just reboot.

Another plus with rinetd is that a port of it exists for Windows! A download exists on its site and it is also possible to configure it as a Windows service using ServiceEx. I haven’t tried it myself, but I did find some pretty detailed instructions about configuring it on Windows.

Was recently working on an issue that was so perplexing, that I almost had David Penton, our resident SQL expert stumped.

We had an issue with some username handling and it ended up boiling down to there we could go into SQL Server, run this query, and get a result when you’d normally expect it to not match:

select 1 where N'ß' = N'ss'

After that, I was truly stumped. I’d Googled everything I could think of any came up empty. Had I known what the German sharp S was, life would have been easier. Come to find out through some sources, that some of the SQL standards (in this case, SQL92 is referenced) call for the German sharp S is translated down to “ss” in string comparisons. So although it might seem like a bug, it is to-spec. Oracle is also reported as working this way.

To work around the behavior, one option is a workaround function for SQL Server which essentially has you convert the string to a binary column and store it. This may be outside what you can do in your application though, as in ours, we’d need to update every place that does string comparisons on the username as well as altering the ASP.NET Membership tables and stored procedures.

An alternative is to change your database collation to use one that is more binary dependent such as Latin1_General_BIN. This will treat ß and ‘ss’ separately, though also be aware since it is binary, it is case insensitive and some behaviors such as sorting may also change.

A topic like this is important for any developer for a few reasons:

First, being in the US, it is easy to be ignorant of other culture’s language handling. You may work to add resource labels so your application can be translated and such, but every so often you will find behaviors related to internalization that perplex you because you’ve never heard of the German sharp S (as in this case).

Second, when its an issue in SQL and you’re in doubt, look at collation first. SQL Server has a huge list of collations, and each one does something different. Its either a behavior with the one you’re using, or the user is using a collation you haven’t tried (tested) before.

And finally… need to try harder to stump Penton. It can be done!