matthew manning

Cloud Goodness With the DNSimple ALIAS Record

matt.manning@gmail.com (Matt Manning) — Wed, 18 Jan 2012 00:00:00 +0000

This is the third in a series of posts about setting up this Jekyll-backed blog on Heroku.

Cloud computing and second-level domains a.k.a. root domains (example.com) don’t play nicely together. This is because of RFC 1034 section 3.6.2: “If a CNAME RR is present at a node, no other data should be present; this ensures that the data for a canonical name and its aliases cannot be different.” Since root domains tend to have several types of records – “other data” – like NS or MX records, you can’t use a CNAME for them. That means you’re left with A records, which must specify IP addresses. Hence the rub with cloud computing, which tries to abstract away servers and IP addresses.

Heroku deals with root domains by providing 3 public IP addresses for routing web requests. For SSL, the extremely cloud-unfriendly IP-based SSL add-on is used, which provisions a dedicated server instance to serve your certificate. While these solutions work, they’re not really optimal. For example, if Heroku ever changes those IP addresses, as would be necessary in the case of a large-scale DDoS attack, or a datacenter failure, your root domain would no longer resolve. The IP-based SSL add-on has these same downsides, plus it’s a scaling bottleneck and costs $100/mo. Bummer.

DNSimple has introduced a really clever mechanism for dealing with this problem, the ALIAS record. An ALIAS record is set up much like a CNAME, but it resolves the name it points to and returns its IP addresses as if you had set up A records for them. This provides the flexibility of a CNAME without violating the RFC. You can see it working for this site using the host command:

[~/code/blog] host mwmanning.com
mwmanning.com has address 107.20.209.232
mwmanning.com has address 107.22.180.220
mwmanning.com has address 107.20.154.48
mwmanning.com has address 107.22.181.229
mwmanning.com has address 107.22.173.156
mwmanning.com has address 50.17.208.142
mwmanning.com has address 107.22.180.255
mwmanning.com has address 107.22.178.183

Besides the routing flexibility afforded by using an ALIAS record, you can take advantage of Heroku’s new HTTP stack, which is only available at [appname].herokuapp.com, and not at the 3 IPs mentioned previously. You can also use an ALIAS record to provide SSL at your root domain without using the IP-SSL add-on. If you use the Endpoint SSL add-on (available in the Beta program), you can actually have SSL at your root domain for free!

Jekyll on Heroku Part 2: Rack 'Em

matt.manning@gmail.com (Matt Manning) — Sun, 04 Dec 2011 00:00:00 +0000

This is a follow up to my previous post, Run Your Jekyll Site On Heroku.

Ditching rack-jekyll

The previous post we got going as a Rack app using rack-jekyll. While rack-jekyll is a good project, it does some checking that isn’t really necessary since we’re precompiling the site and don’t need to build anything at runtime.

I first tried replacing rack-jekyll with Rack::Static, but it only works with explicit filenames. For example, with rack static the URL http://mwmanning.com/index.html would work fine, but http://mwmanning.com would return a 404. Luckily I found this blog post by Chris Continanza which clued me in to Rack::TryStatic, which is part of rack-contrib.

Making this switch is pretty easy. Just remove the rack-jekyll gem and add the rack-contrib gem, so your Gemfile looks like this:

source :rubygems

gem 'jekyll'
gem 'rack-contrib'
gem 'RedCloth'
gem 'thin'

Then change your config.ru file to use Rack::TryStatic. The result should look something like this:

require 'rack/contrib/try_static'

use Rack::TryStatic,
    :root => "_site",
    :urls => %w[/],
    :try => ['.html', 'index.html', '/index.html']

run lambda { [404, {'Content-Type' => 'text/html'}, ['Not Found']]}

Rewriting URLs

One of the reasons I really wanted Rack was for rewriting URLs. That’s something the Jekyll server can’t do. My old blog was a Wordpress site hosted by A Small Orange, and its URL scheme was slightly different. I had a few links and some Google results pointing to my old URLs, and I didn’t want to throw all of that away, so I turned to Rack::Rewrite. With Rack:Rewrite you get a little DSL that lets you specify routes that are either rewritten or issue redirects. Regular expressions are available for route matching, which suited my needs perfectly for supporting the old blog URL scheme.

Since I don’t plan to go back to the old scheme from the Wordpress blog, I dedided to use 301 - Moved Permanently redirects.

I added the gem to my Gemfile:

source :rubygems

gem 'jekyll'
gem 'rack-contrib'
gem 'rack-rewrite'
gem 'RedCloth'
gem 'thin'

and set up the appropriate redirects in my config.ru:

require 'rack/contrib/try_static'
require 'rack/rewrite'

# Support links to old Wordpress site
use Rack::Rewrite do
  r301 '/2010/11/29/ec2-micro-instance-as-a-remote-bittorrent-client/', '/2010/11/29/EC2-Micro-Instance-as-a-Remote-Bittorrent-Client.html'
  r301 '/2009/01/13/timemachine-ubuntu-nas/', '/2009/01/13/TimeMachine-Ubuntu-NAS.html'
  r301 '/projects/jack-o-led/', '/projects/jack-o-LED.html'
  r301 %r{/projects/(\S+)/}, '/projects/$1.html'
end

use Rack::TryStatic,
    :root => "_site",
    :urls => %w[/],
    :try => ['.html', 'index.html', '/index.html']

run lambda { [404, {'Content-Type' => 'text/html'}, ['Not Found']]}

Caching

The final piece I wanted for my Rack app was caching. Older Heroku stacks (Aspen and Bamboo, which only support Ruby apps) use Varnish and nginx in front of the app server and can be blazing fast. The Cedar stack does not, because they conflict with some advanced HTTP features it supports.

Luckily, Heroku offers a hosted memcache add-on which can be used with the Rack::Cache middleware. The free version of the add-on gives you 5MB of memory, which is pretty good for my purposes since my whole app (gems included) is only a little over 7MB. To get this working I first added the add-on with this command:

heroku addons:add memcache:5mb

Next, I needed a couple of gems, dalli and rack-cache. With those installed my Gemfile looked like this:

source :rubygems

gem 'dalli'
gem 'jekyll'
gem 'rack-cache'
gem 'rack-contrib'
gem 'rack-rewrite'
gem 'RedCloth'
gem 'thin'

The last bit was to set up Rack::Cache in the config.ru file.

require 'dalli'
require 'rack/cache'
require 'rack/contrib/try_static'
require 'rack/rewrite'

$cache = Dalli::Client.new
use Rack::Cache,
  :verbose => true,
  :metastore => $cache,
  :entitystore => $cache

# Support links to old Wordpress site
use Rack::Rewrite do
  r301 '/2010/11/29/ec2-micro-instance-as-a-remote-bittorrent-client/', '/2010/11/29/EC2-Micro-Instance-as-a-Remote-Bittorrent-Client.html'
  r301 '/2009/01/13/timemachine-ubuntu-nas/', '/2009/01/13/TimeMachine-Ubuntu-NAS.html'
  r301 '/projects/jack-o-led/', '/projects/jack-o-LED.html'
  r301 %r{/projects/(\S+)/}, '/projects/$1.html'
end

use Rack::TryStatic,
    :root => "_site",
    :urls => %w[/],
    :try => ['.html', 'index.html', '/index.html']

run lambda { [404, {'Content-Type' => 'text/html'}, ['Not Found']]}

I set this up without really thinking too much about it. Caching Is Good™, right? Well I decided to do a quick test with ApacheBench to see what kind of average response times I got. (If you’re having trouble with ApacheBench on OS X Lion, here’s how to fix it). First I wanted to get a baseline number without caching, so I removed that part from config.ru and ran this test on one dyno.

[~/code/blog] ab -n 1000 -c 5 http://www.mwmanning.com/2010/11/29/EC2-Micro-Instance-as-a-Remote-Bittorrent-Client.html
This is ApacheBench, Version 2.3 <$Revision: 1178079 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking www.mwmanning.com (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:        thin
Server Hostname:        www.mwmanning.com
Server Port:            80

Document Path:          /2010/11/29/EC2-Micro-Instance-as-a-Remote-Bittorrent-Client.html
Document Length:        11274 bytes

Concurrency Level:      5
Time taken for tests:   42.249 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      11451000 bytes
HTML transferred:       11274000 bytes
Requests per second:    23.67 [#/sec] (mean)
Time per request:       211.247 [ms] (mean)
Time per request:       42.249 [ms] (mean, across all concurrent requests)
Transfer rate:          264.68 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       29   43  12.6     39     117
Processing:    75  168 127.0    140    1870
Waiting:       32   59  27.4     53     422
Total:        109  211 127.5    184    1913

Percentage of the requests served within a certain time (ms)
  50%    184
  66%    199
  75%    209
  80%    218
  90%    255
  95%    310
  98%    582
  99%    853
 100%   1913 (longest request)

Not bad. The average response time was about 211ms, with 2/3 of requests taking less than 200ms. Next I turned caching back on and ran the same test.

[~/code/blog] ab -n 1000 -c 5 http://www.mwmanning.com/2010/11/29/EC2-Micro-Instance-as-a-Remote-Bittorrent-Client.html
This is ApacheBench, Version 2.3 <$Revision: 1178079 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking www.mwmanning.com (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:        thin
Server Hostname:        www.mwmanning.com
Server Port:            80

Document Path:          /2010/11/29/EC2-Micro-Instance-as-a-Remote-Bittorrent-Client.html
Document Length:        11274 bytes

Concurrency Level:      5
Time taken for tests:   49.630 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      11593000 bytes
HTML transferred:       11274000 bytes
Requests per second:    20.15 [#/sec] (mean)
Time per request:       248.152 [ms] (mean)
Time per request:       49.630 [ms] (mean, across all concurrent requests)
Transfer rate:          228.11 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       28   43  60.9     35    1142
Processing:    76  205 262.7    142    3830
Waiting:       41   77 140.4     59    2418
Total:        110  248 285.4    183    3932

Percentage of the requests served within a certain time (ms)
  50%    183
  66%    199
  75%    214
  80%    228
  90%    392
  95%    548
  98%    970
  99%   1279
 100%   3932 (longest request)

What’s this? Not only were the caching results not any better, the averages were slightly worse. This was a bit of a facepalm moment. The main purpose of caching is to prevent calculations or database transactions being made by the web app backend, but in this case the web app backend is doing very little other than serving static files off of disk. Plus the Memcached instance being used here is network-attached, so there’s some unavoidable latency in that connection. In this case serving files off of disk is faster than serving out of memory over a network connection, so I disabled caching. Hooray for the scientific method!

Run Your Jekyll Site On Heroku

matt.manning@gmail.com (Matt Manning) — Tue, 29 Nov 2011 00:00:00 +0000

A few months back I decided I wanted to blog like a hacker, so I started revamping my old Wordpress blog into a more minimal site generated by Jekyll. You’re reading the result of that work right now.

I was happy with how easy Jekyll was to set up and use, and it’s a relief to have my post archives as markdown files under Git version control instead of in a proprietary blog engine format in some database.

One of the cool things about a Jekyll site is that you can push it to Github, and Github will serve it. Not just the source, but the actual site itself. You can see it working with this site. Here is the source, and here it is running. While this is pretty cool, Github isn’t really an app platform. I wanted to run my site on Heroku.

Through some searching and tinkering I figured out a couple of different ways of getting my site running on Heroku. I’ll share both of them here. The first method is great for its simplicity but is short on configurability and efficiency. The second method is more complicated but is performant and endlessly tweakable.

Method 1: Running the Jekyll Server on Heroku

Heroku’s latest application stack, Cedar, provides an amazing amount of flexibility with some very simple configuration techniques. It makes running your Jekyll site on Heroku as simple as running it on your own computer – you install some gems and run the server.

I used this example app created by Mark Pundsack as a guide. Its REAME explains the setup process very well. Here is a simple series of steps to get up and running for free on Heroku, assuming you already have a working Jekyll site.

Create a Heroku app on the Cedar stack

From the top level of your Jekyll site’s repo, create the Heroku site. This will provision the Heroku site and add a git remote named “heroku”.

heroku create -s cedar

Install the gems

Create a Gemfile that looks like this:

source :rubygems

gem 'jekyll'
gem 'RedCloth'

Then run:

bundle install

to install the gems and create a Gemfile.lock file.

Configure the Jekyll server

Apps on the Cedar stack define their process types via a Procfile. Your Procfile needs one line defining the Jekyll server like this:

web: bundle exec jekyll --server -p $PORT

Deploy to Heroku

git push heroku master

That’s it! Now your Jekyll site is running on Heroku.

Method 2: Running Your Site As a Rack App

While the method above is simple and straightforward, it’s not the ideal way to run the site. The Jekyll server isn’t meant for production as it generates pages on the fly for every request. I wanted to take advantage of the real power of Jekyll, its ability to generate a static site that can be served efficiently.

Instead of using the Jekyll server, I really wanted to serve my site as a Rack app. That way I could choose my own app server and have a great deal more control over things like caching using Rack middleware.

Setting Up rack-jekyll

Luckily this can be done quite easily using the rack-jekyll project. Here’s how to set it up.

First add the gem to your Gemfile. I also added the Thin gem, so I could use it as my server. Now your Gemfile should look something like this:

source :rubygems

gem 'jekyll'
gem 'rack-jekyll'
gem 'RedCloth'
gem 'thin'

Next set up a config.ru file that looks like this:

require 'rack/jekyll'

run Rack::Jekyll.new

Finally, modify your Procfile to run your new Rack app. With Thin, mine looks like this:

web: bundle exec thin start -p $PORT -e $RACK_ENV

Building Your Site the Right Way

The documentation for rack-jekyll suggests building the files and then committing the _site directory to your repo before deploying to Heroku. While that will work, it’s a bit messy. I really didn’t want builds to be checked into version control.

An alternative would have been letting Jekyll build the site on the first request to each dyno, but that’s no good either, because it mixes the build and run phases. Plus, depending on writes across multiple requests is not the intended use of Heroku’s ephemeral writable filesystem and could lead to bugs.

What I really needed was to build the site during Heroku’s build phase. This can actually be done by leveraging a new experimental feature known as a “custom buildpack.” Buildpacks are the key to Heroku’s polyglot platform. Each language or framework supported on Heroku such as Ruby, Node.js, or Clojure has it’s own buildpack containing the tools necessary to build an app.

In a normal deployment Heroku automatically detects what buildpack to apply, but you can actually tell Heroku to use a specific buildpack. Either pass an option for it when you create the app:

heroku create -s cedar --buildpack [URL]

or provide the URL for the buildpack repo using the BUILDPACK_URL config variable.

Since it already had all the tools for Rack apps, I forked the Ruby buildpack and added a few lines so that my Jekyll site will be built by Heroku during slug compilation. Finally I set my BUILDPACK_URL with the following command:

heroku config:add BUILDPACK_URL=http://github.com/mattmanning/heroku-buildpack-ruby-jekyll.git

Now when I deploy I see an extra “Building jekyll site” line in the output,

[~/code/blog] git push heroku master
Counting objects: 12, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (8/8), done.
Writing objects: 100% (8/8), 1.10 KiB, done.
Total 8 (delta 4), reused 0 (delta 0)

-----> Heroku receiving push
-----> Fetching custom build pack... done
-----> Ruby/Rack app detected
-----> Installing dependencies using Bundler version 1.1.rc
       Running: bundle install --without development:test --path vendor/bundle --binstubs bin/ --deployment
       Fetching gem metadata from http://rubygems.org/.......
       Using RedCloth (4.2.8)
       Using posix-spawn (0.3.6)
       Using albino (1.3.3)
       Using fast-stemmer (1.0.0)
       Using classifier (1.3.3)
       Using daemons (1.1.4)
       Using directory_watcher (1.4.1)
       Using eventmachine (0.12.10)
       Using kramdown (0.13.3)
       Using liquid (2.3.0)
       Using syntax (1.0.0)
       Using maruku (0.6.0)
       Using jekyll (0.11.0)
       Using rack (1.3.5)
       Using thin (1.3.1)
       Using bundler (1.1.rc)
       Your bundle is complete! It was installed into ./vendor/bundle
       Cleaning up the bundler cache.
       Building jekyll site
-----> Discovering process types
       Procfile declares types     -> web
       Default types for Ruby/Rack -> console, rake
-----> Compiled slug size is 7.2MB
-----> Launching... done, v47
-----> Deploy hooks scheduled, check output in your logs
       http://www.mwmanning.com deployed to Heroku

To git@heroku.com:mattmanning.git
   8f84bc4..9350a12  master -> master

and my site runs as a Rack app without polluting the repo with build files!

EC2 Micro Instance as a Remote Bittorrent Client

matt.manning@gmail.com (Matt Manning) — Mon, 29 Nov 2010 00:00:00 +0000

Bittorrent is an efficient, fault-tolerant way to distribute files across the Internet. When a file is popular, it can get shared really quickly as lots of users swarm to download and upload the file simultaneously. When a file is less popular, however, it can take hours or even days to download because of a diminished number of seeders. If you use a desktop computer that runs 24/7 and is always connected to a reliable Internet connection this is no problem, but if your primary machine is a laptop and especially if you use it on the go, slow torrents can turn into a problem.

Wouldn’t it be nice if you had a remote Bittorrent client running on a reliable server somewhere? You could start a download any time from any computer, then pull down the file over a fast, reliable connection when it’s finished. You could even be nice and seed the file for a few days, all the while using (or more importantly NOT using) your laptop as normal.

Amazon’s new, cheap — or in limited cases free — “micro” EC2 instances make this a possibility. Below you’ll find a step-by-step guide to set up your own remote bittorrent client using the Transmission application on an EC2 micro instance running Ubuntu.

Amazon Web Services

First you’ll need to create an Amazon Web Services (AWS) account. If you already have an AWS account you can skip ahead to the server creation. If not head on over to http://aws.amazon.com/free and sign up for an account and get your free Linux-based micro instance (this is what we’ll be using to run our client).

Creating and Booting an Instance

Now that you’re all signed up, we need to create and boot an EC2 micro instance to run our client. EC2 instances are based on Amazon Machine Image (AMI) files. For this guide, we’ll be using one of the official Ubuntu 10.4 LTS AMIs found here. You should chose a 32-bit, EBS-based AMI from a region geographically close to you. I chose ami-480df921.

Once you have the ID of the AMI you want to use, create a new instance based on it. Log into the AWS management console and click on the “Instances” link in the left navbar, then click the “Launch Instance” button.

Click the “Community Instances” tab in the wizard that pops up and paste your AMI ID into the search field. Click the “Select” button next the the search result for your image.

Change the “Instance Type” to “Micro (t1.micro, 613 MB) and click “Continue.”

Click “Continue” on the next screen to use the default Kernel ID and RAM Disk ID.

On the next screen, give your instance a name. I called mine “Remote Bittorrent.” Click “Continue.”

If you don’t already have a key pair set up, click the “Create New Key Pair” radio button and follow the directions to create a new key pair.

On the next screen, click the “Create a New Security Group” radio button. Here we will open port 22 so we can SSH into the server for administration. Add an entry for SSH by choosing it from the select in the “Application” column and clicking the “Add Rule” button. Click “Continue.”

Finally click the “Launch” button to start your instance. It may take a few minutes to launch your instance for the first time.

Return to the management console and click “Security Groups” in the left navbar. Click the name of the security group we just created; we need to a few more rules to it.

First we’ll open port 9091, which is the port used by the Transmission web client. To do this choose “Custom” from the “Connection Method” select box. Chose TCP as the Protocol, From Port and To Port should be 9091, and the source IP should be 0.0.0.0/0. Click the “Add Rule” button. If you don’t see the new rule in the list right away, click the “Refresh” button near the top of the page.

Next we’ll open a range of ports for the actual bittorrent application to use. Again chose “Custom” as the Connection Method, TCP as the Protocol, 49152 as the From Port, 65535 as the To Port, and 0.0.0.0/0 as the Source IP. Click “Add Rule.”

Update/Installation

Now we’ll connect to our new server and configure all of the software we’ll be using. Click “Instances” in the left navbar. Select your instance and select “Connect” under the “Instance Actions” select box. Follow the directions to SSH into your server, but use the user “ubuntu” instead of “root”.

ssh -i bittorrent.pem ubuntu@ec2-184-72-148-12.compute-1.amazonaws.com

Now we’ll upgrade all of the installed packages. Issue the following commands:

ubuntu@ip-10-203-65-125:~$ sudo aptitude update
ubuntu@ip-10-203-65-125:~$ sudo aptitude upgrade

Next, install the Transmission packages

ubuntu@ip-10-203-65-125:~$ sudo aptitude install transmission-cli transmission-daemon

Configuration

Now we want to stop transmission-daemon so we can edit its config.

ubuntu@ip-10-203-65-125:~$ sudo /etc/init.d/transmission-daemon stop

Open the settings.json config file with your favorite editor.

ubuntu@ip-10-203-65-125:~$ sudo nano /etc/transmission-daemon/settings.json

Save downloaded files in a more convenient place: “download-dir”: “/home/ubuntu/Downloads”

Come up with a password to protect your server: “rpc-password”: “mysecretpassword”. (Don’t worry. This password will be obscured to a hash once we restart the transmission-daemon.) Turn off the rpc-whitelist so we can access this client from any IP: “rpc-whitelist-enabled”: false,

Now start transmission-daemon back up:

ubuntu@ip-10-203-65-125:~$ sudo /etc/init.d/transmission-daemon start

Finally, we need to create the /home/ubuntu/Downloads directory and give transmission-daemon the ability to write to it.

ubuntu@ip-10-203-65-125:~$ mkdir /home/ubuntu/Downloads/
ubuntu@ip-10-203-65-125:~$ chown ubuntu:debian-transmission /home/ubuntu/Downloads/
ubuntu@ip-10-203-65-125:~$ sudo chmod g+w /home/ubuntu/Downloads/

And we’re done!

Now point your browser at your instance on port 9091. You can find the URL by clicking on your instance in the AWS management console and checking the value of the “Public DNS” field. For my instance, the URL is http://ec2-184-72-148-12.compute-1.amazonaws.com:9091. You will be prompted to log in. The username is “transmission” (unless you changed it in settings.json) and the password is whatever password you chose in settings.json.

Now you have a client that you can browse to at any time on any computer. You can upload torrent files directly, or you can simply enter the URL of the torrent file and have the client do the rest.

Amazon gets pretty decent download speeds too :)

TimeMachine + Ubuntu NAS

matt.manning@gmail.com (Matt Manning) — Tue, 13 Jan 2009 00:00:00 +0000

So after a few months of hem-hawing around, I finally implemented a respectable backup solution for my Mac. It’s getting old enough (almost 4 years) that I’m starting to worry about random hardware failures, so having a reliable backup is very important.

At home, I have a computer running Ubuntu Linux that I use as a file/media/print server. It has four 250 GB drives in a RAID5 configuration. Since RAID5 can be rebuilt if a drive dies, I feel like this is a pretty safe place to store backups, although eventually I hope to have an offsite backup (ex: Amazon S3) as well.

A while back a little command that would let TimeMachine use network drive started floating around the Internet:

defaults write com.apple.systempreferences TMShowUnsupportedNetworkVolumes 1

I already had SAMBA set up, but I had some problems, so I ended up following this tutorial — which I highly recommend — and setting up netatalk so I could share the volume over AFP. Looking back, I don’t think SAMBA was the problem, I think the issue was with creating the initial backup disk image. I had to make the image locally and copy it over, as is explained in the troubleshooting section, but after that everything worked perfectly.