Schneems

News flash, writing a Rails app without knowing your routes is pretty much impossible, and this just in $ rake routes takes forever [1] to run. So how can we build a Rails site with a minimum of time and a maximum of awesome? You can use the recently released Sextant Gem [2] to generate routes in your Rails app.

As it turns out if you benchmark the code in $ rake routes it is reasonably fast, the slow part comes from having to initialize the app environment every time it’s run. So lets take some of that code and put it in your already initialized web app. What you get is a whole lot of awesome in a tiny package. Next time you want to see your routes, just visit /rails/routes in your browser if you’ve got sextant installed, and you’ll see your routes in no time.

To install add this to your Gemfile:

gem 'sextant' 

Then run bundle install and you’re good to go. Start your server and visit /rails/routes

This in-browser information is local only. If you like Sextant tell your friends, and tell me @schneems

• [1] Forever is measured in seconds
• [2] Sextant is an old device for navigating. It can help you find your route on a long journey.

The cats out of the bag, Ruby isn’t immune to legacy code problems. Just because your app is written in a hip, fun, and dynamic language doesn’t mean that your codebase can’t stagnate, bloat, and quickly turn into an unmaintainable ball of mud. Before Gowalla was purchased by Facebook, the Rails code base stood at close to seven thousand files, with the largest model clocking in at around 3,500 lines of code. While we were somewhat unique, being originally written in Merb and then ported to Rails, applications of this size aren’t all that uncommon. If you’ve got a large app there are a number of things you can do make your situation better, one of the simplest with the greatest impact is splitting up models into concerns.

If you’re not familiar with concerns, you can read up about them at Concerned about Code Reuse?. Go ahead, we’ll wait.

Rails has long advocated a thin controller, fat model approach to development, which works great early but can lead to a model obesity epidemic. If we split out our models into different concerns we can group related code, and even make re-using code between projects easier. Best of all, if you start early on it’s a pretty painless process.

The User Model

There’s only two businesses that refer to their customers as ‘users’ and software is one of them. It’s also bound to be one of the largest models in your app since so much will likely need to be connected to a user. It’s a good place to start splitting up a model into a concern. Lets say that we want to add some methods on our user object so they can access Facebook information we can start by creating a new file app/models/user/facebook_methods.rb (you’ll need to create the user folder).

This file is where we’ll group all of our methods related to Facebook. For this example i’ll be using the Koala Facebook gem, and we assume our user model has a facebook_token attribute persisted to the database.

module User::FacebookMethods
  extend ActiveSupport::Concern

  def facebook_graph
    @facebook_graph ||= Koala::Facebook::API.new(facebook_token)
  end
end

Not a bad start, now we want to add this ability to our user model, open up app/models/user.rb and add our concern.

class User < ActiveRecord::Base
  include User::FacebookMethods
end

Great! Now we can construct our Facebook graph object straight from our user.

user = User.where("facebook_token is not null").first
user.facebook_graph
# => <# Koala #...

user.facebook_graph.get_connections("me", "friends")
# => {52930 => 'Terence Lee',  12345 => "Ruby Ku" #...

That Was Easy, but…

What did that buy us? First we’ve got an obvious place to store our code. Want to write a method that pulls out all of a user’s Facebook friends? Put it in the facebook_methods.rb file. If you forget the name of that method, check out your Facebook methods. If you need the facebook_graph method, bet you money it’s in that concern. If all related code is in one place its a lot easier to scan visually and to search for keywords.

Won’t This add More Code to the Codebase?

In the example above, we added 4 extra lines of code to save us 3 measly lines in our user.rb file. While this isn’t ideal for such a small concern, as it grows in size its much easier to keep track of the components, which in turn helps keep your code small and manageable. It also nudges developers to create unit tests for those specific concerns. This sounds minor, but when you get to a file with 3500 lines of code, you start duplicating functionality that you didn’t know existed. Either it was added by another developer, or you forgot you added it months ago. Keeping everything in its place helps keep your code sane.

How Should I Break up My Code

Often times I like breaking out my concerns based on knowledge of third party services. For example I broke out a concern for all the Facebook methods above. I use websolr and I like having a separate concern for all the search related methods. Recently I played around with the Ice Cube gem which is a library for creating an iCal formatted recurring date syntax. I split that code out into a concern, not because it was touching another service, but because I might want to re-use that code in another model some day, and it’s easier to break out the code now. There are no hard and fast rules, just don’t go overboard and have 100 concerns for every model with 3 lines of code in each of them, and on the flip side don’t have one concern with everything.

Just think of the different areas of ‘concern’, that your code covers. Get it?

Legacy Code

While building out the final version of the Gowalla service we managed to promote Redis and Cassandra to first class data storage citizens, completely re-write all web controllers, and split out a brand new api into a separate set of controllers (more on this in a later date). It was a ton of work, we were completely changing the way our service worked and creating new paradigms such as a “Stories” where multiple users could check each other in and at the same time, we still had to support a ton of 3rd party client applications using the old API.

So, how did concerns help? We used concerns to isolate new code and new services. It also helped us to add more & better unit testing by focusing different spec files on different areas so models/users/following_methods.rb would be tested by spec/models/users/following_methods_spec.rb. Most of the developers used some form of automatic test runner such as Guard Rspec, and it is nice being able to run only the unit tests associated with the concern you’re writing without having to run all the unit tests for that model.

Bonus! Ever find a method that you were pretty sure wasn’t being called by anything. Maybe it’s in a model that wasn’t exactly 100% tested. You could try making a concern for methods of questionable value. In a month if it’s still there, delete the sucker, deploy to staging, validate and commit to master.

Shared Concerns

If you have code that needs to be shared by multiple models in your project, you can keep this in your lib folder. I actually like to have a concerns folder like lib/concerns/models/duplicate_code.rb. When we added Cassandra to the Gowalla project we needed a way to get our Postgres models to play nice. Thats when Adam Keys and Bill Doughty pulled out common logic and put it into a concern using another library called Chronologic. lib/concerns/models/chronologify.rb

Then any time you wanted this shared code into your model, you just had to include it.

# models/checkin.rb
class Checkin < ActiveRecord::Base
  include Concerns::Models::Chronologify
end

Don’t forget to add the concern folder in your lib to your search path.

Wrap it Up

There have been tomes written about dealing with legacy code in software, I’ve been recently recommended Working Effectively with Legacy Code . Using concerns won’t be a magic bullet, but it will help keep your code nicely organized. Even if you’re dealing with a pristine new app, concerns are one way to help it stay that way. Give concerns a try and let me know if you have a good or bad experience @schneems.

Right out of the gate, Ruby gives us some powerful ways to re-use instance and class methods without relying on inheritance. Modules in Ruby can be used to mixin methods to classes fairly easily. For example, we can add new instance methods using include.

module DogFort
  def call_dog
    puts "this is dog!"
  end
end

class Dog
  include DogFort
end

Now we’re able to call any methods defined in our DogFort Module as if they were simply slipped into (included) into our Dog class.

dog_instance = Dog.new
dog_instance.call_dog
# => "this is dog!"

Using Modules a fairly easy way to re-use methods, if you want you can extend a Module to add methods to a class directly.

module DogFort
  def board_the_doors
    puts "no catz allowed"
  end
end

class Dog
  extend DogFort
end

Now if we were to call Dog.new.board_the_doors we would get an error, since we’ve added it as a class method instead.

Dog.board_the_doors
# => "no catz allowed"

Dog.class
# => Class

Sweet! Though what if you wanted to add an instance method and a class method to a class. We could have two Modules, one to be included and one to be extended, wouldn’t be to hard but it would be nice if we only had to use one include statement, especially if the two Modules are related. So is it possible to add instance and class methods with only one include statement? Of course…

Enter Concerns

A concern is a Module that adds instance methods (like Dog.new.call_dog) and class methods (like Dog.board_the_doars) to a class. If you’ve poked around the Rails source code you’ll see this everywhere. It’s so common that Active Support added a helper Module to create concerns. To use it require ActiveSupport and then extend ActiveSupport::Concern

require 'active_support/concern'

module DogFort
  extend ActiveSupport::Concern
  # ...
end

Now any methods you put into this Module will be instance methods (methods on a new instance of a class Dog.new) and any methods that you put into a Module named ClassMethods will be added on to the class directly (such as Dog).

require 'active_support/concern'

module YoDawgFort
  extend ActiveSupport::Concern

  def call_dawg
    puts "yo dawg, this is dawg!"
  end


  # Anything in ClassMethods becomes a class method
  module ClassMethods
    def board_the_doors
      puts "yo dawg, no catz allowed"
    end
  end
end

So now when we add this new Module to a class, we’ll get instance and class methods

class YoDawg
  include YoDawgFort
end

YoDawg.board_the_doars
# => "yo dawg, no catz allowed"

yodawg_instance = YoDawg.new
yodawg_instance.call_dawg
# => "yo dawg, this is dawg!"

Pretty cool huh?

Included

That’s not all, Active Support also gives us a special method called included that we can use to call methods during include time. If you add included to your ActiveSupport::Concern any code in there will be called when it is included

module DogCatcher
  extend ActiveSupport::Concern

  included do
    if self.is_a? Dog
      puts "gotcha!!"
    else
      puts "you may go"
    end
  end
end

So when we include DogCatcher in a class it’s included block will be called immediately.

class Dog
  include DogCatcher
end
# => "gotcha!!"

class Cat
  include DogCatcher
end
# => "you may go"

While this is a contrived example, you can imagine wanting to maybe make a concern for Rails controllers and wanting to add before_filter’s to our code. We can do this easily adding the included block.

Is this magic?

Nope, under the hood we’re just using good old fashioned Ruby. If you want to learn more about all the fun things you can do with Modules I recommend checking out one of my favorite Ruby books Metaprogramming Ruby and Dave Thomas also has a fantastic screencast series.

Gotcha

When you’re writing Modules I guarantee that you’ll slip up and accidentally try to create a class method using self or class << self but it won’t work because it’s now a method on the Module.

module DogFort
  def self.call_dog
    puts "this is dog!"
  end
ene

In the example above the context of self is actually the Module object DogFort so when we include it into another class we won’t see the method.

class Wolf
  include DogFort
end

Wolf.call_dog
# NameError: undefined local variable or method `call_dog'

wolf_instance = Wolf.new
wolf_instance.call_dog
# NameError: undefined local variable or method `call_dog'

If you want to use that method in this context you will need to call the Module directly

DogFort.call_dog
# => "this is dog!"
puts DogFort.class
# => Module

Fin

That’s all for today, in my next post I’m going to show you how to clean up your legacy code base with concerns. Let me know if you have any questions @schneems!

You may also be interested in Concerning Yourself with ActiveSupport::Concern, Concerns in ActiveRecord and Better Ruby Idioms.

Haven’t you always wanted to make some changes to your server and then absolutely slam it with traffic to see the result? Thats pretty much what I did last week while writing how to Super Charge your Rails App with Rack Cache, using the BlitzIO tool.

If you’re interested in trying out Blitz watch a demo in the screencast below.

For a walkthrough on how to add Rack::Cache and Memcache to your server read my performance article on the Heroku Dev Center.

Slow is sweeping the nation: slow food, slow living, and slow reading. Unfortunately your app called, it said it wants to be fast. Web apps that respond quickly are more enjoyable to work with and Google even gives them a small SEO bump. Recently basecamp next got quite a bit of customer love based on how quickly it responds. The fact of the matter is if it’s on the web, fast matters.

This is based on an article I wrote for the Heroku Dev Center Using Rack::Cache with Memcached for Static Asset Caching in Rails 3.1+. If you have any questions ping me @schneems.

One of the quickest ways we can speed up your whole application is to add on HTTP caching. Not only does this mean we return static files quickly, we also reduce the overall load of your application. The easy answer to this hard problem is to configure your application to use Rack::Cache with Memcache.

Performance Testing

I did some load testing of the example app with the trial version of the BlitzIO addon. BlitzIO will hit whatever url you specify with as many simulated users as you want and graph the result for you. This is a run from 1 to 250 concurrent users over the course of 60 seconds with four dynos. Compare the default production settings to using Memcache with Rack::Cache.

Rails Default Settings

(Left is response time [peaks around 500ms] right is number of concurrent users represented by straight line, bottom is time of test )

Memcache & Rack Cache

(Left is response time [peaks around 100ms] right is number of concurrent users represented by straight line, bottom is time of test )

Here the Memcache & Rack::Cache combo smokes the default rails settings. If you want to run your own tests, you should know that each time you run BlitzIO, your performance graph will be different. I recommend running a few tests to make sure you’re not seeing a fluke. I also highly recommend doing any type of load testing on a staging server instead of production, or it could take down your site and your users wouldn’t be too happy.

Settings

So how can you, the speedfreak you are, get these types of results in your app? Long story short, you want to add this to your config/production.rb file:

config.action_dispatch.rack_cache = {
                        :metastore    => Dalli::Client.new,
                        :entitystore  => 'file:tmp/cache/rack/body',
                        :allow_reload => false }
config.static_cache_control = "public, max-age=2592000"

For more details on how to implement add this to your Rails app and why we chose these settings, please read the devcenter article or browse source on my demo app. Good luck, and enjoy the speed!

Heroku just announced their support of hstore in their dedicated Postgres 9.1 instances. Hstore is a schema less key value store inside of PostgreSQL that allows us to store data like hashes directly inside of a column. It’s great for when you don’t know exactly what types of attributes you need to store on a model, or if you need to support many different attributes for the same model.

Update: You can now use Hstore with development databases on Heroku

A good example is storing attributes for a Product model. We might start out only selling books, which have an author, number of pages, but then transition over to selling laptops which have cpu speed and display resolution. Using Hstore allows us to easily store all these values without having to make a bunch mostly blank columns.

To get started with Rails and hstore you can watch the screencast below or visit the hstore example app running on Heroku.

More on Hstore

Hstore in Rails functions much like serializing hashes, except that we can query our data much faster since hstore is a native data type. It is supported natively in Rails 4, but until then we’ll need to use the activerecord-postgres-hstore gem.

Getting Started

You will need a version of PostgreSQL locally that supports the hstore extension. I recommend installing postgres using homebrew on OS X. Once you’ve done that you can enable hstore usage by running this in Postgres

CREATE EXTENSION hstore;

You can put this in a migration if you prefer

class SetupHstore < ActiveRecord::Migration
  def self.up
    execute "CREATE EXTENSION hstore"
  end

  def self.down
    execute "DROP EXTENSION hstore"
  end
end

Once that is done you will need to create a column with a type of hstore, here we are giving our Product model a column called data with hstore type.

class CreateProducts < ActiveRecord::Migration
  def change
    create_table :products do |t|
      t.string  :name
      t.hstore  :data
      t.timestamps
    end
  end
end

Once we’ve done that we can now store any type of attributes in the data column.

Product.create(:name => "Geek Love: A Novel", :data => {'author' => 'Katherine Dunn', 'pages' => 368, 'category' => 'fiction'})
Product.last.data['category']  # => 'fiction'

Querying

Not only does hstore allow us to store arbitrary keys and values it allows us to quickly query them.

  # Find all products that have a key of 'author' in data
  Product.where("data ? :key", :key => 'author')

  # Find all products that have a 'pages' and '368' key value pair in data
  Product.where("data @> (:key => :value)", :key => 'pages', :value => '368')

  # Find all products that don't have a key value pair 'pages' and '999' in data
  Product.where("not data @> (:key => :value)", :key => 'pages', :value => '999')

  # Find all products having key 'author' and value like 'ba' in data
  Product.where("data -> :key LIKE :value",     :key => 'author, :value => "%Kat%")

More information available in the Postgres hstore docs. Though like a normal column if you query it frequently, you can get even more speed by adding an index. You can do this using one of two indexes that also speed up full text searches. They’re GiST (Generalized Search Tree) or GIN (Generalized Inverted iNdex). Which sill speed up queries using the @> and ? postgres operators.

class Index < ActiveRecord::Migration
  def up
    execute "CREATE INDEX products_gin_data ON products USING GIN(data)"
  end

  def down
    execute "DROP INDEX products_gin_data"
  end
end

Use It

Try out the hstore example app, clone the Github repo, and let me know what cool things you build on twitter @schneems.

Thanks

Special thanks to Aaron Patterson and Joel Hoffman for their work with hstore & Rails4, to the team at Softa for writing this gem, & and the team at Heroku for their contributions to Postgres, and supporting this feature.

We’ve all been there, we start a new project with grand hopes and aspirations. But soon enough our project turns into one big ball of mud. In this presentation I talk about some common mistakes in Rails and how to keep your project Sane.

I gave this presentation at Austin On Rails and it was filmed by Austin Tech Videos.

If you want to make a change to Rails, you probably want to test out your change in an existing project. Here is what I needed to do to get an example project running on my fork of Rails:

First: Fork Rails

Before you can run a custom copy of Rails you’ll want to make a fork and then clone your fork to your local machine.

Go to https://github.com/rails/rails and click the Fork button.

Once you’ve got a fork clone it locally:

$ cd documents/projects

$ git clone git@github.com:schneems/rails.git 

$ cd rails

Next Modify Your Gemfile

Now that you’ve got a development copy of Rails locally you’ll need to modify the Gemfile of your existing Rails project that you’re testing against:

You need to point ‘rails’ to your local copy, mine is at /Users/schneems/documents/projects/rails, you also need to tell it where to find journey and arel, for those I just pointed at the existing repos online.

Here is my full Gemfile:

You may need to run bundle exec rake gem in your Rails copy directory to regenerate gemspec files.

Run your Local project

Go into your project directory with the modified Gemfile and run Bundler

bundle install

Then start your server using

bundle exec rails s

Now you can try running your changes against your own Rails project.

If you’ve never contributed to rails check out the Rails contribution Guide or ask someone who has. Don’t forget to make a new branch before you make any changes.

For a great example on contributing to an open source project for the first time, check out Contributing to Open Source.

If you have problems message me @schneems or check out #rails-contrib on IRC freenode, they were very helpful getting me set up. Especially @nashby_ for helping with the Gemfile. Good luck and happy contributing.