Schneems

This question comes up a lot, people want to have an object, lets call it a Product that they want to create in several different steps. Let’s say our product has a few fields name, price, and category and to have a valid product all these fields must be present.

This is a re-post of a wiki I wrote for Wicked. While it was written to be used with a wizard, the pattern can be used without it. Enjoy!

The Problem

We want to build an object in several different steps but we can’t because that object needs validations. Lets take a look at our Product model.

class Product < ActiveRecord::Base
  validates :name, :price, :category, :presence => true

end

So we have a product that relies on name, price, and category to all be there. Lets take a look at a simple Wizard controller for ProductController.

class ProductController < ApplicationController
  include Wicked::Wizard

  steps :add_name, :add_price, :add_category

  def show
    @product = Product.find(params[:product_id])
    render_wizard
  end


  def update
    @product = Product.find(params[:product_id])
    @product.update_attributes(params[:product])
    render_wizard @product
  end


  def create
    @product = Product.create
    redirect_to wizard_path(steps.first, :product_id => @product.id)
  end
end

Here the create action won’t work because our product didn’t save. OhNo!

The Solution

The best way to build an object incrementally with validations is to save the state of our product in the database and use conditional validation. To do this we’re going to add a status field to our Product class.

class ProductStatus < ActiveRecord::Migration
  def up
    add_column :products, :status, :string
  end

  def down
    remove_column :product, :status
  end
end

Now we want to add an active state to our Product model.

def active?
  status == 'active'
end

And we can add a conditional validation to our model.

class Product < ActiveRecord::Base
  validates :name, :price, :category, :presence => true, :if => :active?

  def active?
    status == 'active'
  end
end

Now we can create our Product and we won’t have any validation errors, when the time comes that we want to release the product into the wild you’ll want to remember to change the status of our Product on the last step.

class ProductController < ApplicationController
  include Wicked::Wizard

  steps :add_name, :add_price, :add_category

  def update
    @product = Product.find(params[:product_id])
    params[:product][:status] = 'active' if step == steps.last
    @product.update_attributes(params[:product])
    render_wizard @product
  end

Great, but…

So that works well, but what if we want to disallow a user to go to the next step unless they’ve properly set the value before it. We’ll need to split up or validations to support multiple conditional validations.

class Product < ActiveRecord::Base
  validates :name,      :presence => true, :if => :active_or_name?
  validates :price,     :presence => true, :if => :active_or_price?
  validates :category,  :presence => true, :if => :active_or_category?

  def active?
    status == 'active'
  end

  def active_or_name?
    status.include?('name') || active?
  end

  def active_or_price?
    status.include?('price') || active?
  end

  def active_or_category?
    status.include?('category') || active?
  end

end

Then in our ProductController Wizard we can set the status to the current step name in in our update.

  def update
    @product = Product.find(params[:product_id])
    params[:product][:status] = step
    params[:product][:status] = 'active' if step == steps.last
    @product.update_attributes(params[:product])
    render_wizard @product
  end

So on the :add_name step status.include?('name') will be true and our product will not save if it isn’t present. So in the update action of our controller if @product.save returns false then the render_wizard @product will direct the user back to the same step :add_name. We still set our status to active on the last step since we want all of our validations to run.

Wow that’s cool, but seems like a bunch of work

What you’re trying to do is fairly complicated, we’re essentially turning our Product model into a state machine, and we’re building it inside of our wizard which is a state machine. Yo dawg, i heard you like state machines… This is a very manual process which gives you, the programmer, as much control as you like.

Cleaning up

If you have conditional validation it can be easy to have incomplete Product’s laying around in your database, you should set up a sweeper task using something like Cron, or Heroku’s scheduler to clean up Product’s that are not complete.

lib/tasks/cleanup.rake

namespace :cleanup do
  desc "removes stale and inactive products from the database"
  task :products => :environment do
    # Find all the products older than yesterday, that are not active yet
    stale_products = Product.where("DATE(created_at) < DATE(?)", Date.yesterday).where("status is not 'active'")

    # delete them
    stale_products.map(&:destroy)
  end
end

When cleaning up stale data, be very very sure that your query is correct before running the code. You should also be backing up your whole database periodically using a tool such as Heroku’s PGBackups incase you accidentally delete incorrect data.

Wrap it up

Hope this helps, I’ll try to do a screencast on this pattern. It will really help if you’ve had problems implementing this, to let me know what they were. Also if you have another method of doing partial model validation with a wizard, I’m interested in that too. As always you can find me on the internet @schneems. Thanks for using Wicked!

Haven’t you always wanted to make some changes to your server and then absolutely slam it with traffic to see the result? Thats pretty much what I did last week while writing how to Super Charge your Rails App with Rack Cache, using the BlitzIO tool.

If you’re interested in trying out Blitz watch a demo in the screencast below.

For a walkthrough on how to add Rack::Cache and Memcache to your server read my performance article on the Heroku Dev Center.

Slow is sweeping the nation: slow food, slow living, and slow reading. Unfortunately your app called, it said it wants to be fast. Web apps that respond quickly are more enjoyable to work with and Google even gives them a small SEO bump. Recently basecamp next got quite a bit of customer love based on how quickly it responds. The fact of the matter is if it’s on the web, fast matters.

This is based on an article I wrote for the Heroku Dev Center Using Rack::Cache with Memcached for Static Asset Caching in Rails 3.1+. If you have any questions ping me @schneems.

One of the quickest ways we can speed up your whole application is to add on HTTP caching. Not only does this mean we return static files quickly, we also reduce the overall load of your application. The easy answer to this hard problem is to configure your application to use Rack::Cache with Memcache.

Performance Testing

I did some load testing of the example app with the trial version of the BlitzIO addon. BlitzIO will hit whatever url you specify with as many simulated users as you want and graph the result for you. This is a run from 1 to 250 concurrent users over the course of 60 seconds with four dynos. Compare the default production settings to using Memcache with Rack::Cache.

Rails Default Settings

(Left is response time [peaks around 500ms] right is number of concurrent users represented by straight line, bottom is time of test )

Memcache & Rack Cache

(Left is response time [peaks around 100ms] right is number of concurrent users represented by straight line, bottom is time of test )

Here the Memcache & Rack::Cache combo smokes the default rails settings. If you want to run your own tests, you should know that each time you run BlitzIO, your performance graph will be different. I recommend running a few tests to make sure you’re not seeing a fluke. I also highly recommend doing any type of load testing on a staging server instead of production, or it could take down your site and your users wouldn’t be too happy.

Settings

So how can you, the speedfreak you are, get these types of results in your app? Long story short, you want to add this to your config/production.rb file:

config.action_dispatch.rack_cache = {
                        :metastore    => Dalli::Client.new,
                        :entitystore  => 'file:tmp/cache/rack/body',
                        :allow_reload => false }
config.static_cache_control = "public, max-age=2592000"

For more details on how to implement add this to your Rails app and why we chose these settings, please read the devcenter article or browse source on my demo app. Good luck, and enjoy the speed!

Heroku just announced their support of hstore in their dedicated Postgres 9.1 instances. Hstore is a schema less key value store inside of PostgreSQL that allows us to store data like hashes directly inside of a column. It’s great for when you don’t know exactly what types of attributes you need to store on a model, or if you need to support many different attributes for the same model.

Update: You can now use Hstore with development databases on Heroku

A good example is storing attributes for a Product model. We might start out only selling books, which have an author, number of pages, but then transition over to selling laptops which have cpu speed and display resolution. Using Hstore allows us to easily store all these values without having to make a bunch mostly blank columns.

To get started with Rails and hstore you can watch the screencast below or visit the hstore example app running on Heroku.

More on Hstore

Hstore in Rails functions much like serializing hashes, except that we can query our data much faster since hstore is a native data type. It is supported natively in Rails 4, but until then we’ll need to use the activerecord-postgres-hstore gem.

Getting Started

You will need a version of PostgreSQL locally that supports the hstore extension. I recommend installing postgres using homebrew on OS X. Once you’ve done that you can enable hstore usage by running this in Postgres

CREATE EXTENSION hstore;

You can put this in a migration if you prefer

class SetupHstore < ActiveRecord::Migration
  def self.up
    execute "CREATE EXTENSION hstore"
  end

  def self.down
    execute "DROP EXTENSION hstore"
  end
end

Once that is done you will need to create a column with a type of hstore, here we are giving our Product model a column called data with hstore type.

class CreateProducts < ActiveRecord::Migration
  def change
    create_table :products do |t|
      t.string  :name
      t.hstore  :data
      t.timestamps
    end
  end
end

Once we’ve done that we can now store any type of attributes in the data column.

Product.create(:name => "Geek Love: A Novel", :data => {'author' => 'Katherine Dunn', 'pages' => 368, 'category' => 'fiction'})
Product.last.data['category']  # => 'fiction'

Querying

Not only does hstore allow us to store arbitrary keys and values it allows us to quickly query them.

  # Find all products that have a key of 'author' in data
  Product.where("data ? :key", :key => 'author')

  # Find all products that have a 'pages' and '368' key value pair in data
  Product.where("data @> (:key => :value)", :key => 'pages', :value => '368')

  # Find all products that don't have a key value pair 'pages' and '999' in data
  Product.where("not data @> (:key => :value)", :key => 'pages', :value => '999')

  # Find all products having key 'author' and value like 'ba' in data
  Product.where("data -> :key LIKE :value",     :key => 'author, :value => "%Kat%")

More information available in the Postgres hstore docs. Though like a normal column if you query it frequently, you can get even more speed by adding an index. You can do this using one of two indexes that also speed up full text searches. They’re GiST (Generalized Search Tree) or GIN (Generalized Inverted iNdex). Which sill speed up queries using the @> and ? postgres operators.

class Index < ActiveRecord::Migration
  def up
    execute "CREATE INDEX products_gin_data ON products USING GIN(data)"
  end

  def down
    execute "DROP INDEX products_gin_data"
  end
end

Use It

Try out the hstore example app, clone the Github repo, and let me know what cool things you build on twitter @schneems.

Thanks

Special thanks to Aaron Patterson and Joel Hoffman for their work with hstore & Rails4, to the team at Softa for writing this gem, & and the team at Heroku for their contributions to Postgres, and supporting this feature.

Join me for a quick demo of Induction, the latest project by @Mattt from Heroku. Induction will let you view your data-stores including: PostgreSQL, MySQL, SQLite, Redis, MongoDB, and so much more. The alpha product also lets you run queries, and visualize data.

Induction is completely free and open source, for more info visit the website or clone the repo.

Lets admit it, code and Tumblr don’t exactly get along right now. Sure you can write your posts in markdown, but there isn’t really an out of the box experience for syntax highlighting in code blocks.

In the past I’ve resorted to using Gists to store code in my blog, but this can be a hassle to manage, especially if you need to modify your code later. One day I noticed that one of my favorite blogs The Changelog has great code highlighting on their Tumblr blog. With a little view source-ing and a little googling I eventually found out that the best way to add code highlighting to your blog is through a javascript library called prettify.js. I followed these directions by blairvanderhoof, and came up with something not too bad. While his instructions are good, I made a few modifications. Lets take a look.

1. Link to the prettify.js CDN in your html, by customizing and putting this in right before the<body> tag:

   <script type="text/javascript" src="http://cdnjs.cloudflare.com/ajax/libs/prettify/188.0.0/prettify.js"></script>
   

2. Add an the ‘prettyprint’ class to all <pre> tags that contain a <code> tag. Then initialize the prettyPrint() javascript method by adding this in your HTML:

   $(document).ready( function(){
     $('code').parent('pre').addClass('prettyprint');
     prettyPrint();
   });
   

3. Pick a prettify theme. I chose the Desert theme. Add that style to the Add Custom CSS under the advanced section, or directly to your HTML.

    /* desert scheme ported from vim to google prettify */
   pre { display: block; background-color: #333; overflow:scroll; font-size:12px;  }
   pre .nocode { background-color: none; color: #000 }
   pre .str { color: #ffa0a0 } /* string  - pink */
   pre .kwd { color: #f0e68c; font-weight: bold }
   pre .com { color: #87ceeb } /* comment - skyblue */
   pre .typ { color: #98fb98 } /* type    - lightgreen */
   pre .lit { color: #cd5c5c } /* literal - darkred */
   pre .pun { color: #fff }    /* punctuation */
   pre .pln { color: #fff }    /* plaintext */
   pre .tag { color: #f0e68c; font-weight: bold } /* html/xml tag    - lightyellow */
   pre .atn { color: #bdb76b; font-weight: bold } /* attribute name  - khaki */
   pre .atv { color: #ffa0a0 } /* attribute value - pink */
   pre .dec { color: #98fb98 } /* decimal         - lightgreen */
   
   /* Specify class=linenums on a pre to get line numbering */
   ol.linenums { margin-top: 0; margin-bottom: 0; color: #AEAEAE } /* IE indents via margin-left */
   li.L0,li.L1,li.L2,li.L3,li.L5,li.L6,li.L7,li.L8 { list-style-type: none }
   /* Alternate shading for lines */
   li.L1,li.L3,li.L5,li.L7,li.L9 { }
   
   @media print {
     pre { background-color: none }
     pre .str, code .str { color: #060 }
     pre .kwd, code .kwd { color: #006; font-weight: bold }
     pre .com, code .com { color: #600; font-style: italic }
     pre .typ, code .typ { color: #404; font-weight: bold }
     pre .lit, code .lit { color: #044 }
     pre .pun, code .pun { color: #440 }
     pre .pln, code .pln { color: #000 }
     pre .tag, code .tag { color: #006; font-weight: bold }
     pre .atn, code .atn { color: #404 }
     pre .atv, code .atv { color: #060 }
   }
   .content pre code {background-color:inherit; color:inherit}
   .content pre      { padding:1em; margin: 0.5em 0px; }
   

Note: I changed some of the Desert theme to play nice with my existing styles.

If your blog includes code snippets, you owe it to your audience to have good looking and readable syntax highlighting.

If you’re interested in getting started and aren’t already locked into a blogging platform, I recommend trying out Jekyll on Heroku it’s easy to set up and a real hackers blog. You also get syntax highlighting out of the box. Hope this helped & happy blogging.

If insanity is doing the same thing and expecting different results, I was going crazy writing step-by-step wizards. I was never happy with the end result, they did what I wanted, but were messy and had too many moving parts. I wanted a simple & re-usable way to create restful-ish controllers. Thats when I decided to rip out all that scary controller logic and bake it into in a Gem I call Wicked.

Wizards can be used for a number of things, they appear frequently after signing up for a service. They typically ask for additional information, or they give the user a tour of a service. Before I started working for Heroku, I helped out a bit on the peer learning site, Hourschool. Since students are most interested in the courses that are close to them, Hourschool asks users for their zip code. When a student signs up via Facebook, the zipcode isn’t directly exposed via the API. So to get that extra input, we decided to add an after register wizard to ask for this information and more. If you want to add similar functionality to your Rails site, you can watch the screencast, download an example, browse the source, or check out the getting started guide below.

Get Started

First install the gem, then inherit from Wicked::WizardController and you can specify a set of steps. Here we have a controller called AfterRegisterController with existing routes.

class AfterRegisterController < Wicked::WizardController

  steps :add_zip, :add_twitter, :add_friends
  # ...

Create a Show and Update action, calling render_wizard at the end of each which allows us to show the appropriate view for the step.

class AfterRegisterController < Wicked::WizardController

  steps :add_zip, :add_twitter, :add_friends

  def show
    @user = current_user
    render_wizard
  end

  def update
    @user = current_user
    render_wizard
  end
end

By default the wizard will render a view with the same name as the step. So you need to create view files for each step, in this case views/after_register/add_zip.html.erb & views/after_register/add_twitter.html.erb. In those views we can use wizard helpers to create links to the next step.

<%= link_to 'skip', next_wizard_path %>

Or you can manually specify which wizard action you want to link to by using the wizard_path helper.

<%= link_to 'skip', wizard_path(:find_friends) %>

Our :add_zip action can have a form that uses the wizard path and calls the update action. To get to this update action, you simply need to submit a form that PUT’s (or PATCH’s) to the same url:

<p>Please Fill in your zip code!</p>
<%= form_for(@user, :url => wizard_path, :method => :put) do |f| %>
  <%= f.text_field :zip, :placeholder => "zip code" %>
  <%= f.submit 'Next', :class => 'btn btn-primary' %>
<% end %>

<%= link_to 'skip', next_wizard_path %>

We then need to make sure our Signup wizard’s update action updates the current_user’s attributes. After modifying the user we can pass it into render_wizard, which will show the next step if the object saves or re-render the previous view if the user has validation errors.

def update
  @user = current_user
  @user.update_attributes(params[:user])
  render_wizard @user
end

If you want to skip showing a step under certain conditions you can do it by using the step method which returns a symbol of the current step you’re on, and skip_step if you don’t want to render the current step. So if we wanted to skip asking our users to authenticate with twitter if they skipped the add_zip step, we can do it like this:

def show
  @user = current_user
  case step
  when :add_twitter    
    skip_step if @user.zip.blank?
  end
  render_wizard
end

Redirect the user here after they are created in your system, and you have a fully baked after registration wizard. Thats all there is to it, just create a new view file every time you add a new step and you’re good to go. Please give this a try and message me at @schneems if you find any other really killer ‘wizard’ applications