opensoul.org

If you’re running Passenger in development, here is how to make Windows running in a virtual machine connect to your app in Passenger.

1. Boot up the VM and open up the Windows command prompt (go to “Start->Run…”, enter “cmd” and press enter)
2. Type ipconfig to see the network configuration. Take note of the “Default Gateway” address.
3. Navigate to C:\WINDOWS\system32\drivers\etc and edit the hosts file. Add a line with the gateway address pointing to your app’s domain (you can even list multiple on the same line).

   172.16.248.2    awesomeapp.local otherawesomeapp.local

4. Open up a browser in the VM and type in the address.

Tada! Now you can test out your app in those other browsers.

This trick has been around for a while, but I’ve talked with several people that didn’t know about it.

When deploying apps with Capistrano, your server needs to have access to the Git repository. Generating an SSH key for each server is a bit of a pain, but there’s an easier way: SSH agent forwarding enables you to use any of your local SSH keys on the server. It’s really easy to set up.

Enable SSH forwarding in deploy.rb:

set :ssh_options, {:forward_agent => true}

The only other thing you need to do is tell the SSH agent about your key.

$ ssh-add -K

The -K option only works on OS X and it adds your key to your keychain so you don’t have to run ssh-add after you reboot (and if you have a passphrase set, you don’t have to type it every time). You can also pass it the path to an SSH private key in a different location.

Now the server can pull from any Git repository that you have access to.

I love writing apps that make heavy use of search indexes, but testing them can be a bit of a pain. Here is how I got ThinkingSphinx to play nice with Cucumber.

Here is the relevant part of what I put in features/support/env.rb:

# Cucumber::Rails.use_transactional_fixtures

# http://github.com/bmabey/database_cleaner
require 'database_cleaner'
DatabaseCleaner.strategy = :truncation
Before do
  DatabaseCleaner.clean
end

ts = ThinkingSphinx::Configuration.instance
ts.build
FileUtils.mkdir_p ts.searchd_file_path
ts.controller.index
ts.controller.start
at_exit do
  ts.controller.stop
end
ThinkingSphinx.deltas_enabled = true
ThinkingSphinx.updates_enabled = true
ThinkingSphinx.suppress_delta_output = true

# Re-generate the index before each Scenario
Before do
  ts.controller.index
end

What’s going on here?

Start by commenting out the line about using transactional fixtures in env.rb. Using transactional fixtures will run each scenario inside of a transaction and roll it back at the end of the scenario to revert the database state. Thinking Sphinx uses an after_commit callback for kicking off the delta indexing, but the callback never gets run when transactional fixtures are enabled because the entire scenario is run inside of a big transaction.

Once we’ve disabled transactional fixtures, our test database will start to fill up, likely causing some problems. So we need to add a Before block that clears out the database before each scenario. I’m using database_cleaner, which gives you some different strategies for cleaning the database. Alternatively, the brute-force solution is just to reload the schema before each scenario, but this is slower than truncating the data.

Before do
  ActiveRecord::Base.establish_connection(ActiveRecord::Base.configurations['test'])
  ActiveRecord::Schema.verbose = false
  load "#{RAILS_ROOT}/db/schema.rb"
end

Next, we start Sphinx when env.rb is loaded, and shut it down when the Ruby process exists. We also enable deltas and updates, which are disabled by default in test mode. Finally, we define a Before block that updates the index before each scenario so we don’t end up with a stale index.

Putting it all together

I’m using Sphinx’s delayed delta support, so whenever I update records, I need to have delayed_job process jobs. Instead of trying to get delayed_job to run in the background, I took the easy way out and defined a step: “When the system processes jobs”.

Scenario: Posting a new listing
  Given I am logged in as "MovinMan" 
  When I create a new listing titled "Lots of Boxes" near "49423" 
  And the system processes jobs
  And I browse listings near "49423" 
  Then I can see a listing titled "Lots of Boxes" 

Which is just implemented as:

When 'the system processes jobs' do
  Delayed::Job.work_off
end

If you’re just using the default deltas, and not delayed deltas, then you can update the index like this:

When /^the system updates the index$/ do
  MyModel.sphinx_indexes.first.delta_object.index(MyModel)
end

I hope that helps. Post your suggestions in the comments for improving this.

Unless you’re a Rails genius, you probably need to frequently reference the Rails API docs. And if you haven’t discovered it yet, railsapi.com is awesome.

John Nunemaker suggested that I create a site-specific browser and point it to a local copy of the docs from railsapi.com. I did and have been loving it, so I’m suggesting that you do it too.

Download Fluid (or the comparable app for your platform if you’re not on a Mac). Then download a copy of the docs from railsapi.com and create a new app pointing to that local copy.

Better yet, head over to railslogo.com and grab the Creative Commons licensed logo to use as the icon.

Now my only complaint is that I don’t have docs for Ruby and other gems in this app, but I have a hunch that it won’t be long until that changes.

<infomercial-voice>Stale indexes got you down? Embarrassed that your users’ searches are coming up empty? Act now and you can avoid stale indexes with NEW and IMPROVED delayed delta indexing!!</infomercial-voice>

Ok, maybe it’s not new and improved. It’s actually been around since January, but it’s still awesome. Thinking Sphinx can use delayed_job to keep indexes fresh.

I was slow at jumping on the Sphinx bandwagon for one reason: the index has to be rebuilt to incorporate new data. Delta indexing alleviated some of this by storing frequent changes in a small separate index, but it still had to be occasionally reindexed. It also seemed to only index existing records in my trials with it. New records didn’t ever seem to show up until I rebuilt the whole index.

From what I can tell, delayed delta indexing makes everything Just Work™, and here’s how to use it…

After you’ve setup ThinkingSphinx, set the :delayed property to :delta in your index:

class Listing < ActiveRecord::Base
  define_index do
    indexes title
    indexes description
    indexes user.login, :as => :user

    set_property :delta => :delayed
  end
end

The delayed delta support depends on delayed_job, but if you’re using the gem version, it’s already bundled in. I’m using delayed job for some other things in my project, so I still have it installed separately and that seems to be working just fine.

delayed_job uses a table to keep track of the jobs that need run, so create a migration containing:

create_table :delayed_jobs, :force => true do |table| 
   table.integer  :priority, :default => 0 
   table.integer  :attempts, :default => 0 
   table.text     :handler 
   table.string   :last_error 
   table.datetime :run_at 
   table.datetime :locked_at 
   table.datetime :failed_at 
   table.string   :locked_by 
   table.timestamps 
end

And lastly, all you need to do is fire up the worker process:

$ rake thinking_sphinx:delayed_delta

Now whenever changes are made to your models, the index will be updated moments later. And that’s how you keep it fresh!

Sphinx, everybody’s favorite search engine, has support for location-based search, giving you geo-aware, full-text searching. So now you can find all of the garage sales on Saturday within 20 miles that have LPs and a reel mower.

All you need to do is add latitude and longitude (in radians) to the index, allowing you to limit the results to records within a distance of a point. The hardest part is getting the coordinates, but acts_as_geocodable makes that really easy.

To start, install acts_as_geocodable. Once you have that configured properly, install ThinkingSphinx, define an index on your geocodable model and add the coordinates to the index:

class Listing < ActiveRecord::Base
  acts_as_geocodable

  define_index do
    indexes title
    indexes description

    has geocoding.geocode(:id), :as => :geocode_id
    has 'RADIANS(geocodes.latitude)', :as => :latitude, :type => :float
    has 'RADIANS(geocodes.longitude)', :as => :longitude, :type => :float
  end
end

The three lines that start with has add the geocode id, and the latitude and longitude in radians to the index. Our index doesn’t need the geocode id, but we have to add it so that ThinkingSphinx properly joins the geocodes table.

After you rebuild the index and start the daemon, you can search for records by location. Here’s an example of taking a zip code from a user and finding all records with in 20 miles. (Note: you will need to grab the latest version, 0.2.9, of Graticule for this next bit of code to work)

def search
  location = Geocode.geocoder.locate(params[:zip]).coordinates.map(&:to_radians)
  @listings = Listing.search(params[:q], :geo => location,
    :with => {'@geodist' => 0.0..(20 * 1610.0)})
end

After looking up the coordinates of the zip code that the user entered, we do a search with the :geo parameter, limiting the results using the special @geodist condition. We have to pass in a range of floats that represent the distance of the points in meters (and since the US is in the stone age, I’m converting from miles).

That’s all there is to it. Now go write some cool location-based search and comment here about it.

Last week we launched Idea Foundry, the latest edition of our awesome training. We announced several great classes, including Advanced Rails and jQuery.

Advanced Rails

May 26–29 ● Holland, Michigan

So you drank the koolaid, learned Ruby on Rails and built some awesome sites, now what? Join us for a survey of advanced topics that will take your Rails apps to the next level. Scaling, building and consuming web services, writing plugins, contributing to Rails and so much more. See the Advanced Rails page for more info, and stay tuned for a sneak peek of the topics.

jQuery

May 13–15 ● Holland, Michigan

We’re excited to have Karl Swedberg, author of Learning jQuery and member of the jQuery project team, teaching a class on the ins-and-outs of jQuery. Karl’s an energetic teacher who knows jQuery better than almost anyone (he’s consistently one of the top posters on the jQuery mailing list). More Info

Other Classes

We’re also offering new installments of our popular Ruby on Rails and ExpressionEngine classes.

• ExpressionEngine — May 19–22 in Holland, Michigan
• Ruby on Rails — June 9–12 in Holland, Michigan

And we have one more awesome class that we’ll be announcing any day now. Stay tuned.

For those that haven’t heard: Cucumber is pretty much the greatest thing since sliced bread. It dramatically improved the quality and stability of our applications, and the outside-in approach that it encourages forces you to stay focused on what’s important.

When we started working on a Facebook application a few months ago, we couldn’t fathom not using Cucumber. So we had to figure out a way to test it. It took us a few months to evolve it to a point where we could extract it, but this week we pushed a change to Facebooker to make life a little easier. So grab the latest version of Facebooker and keep on reading…

First, in features/support/env.rb, replace the default Rails world with the one in Facebooker:

# require 'cucumber/rails/world'
require 'facebooker/rails/cucumber'

Given I am logged in as a Facebook user

Most of our Facebook application requires that a user be logged in. So most of our scenarios started with “Given I am logged in as a Facebook user”.

Scenario: Uploading a video
  Given I am logged in as a Facebook user
  When I upload a video
  Then I can see a video on my blog

And here is our implementation for that step:

Given "I am logged in as a Facebook user" do
  @current_user = User.create! :facebook_id => 1
  @current_user.facebook_user.friends = [
    Facebooker::User.new(:id => 2, :name => 'Bob'),
    Facebooker::User.new(:id => 3, :name => 'Sam')
  ]
  @integration_session.default_request_params.merge!(
    :fb_sig_user => @current_user.facebook_id,
    :fb_sig_friends => @current_user.facebook_user.friends.map(&:id).join(',')
  )
end

Our application has a User model with a facebook_id attribute and a #facebook_user method which returned an instance of Facebooker::User. Due to how the Facebook session is being mocked, it is important that we set our fake user’s id to 1 for now (I’ll try to figure out a way around this). We also manually add some friends for our application to use. Lastly, we merge in our user’s id and friend ids into the default request params so that any requests we make include those parameters.

Drop your…canvas

There were some places in our app where requests don’t go through the canvas. For example, we have a few multipart forms, which have to submit directly to your application. To mimic this, wrap webrat calls in #without_canvas:

When "I upload a video" do
  visit root_path

  without_canvas do
    fill_in 'Title', :with => 'A video'
    fill_in 'Description', :with => 'Caption for video'
    attach_file 'Video', "#{RAILS_ROOT}/features/support/sample.mpg", "video/mpeg"
    click_button 'Upload Video'
  end
  follow_redirect!
end

Note that if your action redirects to a URL with :canvas => true, webrat will see that as an “external” redirect and won’t follow it. Just call #follow_redirect! and it’ll go on it’s merry way.

Accessing the Facebook API

Instead of making requests to the Facebook API in your tests, Facebooker will try to read a canned response fixture from features/support/facebook/. It will give you a friendly error whenever this happens, so just follow the directions and you should be on your way.

Feedback & Patches

I’m sure there are things that don’t work right, so let me know if you run into any troubles. If you have any ideas for making the Cucumber support better, please share them here or on the Facebooker mailing list.

The interwebs are all a’buzz about Rack. For those that haven’t been following along, Rack is a specification and a library for connecting web servers to Ruby libraries (a.k.a. “Middleware”). It’s basically the Web 2.0 version of CGI, except that it doesn’t suck and it’s just for Ruby.

Rails 2.3 has Rack baked in. It uses Rack for things like sessions and parameter parsing. But what if you want to add your own middleware to a Rails app?

It’s really easy! In the init.rb of a plugin, or just in a Rails initializer:

ActionController::Dispatcher.middleware.use MyMiddleware

This will append your middleware to the end of the “stack”, so it will be executed after all of Rails’ middleware, but before anything else in the Rails framework.

But what if you need to massage request parameters before Rails parses them? All you need to do is insert your middleware in the stack before Rails parses the params. You can find the current list of middleware by inspecting ActionController::Dispatcher.middleware. In there you’ll find ActionController::ParamsParser, which is what we want to insert our middleware before. Unfortunately, there’s not an insert_before method (yet), so we’ll need to use insert_after, giving it a middleware earlier in the stack:

ActionController::Dispatcher.middleware.insert_after 'ActionController::Failsafe', MyMiddleware

If you don’t like the way that one of the existing pieces of middleware handles things, you can swap it out for your own version:

ActionController::Dispatcher.middleware.swap 'Rails::Rack::Metal', HeavyMetal

So there you have it. Eat, drink, and go write middleware.

Need to force all URLs in a controller or action to be absolute URLs, but don’t want to use the *_url named routes or, even worse, :only_path => false everywhere?

Background

As we were putting the finishing touches on an awesome Facebook app today (yes I said “awesome” in the same sentence as “facebook app”…you’ll see), we needed to make it so that users can add the app as a tab in their profile. Facebook makes this very easy, but there’s an interesting quirk feature. From the developer’s wiki:

• Absolute URLs on a tab take the user to your application’s canvas page.
• Relative URLs are treated as if they are relative to your home canvas page and AJAX-loads in the tab.

We needed all of the URLs on the page to be absolute so that they would take the user out of the tab and into our application. So we were faced with two options:

1. Change all of our named routes from *_path to *_url and use explicit paths instead of polymorphic urls (you know, that thing that lets you do form_for @model, or link_to h(@model.name), @model)
2. Figure out some quick hack that forced Rails to generate absolute URLs on this one page and go on about our lives

The page reused a lot of components from all over the site, so we weren’t excited about option 1. And, like all good developers, we were just adverse to that much work.

So we started looking for a good fit for option 2. We needed to force :only_path to be false on any URL generation. There’s a way to specify default_url_options, but as the name implies, those are only defaults. Using the *_path named routes forces :only_path to true.

Solution

It turns out, Rails passes all URL generation through a method on your controller called #rewrite_options. So we just overrode that method to force :only_path to false on our one action. Here’s the code we used:

class ContentController < ApplicationController
  # …

  def profile
    # profile action
  end

private
  # Force absolute URLs on the profile tab.
  def rewrite_options(options)
    super(action_name == 'profile' ? options.merge(:only_path => false) : options)
  end
end

It’s slightly evil, but it worked beautifully.

My experience is that most people think of “outsourcing” and “offshoring” as synonyms. We talk about the general category of outsourcing with slight repugnance, acknowledging that it is occasionally useful for more mundane and unskilled tasks, but it simply won’t work for the more creative tasks that require an American to do it well. (Maybe nobody blatantly says it requires an American, but we know what you’re implying.)

I don’t intend to address the “Americans are better at creative work” myth, but I will say this: my hypothesis is that any perceived difference in the quality of outsourced work has more to do with the size of the team, access to the stakeholders and end-user, and the ownership and responsibility felt by the people doing the work than it does their nationality, all factors which can also affect an internal team. But I have no experience with offshoring, so I can’t speak to that.

Instead, I want to clarify that offshoring is but one category of outsourcing.

You may not call it “outsourcing”, but you do it all the time

“Outsourcing” simply means paying someone else do work that you could do, usually because they can do it cheaper, better or faster, or any combination of those. We all outsource all of the time. As cofounder of Keepers Household, Inc., I could have our maintenance staff–a position currently held by yours truely–take the time to figure out why the washing machine sometimes doesn’t spin, but not only do I have no interest in this task, I can pay someone else that can diagnose the problem quicker and probably fix it better than I can. I’m exchanging a little money for a working washing machine and a few hours back to do what I’m good at and enjoy¹.

In nearly three years of doing web application development at Collective Idea, I can only think of one or two small projects where we didn’t “outsource” parts of the project. On every other project, we’ve worked in conjunction with one or more companies or individuals on various aspects.

It’s not just about getting it cheaper

Design, for example, is an area that we often outsource. We believe strongly in the importance of good interface design, and doing it as early as possible in the project. We can do design ourselves (or at least pretend to), but it is extremely inefficient for us. It is not a core competency, nor is it an area where we have a real competitive advantage.

So we have two options: hire someone that is good and efficient at design, or outsource it. Most companies choose to hire someone, assuming that they will get a better value and the direct access to the designer will yield quicker and higher quality results.

Instead, we outsource it to one of the designers that we know, and not only is the result much better than we could have managed, it is much cheaper. The result is better because we get to work with better designers—people that we couldn’t afford to employ—and they bring with them the experience of all the other projects that they’ve worked on, successes and failures. While the may bill us at a much higher rate than we could have done it ourselves, in the end, it’s still a better value because they make fewer mistakes and are more efficient.

But ultimately, it’s not about getting design done cheaper, it’s about getting it done better. Cheaper just happens to be a nice side effect. We outsource design because we can’t afford to not do it well, so we want the most creative people we can find.

Not just the mundane

On one end of the spectrum of work is the repetitive, dull tasks that anybody can do. And on the other end is the highly creative or skilled work that only a handful of people can do. I disagree with the notion that only the mundane and uncreative tasks are the candidates for outsourcing. The closer the work is to either edge of that spectrum, the more it could benefit from outsourcing.

Besides design, we’ve also outsourced things that require very specialized knowledge or skills. Occasionally we have interest and can afford to pursue those areas ourselves, but usually it’s more cost-effective and productive to bring in someone that already has experience, for the same reasons we outsource design.

This post is partly inspired by Outsourcing Killed By Django And Ruby On Rails, which argues that the Django and Rails web frameworks have allowed us to compete with the big guns because the frameworks eliminate the 80% of mundane tasks that would otherwise be offshored, allowing us to focus on the creative aspects.

While I agree with their argument that these frameworks eliminate the mundane tasks and allow us to be more efficient, I would argue that more efficiency comes from other factors which have more to do with the culture around these frameworks, such as smaller team sizes, open communication and releasing early which forces you to focus on core features needed by existing users instead of what potential users might want. But I digress…

More importantly, I disagree with their premise that these frameworks have killed outsourcing. From my perspective, they have done exactly the opposite, and it is the ability of small specialized teams to outsource that gives them an even greater edge on the corporate competition.

If frameworks like Django and Rails are killing anything, I would hope that it is the culture of corporate software manufacturing, or the idea that we can just put more cheap labor on the software assembly line and get better results.

1. I’m writing this right now when I should be fixing the washing machine, but I’ve promised my cofounder/business partner that I’ll call someone to fix it.

We’ve been working on a Facebook app (yeah, I know, I’m sorry) and have had all kinds of fun challenges. We wanted to test FBML tags in the view, but the problem is that FBML uses namespaced elements.

It turns out, CSS selectors don’t work with namespaced elements. There is a CSS3 namespace module, but Rails’ #assert_select and RSpec’s #have_tag (built on #assert_select) don’t support them. We were going to have to turn to XPath, but #have_tag only supports CSS selectors.

But Hpricot supports XPath. A little searching revealed the rspec_hpricot_matchers plugin, which replaces have_tag with an Hpricot-backed version. The only problem is that the new version isn’t backwards compatible. So we forked it and renamed #have_tag to #match_element.

Usage

It’s pretty straight forward. Install the rspec_hpricot_matchers plugin, and include the module in the RSpec config in spec_helper.rb.

Spec::Runner.configure do |config|
  config.include RspecHpricotMatchers
end

Our view had something similar to:

<fb:multi-friend-input prefill_ids="<%= @friends.join(', ') %>" />

And in the view spec, we use #match_element with an XPath expression.

describe "things/edit.fbml.erb" do
  it "should prefill selected friends" do
    render "things/edit.fbml.erb"
    response.should match_element("//fb:multi-friend-input[@prefill_ids='333,444,555']")
  end
end

Yes, I’m making the assertion that preordered tree traversal is now cool. And I don’t mean just “pocket protector” cool, because it’s always been that, but now it’s “show your friends” cool.

For those that have no idea what I’m talking about, and don’t really care, but still want to be cool, skip to the next section. For all three of you that want to understand all the gory details, check out this MySQL DevZone article on managing hierarchical data.

What are you talking about?

I’m talking about putting hierarchical data into a relational database, and a plugin to make that easier. There are lots of reasons for trying to do this: organizational structures, genealogies, taxonomies, nested pages of a website, etc. It’s kinda like putting a square peg into a round hole, except that the square peg is made out of Play-Doh, so we can force it through the hole anyway, and we just have a little extra mess to deal with.

There were several Active Record plugins out there that tried to clean up the mess, but they were either buggy or incomplete.

We created awesome_nested_set to try to remedy that.

What makes this so awesome?

There’s a lot of things that makes this awesome, but my personal favorite is that awesome_nested_set makes use of Rails 2.1’s named_scope features¹, so most of the nested set methods return a scope that works as a finder. You can call find methods on it or access other named scopes.

class Department < ActiveRecord::Base
  acts_as_nested_set
  named_scope :in_need_of_review, :lambda => {{
    :conditions => {:reviewed_at > 1.year.ago
  }}
end

chancellor = Department.create(:name => 'Chancellor')
aa = Department.create(:name => 'Academic Affairs').move_to_child_of(chancellor)
Department.create(:name => 'Admissions').move_to_child_of aa
Department.create(:name => 'Student Services',
  :reviewed_at => 3.months.ago).move_to_child_of aa

chancellor.descendants.in_need_of_review
rogue = chancellor.descendants.all(:conditions => 'manager_id IS NULL')

There’s lots more info in the README on GitHub, so check it out. Let us know if you have any suggestions or feedback.

1. It also backports named_scope for those still on Rails 2.0

Git support in Capistrano 2.5 got a little lovin’, but as a result, it may have broke your deploys. By default, Capistrano now uses the -q flag to tell git to not be so chatty. But older versions of git (1.4.x and maybe some early 1.5.x versions) don’t support the -q flag. If upgrading git isn’t an option, the solution is simple. Just tell Capistrano to take the muzzle off of git:

set :scm_verbose, true