delayed_job 2.0

April 3, 2010 code , popular 2 min read

I’ve pushed out the delayed_job 2.0 gem from the Collective Idea fork on GitHub. See the changelog for a summary of changes, or see the full list changes.

Multiple Backends

One of the most significant changes was adding support for multiple backends. You can now use Active Record, MongoMapper, or DataMapper as backends for your job queue. See the README for more info.

Benchmarks

The Active Record backend in delayed_job 2.0 is much faster (6x in the benchmarks I ran), primarily due to reversing the priority column and adding an index. Here are benchmarks for running 10,000 simple jobs on my laptop:

                      user     system      total        real
delayed_job 1.8.5 195.670000  14.020000  209.690000 (230.887172)
delayed_job 2.0    36.200000   0.940000  37.140000  ( 39.959233)

While we’re looking at benchmarks, here is how the current backends compare:

                     user     system      total        real
active_record      36.200000   0.940000  37.140000 ( 39.959233)
mongo_mapper       69.270000   3.220000  72.490000 ( 90.783220)
data_mapper       255.620000   2.880000 258.500000 (275.550383)

I have not done anything to optimize the mongo_mapper or data_mapper backend, so performance patches would be appreciated.

Upgrading

To take full advantage of the Active Record performance improvements, you’ll want to add an index:

add_index :delayed_jobs, [:priority, :run_at], :name => 'delayed_jobs_priority'

The only other issue that most people will run into is that all of the configuration options have been moved to Delayed::Worker. Here’s how to change the options now:

Delayed::Worker.destroy_failed_jobs = false   # Delayed::Job.destroy_failed_jobs = false
Delayed::Worker.max_attempts = 3              # Delayed::Job.const_set("MAX_ATTEMPTS", 3)
Delayed::Worker.max_run_time = 5.minutes      # Delayed::Job.const_set("MAX_RUN_TIME", 5.minutes)
Delayed::Worker.sleep_delay = 60              # Delayed::Worker.const_set("SLEEP", 60)

Feel free to post any comments or questions on the mailing list.

This content is open source. Suggest Improvements.

— @bkeepers