opensoul.org:

RSpec is getting too intimate with my code

The theory is that tests are supposed to be agnostic of the implementation. This leads to less brittle tests and actually tests the outcome (or behavior).

With RSpec, I feel like the common approach of completely mocking your models to test your controllers ends up forcing you to look too much into the implementation of your controller.

Here is an example of a spec that is generated by the rspec_scaffold generator:

describe ThingsController, "handling POST /things" do

  before do
    @params = {}
    @thing = mock_model(Thing, :to_param => "1", :save => true)
    Thing.stub!(:new).and_return(@thing)
  end

  def do_post
    post :create, :thing => @params
  end

  it "should create a new thing" do
    Thing.should_receive(:new).with(@params).and_return(@thing)
    do_post
  end

  it "should redirect to the new thing" do
    do_post
    response.should redirect_to(@thing)
  end
end

This by itself is not too bad, but the problem is that it peers too much into the controller to dictate how the model is used. Why does it matter if my controller calls Thing.new? What if my controller decides to take the Thing.create! and rescue route? What if my model has a special initializer method, like Thing.build_with_foo? My spec for behavior should not fail if I change the implementation.

This problem gets even worse when you have nested resources and are creating multiple models per controller. Some of my setup methods end up being 15 or more lines long and VERY fragile.

RSpec’s intention is to completely isolate your controller logic from your models, which sounds good in theory, but almost runs against the grain for an integrated stack like Rails. Especially if you practice the skinny controller/fat model discipline, the amount of logic in the controller becomes very small, and the setup becomes huge.

So what’s a BDD-wannabe to do? Taking a step back, the behavior that I really want to test is not that my controller calls Thing.new, but that given parameters X, it creates a new thing and redirects to it.

Here’s the approach I’ve been taking:

# spec_helper.rb
def valid_thing_attrs(attrs = {})
  {:name => 'Foo'}.merge(attrs)
end

# things_controller_spec.rb
describe ThingsController, "handling POST /things" do

  def do_post
    post :create, :thing => valid_thing_attrs
  end

  it "should create a new thing" do
    lambda { do_post }.should change { Thing.count }.by(1)
  end

  it "should redirect to the new thing" do
    do_post
    response.should redirect_to(@thing)
  end
end

My spec is now testing that posting certain attributes creates a new thing (in the database) and redirects to it. You’ll notice that I have a method called valid_thing_attrs. I don’t remember where I picked up this pattern, but it is something that has allowed me to minimize all of my test’s dependencies on the model.

My spec is now somewhat dependent on my model–and thus will break if my model gets hosed–but that is a small price to pay, in my opinion, for an implementation-agnostic spec with significantly less overhead.

What do you think?

How to test dependencies on external services?

I’ve been writing a lot of code lately that depends on external services, and I’ve really been struggling with how to test it well.

Tinder

The tests for Tinder, the unofficial Campfire “API”, are pathetic. Partially because it was originally just a proof of concept and partially because I just didn’t have a clue about how to write tests that would actually tests and not just execute the code.

The problem with Tinder is that it is really hard to isolate the tests. By default, every test will have 2 points of failure, the code to set up the test and the code to verify the result.

For example, here would be the test for deleting a room:

c = Tinder::Campfire.new 'mycampfire'
c.login('email', 'pass')
room = c.create_room 'My Test Room'
assert room.destroy
assert !c.find_room_by_name('My Test Room')

How do I know that it is working? All I’m testing is that #destroy returns true and that Tinder thinks that it is gone. Is that good enough? To make matters worse, every other test will depend on at least login with whatever other setup code is required.

Ideally, each API method would be tested by making assertions on the HTTP request/response.

Graticule

With Graticule, I decided that instead of requiring API keys for every service and hitting up against them with every test (especially since some services are pay-per-request), I would just save sample responses and mock the HTTP connection. This works really well as far as unit testing goes, but poses problems if services change their APIs.

Ideally, what I need is to be able to run the exact same test suite locally and remotely, but I haven’t figured out a good way to do the setup.

What is a wanna-be tester to do?

I could keep talking about examples (LDAP, CalDAV, etc.), but you get the point.

Testing isn’t supposed to worry about the internals of the code and only test the API, but with something like Tinder, how do you properly test it without peeking into the guts? What are some things you are doing to handle external dependencies?

rspec: model.should be_valid

One of the things I always check when spec’ing is that the model that I just created is valid.

@mymodel.should be_valid

This uses RSpec’s predicate support (any call to be_something? calls something? on the target object), and returns a correct but very unhelpful error:

expected valid? to return true, got false

Here is a little rspec matcher that I threw together this morning to check if a model is valid and spit out the validation errors:

module Spec
  module Rails
    module Matchers
      class BeValid  #:nodoc:

        def matches?(model)
          @model = model
          @model.valid?
        end

        def failure_message
          "#{@model.class} expected to be valid but had errors:\n  #{@model.errors.full_messages.join("\n  ")}"
        end

        def negative_failure_message
          "#{@model.class} expected to have errors, but it did not"
        end

        def description
          "be valid"
        end

      end

      def be_valid
        BeValid.new
      end
    end
  end
end

Now, I get:

MyModel expected to be valid but had errors:
  Password confirmation can't be blank
  Email has already been taken

This is nothing fancy, but it’s the small stuff that make life meaningful, so I thought I would share it. Throw the matcher code above into your spec_helper and check if your models are valid.

update: added negative_failure_message so “model.should_not be_valid” will work

Cerberus has a sense of humor

Build still broken and getting worse

Today Cerberus officially became my tool of choice for continuous integration when it sent me an email with that subject. I’ve been using Cerberus recently and I really like it. It’s simple and it works, which is more than I can say for any other continuous integration tool that I’ve ever used.

Using breakpointer to debug rails tests

I played with breakpointer for a few minutes to check it out when I was first learning Rails, but I’ve never actually used it for serious debugging. But yesterday I had a unit test that was failing and I could not figure why, so I decided I would try it out. I discovered that breakpointer works really well for debugging tests! I had only used it in the context of debugging actions.

Using the breakpointer is really easy, just insert breakpoint at any point in your code. When you run your tests, it will open up an irb session whenever it encounters a breakpoint in your code.

.....Executing break point at ./test/unit/event_test.rb:36 in `test_duration'
irb(test_duration_with_hash=(EventTest)):001:0>