What makes rspec3 so interesting?

Being an rspec fan I have been waiting for quite some time for the rspec3 final release. rspec3 has finally been released and it’s ready for production use. There are many interesting changes that has been incorporated in rspec3. Thanks to myron, Andy and david and other contributors. Here are the few changes that makes testing more fun:

Changes in rspec-expectations:

Compound Expectations: Composing a expectation using two or more expectations is called compound expectation. Here is the example:

  # In rspec2
  describe String do
    it "expect to be a instance of String and it should be equal to 'RUBY IS AWESOME'"
      string = "RUBY IS AWESOME"

      expect(string).to be_a(String)
      expect(string).to eql("RUBY IS AWESOME")
    end
  end
  # In rspec3
  RSpec.describe String do
    example "expect to be a instance of String and it should be equal to 'RUBY IS AWESOME'"
      string = "RUBY IS AWESOME"

      expect(string).to be_a(String).and eql("RUBY IS AWESOME")
    end
  end

Note: Not only combined pre-defined matchers you can also combined custom matchers also. Below is the example:

  RSpec::Matchers.define :be_a_first do |expected|
    match do |actual|
      actual.first == expected
    end
  end

  RSpec::Matchers.define :be_a_last do |expected|
    match do |actual|
      actual.last == expected
    end
  end

  RSpec.describe Array do
    describe "#sort" do
      example "expect 'AWESOME' be a first and 'RUBY' at last elements respectively" do
        expect("RUBY IS AWESOME".split(" ")).to be_a_first("AWESOME").and be_a_last("RUBY")
      end
    end
  end

Composing Expectation: Composing a expectation with pre-defined expectations or custom expectations so that you can express exactly what you want. Below is the example:

  RSpec.describe Array do
    describe "#sort" do
      example "expect 'AWESOME' be a first and 'RUBY' at last elements respectively" do
        expect("RUBY IS AWESOME".split(" ").sort).to match(be_a_first("AWESOME")).and match(be_a_last("RUBY"))
      end
    end
  end

Changes in rspec-mocks: The following changes would be interesting if you use mocks in your test suite.

Stub chain of methods: Once in a while you might face an issue how to stub chain of methods? In rspec2 you have done as follows:

  describe "String" do
    it "able to call upcase.split.reverse on a string object" do
      string = "ruby is awesome"

      allow(string).to receive(:upcase).and_return(upcase = "RUBY IS AWESOME")
      allow(upcase).to receive(:split).and_return(string_split = ["RUBY", "IS", "AWESOME"])
      allow(string_split).to receive(:reverse).and_return(["AWESOME", "IS", "RUBY"])

      expect(string.upcase.split(" ").reverse).to eql(["AWESOME", "IS", "RUBY"])
    end
  end

While in rspec3 you can do

   describe "String" do
    it "able to call upcase.split.reverse on a string object" do
      string = "ruby is awesome"

      allow(string).to receive_message_chain(:upcase, :split, :reverse).and_return(["AWESOME", "IS", "RUBY"])
      expect(string.upcase.split(" ").reverse).to eql(["AWESOME", "IS", "RUBY"])
    end
  end

Scopes:
Till now you couldn’t stub methods in before(:all) scope. Although you can do in rspec3, rspec team suggested not to use all the times. Here is the example how to use:

   RSpec.describe String do
      before(:context) do
        RSpec::Mocks.with_temporary_scope do
           string = "ruby is awesome"
           allow(string).to receive(:upcase).and_return("RUBY IS AWESOME")
           @hash = string.upcase
        end
      end 

      example "able to split the string" do
        expect(@hash.split(" ")).to eq(["RUBY", "IS", "AWESOME"])
      end 

      example "able to split and reverse" do
        expect(@hash.split(" ").reverse).to eq(["AWESOME", "IS", "RUBY"])
      end
    end

Changes in rspec-rails

In rspec-rails they have changed the structure a little. From rspec3 onwards, rspec team segregated core rspec configuration from rspec-rails configuration. All core rspec configuration like expectations.syntax = :expect, profile_examples = 10 moved to spec_helper and all rspec-rails configuration like use_transactional_fixtures moved to rails_helper. verify partial doubles: Prior to verifying partial double let see what is partial double. Partial double is a real object which acts as a test double in testing. Using verify_partial_doubles option you can cross check that by unknowingly you mock a method which is not there in the class. Here is the example:

  RSpec.describe Array do
    describe "#to_hash" do
      example "convert array to hash" do
        a = [1,2,3]

        allow(a).to receive(:to_hash).and_return({"1" => 0, "2" => 0, "3" => 0})
        expect(a.to_hash).to eql({"1" => 0, "2" => 0, "3" => 0})
       end
     end
   end

   # spec/spec_helper

   config.mock_with :rspec do |mocks|
     mocks.verify_partial_doubles = true
   end

Here is output when I run the specifications.

Failures:

  1) Array#to_hash convert array to hash
     Failure/Error: allow(a).to receive(:to_hash).and_return({"1" => 0, "2" => 0, "3" => 0})
       [1, 2, 3] does not implement: to_hash
     # ./spec/models/person_spec.rb:63:in `block (3 levels) in '

There is no method called to_hash exists on Array class. But I mock that method on that partial double. As method is not defined and I set verify_partial_doubles to true the above spec has failed. Till now we have seen changes in rspec-rails, rspec-mocks and rspec-expectations. Now we will see changes in rspec-core.

If you have observed carefully, I have not used 'it' throughout the post. In rspec3, it is suggested not to use 'it' to define a test case. Instead use 'example' to define a test case as 'it' might deprecated on future versions. Not just 'it', but a few other aliases have been added that make the test cases more readable. In rspec3, before(:example) is now an alias for before(:each) and before(:context) is an alias of before(:all). Not only can you use pre-defined aliases but also define your own aliases. Here is the example:

   RSpec.configure do |config|
     config.alias_example_to :frun, run: true
   end

Now you can use frun to define a test case with metadata as run:true. Here is the example:

   frun "should adds 'run:true' in example metadata" do
     expect(1+1).to eq(2)
   end

Now you can run the test case using run tag as follows:

  $rspec spec/models/person_spec.rb -t run
   Run options: include {:run=>true}
   .

   Finished in 0.02857 seconds (files took 1.1 seconds to load)
   1 example, 0 failures

Apart from few additions there are few things has been removed from rspec3. From no onwards you can’t do the following:

   RSpec.describe "Array" do
     subject { [1,2,3] }

     its(:length) { is_expected.to eq(3)}

     example "contains three items" do
       expect([1,2,3]).to have_at_least(3).items
     end
   end

To make above example work you have to include rspec-its and rspec-collection_matchers gems in your Gemfile. I hope these notable changes in rspec3 help you. Please help me expand this post by commenting on things I may have missed.

Writing unit tests the right way

We all know what unit tests are and which gems we need to use for unit testing. Unit testing means testing a method in isolation. Unit tests are light weight and execute fast when compared to integration tests. You can write unit and integration tests either using rspec or minitest along with any mocking frameworks. Both frameworks have built-in support for mocking frameworks but in rspec they are extracted as a separate gem called rspec-mocks. Here are some pointers

Tests should help in debugging code.

We write tests to check whether we have covered all scenarios for our code. But we forgot to cross-check whether the test cases helps in debugging when an problem raises. Consider the following scenario

  # app/models/project.rb

  class Project
    has_many :buildings
    has_many :units
    has_many :bookings
  end
  # app/models/building.rb

 class Building
   belongs_to :project
   has_many :units
 end
  # app/models/unit.rb

  class Unit
    belongs_to :building
    has_many :bookings
  end
  # app/models/booking.rb

  class Booking
    belongs_to :unit
    belongs_to :project
    has_many :unit_installments
  end

There are 4 kinds of users called as DSGadmin, Partner, Area Sales manager(ASM), Sales Manger(SM) working under same company(technically called as Distributor in my system) are there. Here is the requirement:

  • DSGadmin can see Partner, ASM, SM bookings and his bookings.
  • Partner can see ASM, SM bookings and his bookings
  • ASM can see see SM and his bookings
  • SM can see his bookings only

Now, to test this I wrote some integration tests, thinking that they will be robust and quick.

   describe "Distributor Report", dont_clean: true do
     context "When DSG admin and partner has done a booking each" do
       before(:all) do
         @today = Date.today.to_s

         @project = create(:project)
         @franchise = create(:franchise)
         @dsg_admin = create(:dsg_admin, resource: @franchise)
         @partner = create(:partner, resource: @franchise, parent: @parent)

         @project.distributors = [@franchise]
         @project.save

         create(:distributor_admin_booking, resource: @franchise, sold_by: @dsg_admin, project: @project.id)
         create(:distributor_staff_booking, resource: @franchise, sold_by: @partner, project: @project.id)
       end 

       it "then DSG admin able to see 2 bookings" do
         login(@dsg_admin)

         post sales_reports_path, search: { project_id: @project.id, start_date: @today, end_date: @today, resource_key: "#{@franchise.id}@@Distributor"}

         expect(assigns[:report].first[1]).to eq(2)
       end 

       it "then partner able to see 1 booking" do
         login(@partner)

         post sales_reports_path, search: { project_id: @project.id, start_date: @today, end_date: @today, resource_key: "#{@franchise.id}@@Distributor"}

         expect(assigns[:report].first[1]).to eq(1)
       end
     end
   end
Distributor Report
  When DSG admin and partner has done a booking each
    then DSG admin able to see 2 bookings
    then partner able to see 1 booking (FAILED - 1)

Failures:

  1) Distributor Report When DSG admin and partner has done a booking each then partner able to see 1 booking
     Failure/Error: expect(assigns[:report].first[1]).to eq(1)
       
       expected: 1
            got: 2
       
       (compared using ==)
     # ./spec/requests/distributor_reports_spec.rb:172:in `block (3 levels) in '

Finished in 2.17 seconds
2 examples, 1 failure

As we can see, when I ran tests then one of my integration tests failed. However, it did not tell me where things failed. Now I realized the need to write some good unit tests and got down to it.

  describe "#b_match" do
    context "should return criteria as" do
      before do
        @sales.stub(:c_project).and_return(nil)
        @sales.stub(:resource).and_return(nil)
        @sales.stub(:default_match).and_return({'deleted_at' => nil})
      end 

      it "returns {'deleted_at' => nil} if no project or resource has selected" do
        @sales.b_match
        expect(@sales).to have_received(:default_match)
      end 

      it "returns {{'deleted_at' => nil, sold_by_id: {'$in' => @user.assigned_users_ids}} if selected resource is distributor" do
        expect(@sales.b_match).to eq('$match' => {'deleted_at' => nil, sold_by_id: {'$in' => @partner.assigned_users_ids}})
      end
    end
  end
  Sales 
    #b_match 
      returns criteria as {{'deleted_at' => nil, sold_by_id: {'$in' => @user.assigned_users_ids}} if selected resource is distributor


Failures:

  1) Sales#b_match should return criteria as returns {{'deleted_at' => nil, sold_by_id: {'$in' => @user.assigned_users_ids}} if selected resource is distributor
     Failure/Error: expect(@sales.b_match).to eq('$match' => {'deleted_at' => nil, sold_by_id: {'$in' => @partner.assigned_users_ids}})
       
       expected: {"$match"=>{"deleted_at"=>nil, :sold_by_id=>{"$in"=>["5378c281472d208bc100000d"]}}}
            got: {"$match"=>{"deleted_at"=>nil}}
       
       (compared using ==)
       
       Diff:
       @@ -1,2 +1,2 @@
       -"$match" => {"deleted_at"=>nil, :sold_by_id=>{"$in"=>["5378c281472d208bc100000d"]}}
       +"$match" => {"deleted_at"=>nil}
       
     # ./spec/reports/sales_spec.rb:75:in `block (4 levels) in '

Finished in 0.45649 seconds
9 examples, 1 failure

Now, with this failure, I found the issue in b_match method. In the fix below, it was obvious that the I required the `merge!` instead of the `merge` method. However, the integration tests never pointed to this particular failure – the unit test did!

  def b_match
     match = default_match
     if @current_resource.class == Distributor
+      match.merge!(sold_by_id: {'$in' => @user.assigned_users_ids})
-      match.merge(sold_by_id: {'$in' => @user.assigned_users_ids})
     end
     {'$match' => match}
   end

Unit tests should be independent

Unit tests should be less tightly coupled to avoid dependency. Consider the scenario where we need to calculate the compound interest if a customer didn’t pay their instalment on time. Here is a code snippet

  # app/models/unit_installment.rb

  class UnitInstallment
    field :percentage
    field :amount, type: Float, default: 0
    field :order, type: Integer
    field :received_amount, type: Float, default: 0
    field :interest_amount, type: Integer, default: 0
    field :is_due, type: Boolean, default: false

    belongs_to :booking

    before_create :initialize_is_due

    def initialize_is_due
      self.is_due = unit.building.building_schemes.where(payment_schedule_id: payment_schedule_id, scheme_id: scheme_id).first.try(:due)
      make_due
    end
  end

Now I have written the code to calculate the interest for a installment amount. Here is my test case to check interest calculation:

require 'spec_helper'

describe InterestAmount do
  before do
    UnitInstallment.any_instance.stub(:initialize_is_due).and_return(true)
    UnitInstallment.any_instance.stub(:update_booking).and_return(true)
    UnitInstallment.any_instance.stub(:valid?).and_return(true)
  end 

  let!(:booking) { create(:org_booking) }
  let!(:installment) { create(:unit_installment, booking: booking) }
  let!(:project) { create(:project) }

  context "#calculate" do
    it "should update interest amount on installment" do
      interest = InterestAmount.new(installment: installment, project: project, booking: booking)
      interest.stub_chain(:interest, :round).and_return(100)
      interest.calculate

      expect(installment.reload.interest_amount).to eq(100)
      expect(interest).to have_received(:interest)
    end
  end

Notice that, I have stubbed before_create callback which deals with unit, payment_schedule, scheme objects. So if either unit, payment_schedule or scheme are invalid then I can’t create unit installment object which eventually can’t check interest calculation.

I hope it helps you understanding how to write unit tests that help in debugging.

Handy gems for minitest-rails part2

Mocks and stubs are not new concepts that have been introduced in minitest. If you want to get detailed understanding about what are mocks and stubs you should read mocks aren’t stubs written by Martin Fowler. To achieve mocking and stubbing in minitest-rails you don’t need to include any separate gem like rspec-mocks in rspec. Let see how to do work with mocks and stubs in minitest then check what are gems available to add extra functionality.

Method stubbing
Stubbing a method means, set a pre-defined return value of a method in test-case. Let’s check how to do method stubbing in minitest:

  # app/models/user.rb
  class User < ActiveRecord::Base
    def foo
      "Executes foo"
    end

    def bar
      "Executes bar"
    end

    def new_method
      foo
      bar
    end
  end
  it "stubbing known method" do
    user = User.new
    user.stub(:foo, "Return from test foo") do
      user.new_method.must_equal "Executes bar"
    end
  end

In this example, I stub foo and check what would be the output of new_method. As matter of fact it won’t call actual foo method instead it calls stub foo method. Now that we have  seen how to stub and instance method, let check how to stub on any instance.

minitest stub any instance:

Using this gem you can stub a method on any instance of a class. The below example demonstrates how to do that:

  # app/models/user.rb
  class User < ActiveRecord::Base
    def foo
      "Executes foo"
    end

    def bar
      "Executes bar"
    end

    def new_method
      foo
      bar
    end
  end
  it "stubbing on any instance" do
    User.stub_any_instance(:foo, "Return from test foo") do
      p User.new.foo
      User.new.foo.must_equal "Return from test foo"
    end
    p User.new.foo
  end

Output:
1 tests, 1 assertions, 0 failures, 0 errors, 0 skips
Run options: –seed 36985

# Running tests:

“Return from test foo”
“Executes foo”

Notice that the stub method on any instance of User is valid only in the stub_any_instance block.

minitest-stub-const:

As name suggests you can stub constants of a class. Here is the example:

  it "stubbing a constant" do
    User.stub_const(:CONSTANT, "Test constant") do
      User::CONSTANT.must_equal "Test constant"
    end
  end

Here CONSTANT is a constant defined on User class. Using minitest-stub-const we can not only stub constants but also class methods. Here is the example:

  # app/models/user.rb
  def self.add
    "User created on development"
  end

  def add_user
    User.add
  end
  it "stub a class method" do
    m = MiniTest::Mock.new
    m.expect(:add, "Created user from testing")

    User.stub_const(:User, m) do
      User.new.add_user.must_equal "Created user from testing"
    end

    m.verify
  end

In above example, I stub User with mock object m. Now whenever User is referred, it is replaced with the mock object m. Now, we call the method add on the Mock object but this method is already mocked using m.expect(:add, "Created user from testing").

Mocking:

Mocking an object is one of the ways to do unit testing. Mock objects are the objects that we use in a test environment instead of real objects. We can create mocks using double in rspec to create dummy objects, where as in minitest you can do as Minitest::Mock.new which creates mock object. On mock object we can set expected messages. Here is an example:

  it "mocking a method" do
    user = Minitest::Mock.new
    user.expect(:another_method, "from test bar")

    user.another_method.must_equal "from test bar"

    user.verify
  end

You can also mock objects using minitest firemock as follows:

  it "mocking a method" do
    user = Minitest::FireMock.new("User")
    user.expect(:another_method, "from test bar")

    user.another_method.must_equal "from test bar"

    user.verify
  end

The only difference between Minitest::Mock and Minitest::FireMock is, when you try to mock undefined method then Minitest::FireMock raises an exception whereas Minitest::Mock won’t.

Hope this helps while you writing mocks and stubs in minitest. Any suggestions, queries would be welcome.

Handy gems for minitest-rails

Welcome to ‘minitest’ world! Minitest is yet another ruby testing framework but it’s not a replacement to rspec. In rspec you can readily use things like ‘subject’, ‘metadata’ etc. However, minitest, as the name suggests, is a light-weight testing framework and does not include a lot of these features automatically. Here is a list of gems that you can use to enhance minitest-rails.

1. m

m is a test runner similar to rspec command in rspec framework. Using this we can run single test using line numbers too. Users who don’t like this gem can also run tests using default rake tasks provided by minitest.

2. minitest-metadata

metadata is one of the cool features in rspec. Using this we can tag scenarios across files. Unfortunately minitest doesn’t support this feature by default. We have to include a separate gem called minitest-metadata. We can implement rspec conditional hooks functionality through this gem as follows:

  require "test_helper"

  describe UsersController do

    before do
      @user = create(:user) if metadata[:before_each] == true
    end

    it "must create user" do
      assert_difference('User.count') do
        post :create, user: attributes_for(:user)
      end

      assert_redirected_to user_path(assigns(:user))
    end

    it "must show user", before_each: true do
      get :show, id: @user
      assert_response :success
    end
  end

When we run the file before code block will run only for the latter test case.

3. minitest-implicit-subject

subject is one of the best ways to dry up your specs. To dry up your specs in minitest you have explicity include a gem called minitest-implicit-subject. Here is an example:

  require "test_helper"

  describe User do
    it "foo returns bar" do
      subject.must_equal User
    end
  end

There is slight difference that I have observed in minitest subject from rspec subject. In minitest if you write subject it returns current class name under which it has included. In this scenario it returns output as User class not User object. But in rspec it returns User object.

4. minitest-spec-expect

rspec expectation framework lets you code example more readable. In minitest there is separate gem called minitest-spec-expect available to make your code example readable as same as in rspec. Here is an example:

  require "test_helper"

  describe User do
    it "must be valid" do
      expect(subject.new.valid?).to_equal true
    end
  end

5. minitest-rails-capybara

Using capybara is one of the ways to implement your integration test suite in rspec. We need to include separate gem called minitest-rails-capybara in minitest to write integration tests using the capybara DSL. Here is a simple example:

   require "test_helper"

   feature "User" do
     scenario "should be able save" do
       visit users_path
       click_link 'New User'
       fill_in "user_first_name", with: 'Siva'
       click_button 'Create User'

       expect(User.all.count).to_equal 1
     end
   end

In my next post I will talk about the gems that allow us to mock and stub requests in minitest.

Pro tips for writing better rspec tests

Writing ‘good code’ is always a challenge for every project. Unfortunately, we always associate it with development and not tests. What about automation test code? Ever noticed that you have to write more automation code than the actual lines of development code?

Improper automation code leads to slow running specs, huge LOC in your spec files and a whole lot of confusion when you open a spec file. So, how does one avoid these problems?

Here are a few guidelines that I learned from my experience:

Tip #1: Follow Rspec style guide

That was obvious, wasn’t it? As surprising as it may seem, we invariably do not follow in detail.

  • How often do you see more than one expectation in an example? bad
  • How often do you see lots of stubs and mocks in test code? bad – keep it as simple as you can.
  • How often do you see the use of context, especially in controllers? good

A lot of them are mentioned in the Style guide and we shall repeat below a few which require more details.

Tip #2: Independent tests

We could use the word idempotent here but as a thumb rule there should not be any dependencies across specs and if we run the test multiple times in succession, it should give the same result So, it’s very important that every spec  run in clean database environment. Use database cleaner gem properly, so that you don’t end of having to seed the database before each test! So, if you don’t want roles and countries being deleted,

config.before(:suite) do
  DatabaseCleaner.strategy = :truncation, {:except => %w[roles countries]}
end

Tip #3: Use expect and not should syntax

It’s just nice and clean!  Using the Transpec gem helps you to convert should syntax to expect syntax

expect(User.all.count).to eq(1)

instead of

User.all.count.should == 1

Tip#4: Use the new syntax for object creation

Use the new FactoryGirl syntax that is supported in factory_girl4. This helps the code to be clean of any harsh gem dependent syntax of class names!

create(:user)

instead of

FactoryGirl.create(:user)

Tip #5: Use let variables instead of instance variables

This makes code look cleaner. For better understanding about let read the answer given by Myron. Here is an example

describe Project do
  let(:project) { build(:project) }

  it "project should be valid" do
    expect(project.valid?).to eq(true)
  end
end

Now, when the example executes it defines a method called project if its not defined already or simply returns the value. Fast and efficient.

Tip #6: Use correct expectations for a spec

Obvious as it seems – the test we write should test the write stuff – not just make the test pass! Here is an example of making a test case pass.

context "#edit" do
  it "user should be able to edit his profile" do
    get :edit, id: user.id
    expect(response).to be_success
  end
end

So, your expectation should ideally be

describe UsersController do
  context "#edit" do
    #render_views
    it "user should be able to edit his profile" do
      get :edit, id: user.id
      expect(response).to render_template("edit")
    end
  end
end

Here #edit is a method defined in UsersController. So I defined a context in my specs.

Do remember that by default all controller specs stub the template rendering unless and until you include render_views

Tip #7: Use shared examples.

When you want to test the same functionality across different subjects, for example different kinds of users, use shared examples.

context "#edit" do
  it "admin should be able to edit his profile"
  it "distributor should be able to edit his profile"
end

You can write as

context "#edit" do
  shared_examples "profile_editing" do
    it "user should be able to edit his profile"
  end

  context "admin" do
    include_examples "profile_editing"
    it "should be able see to list of distributors"
  end

  context "distributor" do
    include_examples "profile_editing"
    it "should not able to see list of distributors"
  end
end

A word of caution – you may loose readability if you use too many shared examples in single file.

Tip #8: Use conditional hooks

Sometimes if you want to execute some piece of code only for one example but not for all examples in a context then you can use conditional hooks.

context "#update" do
  before(:each, run: true) do
    # do something
  end

  it "example1", run: :true
  it "example2"
end

Now before(:each) will run only for first example but not for second example.

Tip #9: Profiling your examples

rspec gives you an option to examine how much time does a example group take to run.  Simply run specs with the –profile flag.

rspec ./spec/models/user_spec.rb:51 --profile
# output:
0.22431 seconds ./spec/models/user_spec.rb:51

Tip #10: Tagging

When you have a large test suite and you changed some code, how do you ensure that your change didn’t breaks existing functionality? One way is to run your specs locally then push then your Continuous Integration server that runs all specs. This is time consuming if you want to test only a specific set of tests.

Tag them! Run your module specified specs by using tags. For example, suppose we have a Commission feature and we have written model specs, controller specs and integration specs for it. We can easily tag each file with commission:true.  So when we change something related to commissions,  we can run only commission related specs!. We can tag our scenarios as below

   # spec/models/commission_spec.rb
   describe Commission, commission: true do
     # do something
   end
rspec --tag commission

Note: We can tag entire context or a single example in the spec file. For more information please follow this link

All these tips are a results of practical problems I have faced and overcome and a result of finding out the best we can do to improve our own testing. More suggestions and tips are welcome!