Effective querying using SQL over Rails

Most of the rails developers write queries on daily basis. Some on the other day by mistake we follow rails way instead of SQL way to do tasks like sorting etc. But you might ask what’s the problem regarding which approach I take as long as I am able to get correct results. Yes, you may get correct results but you will end up with performance problems in future. Following a set of scenarios I am observed that SQL querying out powered rails:

1) Apply sorting on a combination of different tables:

I have three tables namely deposits, purchases and withdraws. Now, I want to display latest three records from a combination of three tables.

Let’s check rails way

  Benchmark.ms {(Deposit.all + Purchase.all + Withdraw.all).sort{|x, y| x.updated_at <=> y.updated_at}}

  Deposit Load (0.6ms)  SELECT "deposits".* FROM "deposits"
  Purchase Load (0.4ms)  SELECT "purchases".* FROM "purchases"
  Withdraw Load (0.7ms)  SELECT "withdraws".* FROM "withdraws"
 => 121.91500002518296

Let’s check SQL way

   (SELECT 'deposit', authorization_code, amount, status, updated_at
      FROM DEPOSITS
      ORDER BY updated_at DESC
      LIMIT 3)

      UNION

      (SELECT 'purchase', ref, amount, status, updated_at
      FROM PURCHASES
      ORDER BY updated_at DESC
      LIMIT 3)

      UNION

      (SELECT 'withdraw', ref, amount, status, updated_at
      FROM WITHDRAWS
      ORDER BY updated_at DESC
      LIMIT 3)

      ORDER BY updated_at DESC
      LIMIT 3

  => SQL (2.0ms)

2) Apply sorting on different types of data:

Let’s say a table stores gifs, videos of different duration. Now I want sort videos, gifs in ascending order of content duration separately.

Let’s check rails way

   Benchmark.ms do
      videos = SpotBooking.where(content_type: 'video').order('content_duration DESC')
      gifs = SpotBooking.where(content_type: 'gif').order('content_duration DESC')
      videos + gifs
   end

    SpotBooking Load (0.4ms)  SELECT "spot_bookings".* FROM "spot_bookings" WHERE      "spot_bookings"."content_type" = $1 ORDER BY content_duration DESC  [["content_type", 3]]

  SpotBooking Load (0.4ms)  SELECT "spot_bookings".* FROM "spot_bookings" WHERE "spot_bookings"."content_type" = $1 ORDER BY content_duration DESC  [["content_type", 2]]

 => 12.477000011131167

Let’s check SQL way

   SELECT id, content_duration, row_number()
   OVER (PARTITION BY content_type ORDER BY content_duration DESC)
   FROM spot_bookings

  => SQL (0.8ms) 

3) Formatting the data while querying:

Let’s check rails way:

  Benchmark.ms do
     h = {}
     pricings = SpotPricing.order('starts_at ASC')

     pricings.each do |b|
       h[b.starts_at.strftime("%H %M %P")] = b.price.to_f
     end
  end

  SpotPricing Load (0.5ms)  SELECT "spot_pricings".* FROM "spot_pricings" ORDER BY starts_at ASC

 => 35.459000151604414

    # output
 => {"02:19 AM"=>"1.00", "03:19 AM"=>"2.00", "04:19 AM"=>"3.00",
     "05:19 AM"=>"4.00", "06:19 AM"=>"5.00", "07:19 AM"=>"6.00", "08:19 AM"=>"7.00",
     "09:19 AM"=>"8.00", "10:19 AM"=>"9.00", "11:19 AM"=>"10.00", "12:19 PM"=>"11.00", 
     "13:19 PM"=>"12.00", "14:19 PM"=>"13.00", "15:19 PM"=>"14.00", "16:19 PM"=>"15.00", 
     "17:19 PM"=>"16.00", "18:19 PM"=>"17.00", "19:19 PM"=>"18.00", "20:19 PM"=>"19.00", 
     "21:19 PM"=>"20.00", "23:19 PM"=>"22.00", "00:19 AM"=>"23.00"}

Let’s check SQL way:

  SpotPricing.order('starts_at ASC').pluck("to_char(starts_at, 'HH24:MI AM')", "to_char(price, 'FM999999999.00')")
  
  SELECT to_char(starts_at, 'HH24:MI AM'), to_char(price, 'FM999999999.00') FROM "spot_pricings" ORDER BY ASC

  => SQL (0.5ms)

 => {"02:19 AM"=>"1.00", "03:19 AM"=>"2.00", "04:19 AM"=>"3.00",
     "05:19 AM"=>"4.00", "06:19 AM"=>"5.00", "07:19 AM"=>"6.00", "08:19 AM"=>"7.00",
     "09:19 AM"=>"8.00", "10:19 AM"=>"9.00", "11:19 AM"=>"10.00", "12:19 PM"=>"11.00", 
     "13:19 PM"=>"12.00", "14:19 PM"=>"13.00", "15:19 PM"=>"14.00", "16:19 PM"=>"15.00", 
     "17:19 PM"=>"16.00", "18:19 PM"=>"17.00", "19:19 PM"=>"18.00", "20:19 PM"=>"19.00", 
     "21:19 PM"=>"20.00", "23:19 PM"=>"22.00", "00:19 AM"=>"23.00"}

In all above scenarios, SQL plays best compare to conventional rails way. Sometimes I would feel it’s not a big deal if you not follow design patterns as long as your ability to provide the best outcome.

Any suggestions and feedback would be welcome!!!

Happy querying!!!

Advertisements

Live streaming using ROR, cloud front and S3

Streaming videos are one of the difficult jobs for many developers. Compare to images and remaining media formats videos are large in size. Hence, it is a frustrating experience for user waiting for long time to download the entire video. Then how to solve this problem? Here comes streaming. Streaming means constantly sending data into small chunks to end-user continuously to avoid problems mentioned above. But how to implement this solution. Here comes Amazon cloud front. As per Amazon here is the definition

“Amazon CloudFront is a global content delivery network (CDN) service that accelerates delivery of your websites, APIs, video content or other web assets through CDN caching. It integrates with other Amazon Web Services products to give developers and businesses an easy way to accelerate content to end users with no minimum usage commitments.”

Using cloud front we can stream the video content that has been stored on S3. Below are the steps to follow:

Creating Cloud front distribution:

First sign in into AWS account, then go to cloud front under Content & Network delivery section. Since we are doing live streaming we need to register a web distribution. Two options are most important while creating a distribution. 1) Origin Domain name and 2) SSL certificate. Origin Domain name is nothing but S3 bucket where your media content has been stored. For SSL certificate I would prefer to go with Default CloudFront Certificate since we want to stream through cloud front instead of custom domain. For remaining, you can go with default options.

Origin settings:

Since we want to stream the content through cloud front we need to restrict S3 bucket access to cloud front user only. To make this happen we need to add restriction policy to our S3 bucket through policy generator. Once you added a policy, it would available in your origin identities. By adding this we can restrict S3 bucket will be accessible through cloud front only.

For more details about configuration go through this video tutorial

Generating streaming URLs:

1) Create cloud front key pair to create cloud front URLs. You can create this by using cloud front key pairs under My security credentials.

2) Add aws-sdk to your Gemfile

3)

      # app/controllers/streams_controller.rb

      # To create signed cloud front URLs. Using this we can restrict video access.
      signer = Aws::CloudFront::UrlSigner.new(key_pair_id: <cloud front access ket>,
                                            private_key_path: <cloud front public key>)

      @url = signer.signed_url(<cloud front URL>/<Name of the video which you want to stream>, 
                               expires: Time.zone.now + 10.minutes)
       
   

Note: You can get cloud front URL from cloud front distribution.

Streaming:

To stream video, I am using html5box player.

# app/views/streams/show.html.erb

<div style="display: block; margin: 0px auto; position: relative; max-width: 100%; height:700px;"   class="html5gallery" data-responsive="true" data-responsivefullscreen="true" data-html5player="true" data-skin="darkness" data-width="800" data-height="450" data-showtitle=="false" data-autoplayvideo="true"  data-autoslide="true" data-effect="slide" data-showimagetoolbox="none" data-showplaybutton="false" data-showprevbutton='false' data-shownextbutton='false' data-showcarousel='false'>

  <a href="<%= @url %>"><img src="http://img.youtube.com/vi/YE7VzlLtp-4/2.jpg"></a> 
</div>

Note: html5box not only renders video but also it renders images & gifs as well.

Once you access show page then it starts streaming through cloud front. You can verify from your network tab of your browser.

Happy streaming!!!

seldom used but powerful aggregation operators

Most programmers who use MongoDB may have heard about its aggregation framework but only a few have used it to its full potential. Recently, in one of our projects, we got a complex requirement where the user needed to generate a report of sales done for any day, month or year. Using just MongoDB queries would have been very slow compared to the aggregation framework. So I decided to take up the challenge and learn the aggregation operators and make my code awesome! Here are the operators that helped me:

  # example we would refer throughout our post:

  class Booking
    include Mongoid::Document
    include Mongoid::TimeStamp

    field :agreement_amount, type: Float, default: 0.0
    field :project_name, default: ''
    field :is_active, type: Boolean, default: false
    field :amount_collected, type: Float, default: 0.0
    field :amount_refunded, type: Float, default: 0.0
    field :amount_dishonored, type: Float, default: 0.0
    field :due_installment_amount, type: Float, default: 0.0

    belongs_to :offer
  end

1. $and

$and is one of the MongoDB aggregation conditional operators where you can specify multiple conditions. Any document which satisfies all these conditions would get forwarded to next stage in the pipeline which is nothing but $group stage. Here is an example

# Today's sales value

{ 
  '$project' => 
     'todaysales' =>                                                                                   
         { '$cond' =>                                                                                    
            [                                                                                             
              { '$and' =>                                                                                 
                [                                                                                         
                  { '$gte' => ['$created_at', Time.zone.now.beginning_of_day.mongoize] },                      
                  { '$lte' => ['$created_at', Time.zone.now.end_of_day.mongoize] }
                  { '$eq' => ['$is_cancelled', false] }                                                   
                ]                                                                                         
              }, '$agreement_amount', 0                                                                   
            ]                                                                                             
         },  
     }
}

The above example returns the $agreement_amount if it is today’s booking.

2. $add and $subtract

Similar to conditional operators, we have arithmetic operators where you can do addition or subtraction. Here is an example:

{ 
  '$project' => 
     'yearlycollection' =>                                                                                   
         { '$cond' =>                                                                                    
            [                                                                                             
              { '$and' =>                                                                                 
                [                                                                                         
                  { '$gte' => ['$created_at', Time.zone.now.beginning_of_year.mongoize] },                      
                  { '$lte' => ['$created_at', Time.zone.now.end_of_day.mongoize] }
                  { '$eq' => ['$is_cancelled', false] }                                                   
                ]                                                                                         
              }, "$subtract" => [ { "$amount_collected", "$add" => [ "$amount_refunded", "$amount_dishonoured" ] } ] }, 0                                                                   
            ]                                                                                             
         },  
     }
}

In the above example, if the condition has been satisfied then it adds the $amount_refunded and $amount_dishonored and subtracts the result from the $amount_collected, otherwise it returns 0.

$ifNull:

This is similar to the conditional operator with a short syntax. It returns the value if value is null. Here is an example:

  {
    $project: {
      due_installment_amount: { $ifNull: [ "$offer_id", "$due_installment_amount" ] }
    }
 }

In the above example if offer_id is Null, only then does it return the due_installment_amount of that document.

4. $let:

Can we store an aggregated value in a variable and use in next expression? Yes! $let lets us do just that ūüôā


{ 
  '$project' => 
    {
     'monthlycollection' =>                                                                                  
       {
         '$let' => { 
             collection: { 
                  $cond' =>                                                                                    
                    [                                                                                             
                     { '$and' =>                                                                                 
                         [                                                                                         
                           { '$gte' => ['$created_at', Time.zone.now.beginning_of_year.mongoize] },                      
                           { '$lte' => ['$created_at', Time.zone.now.end_of_day.mongoize] }
                           { '$eq' => ['$is_cancelled', false] }                                                   
                         ]                                                                                         
                     }, "$subtract" => [ { "$amount_collected", 
                                           "$add" => [ "$amount_refunded", "$amount_dishonored" ] } ] }, 0                                                                   
                   ] 
             }
          }
          in: { $cond: { '$gt': ['$$collection', 0] }, '$$collection', 0 }
       }
    }
}
        

In the above example, if amount_collected - (amount_refunded + amount_dishonored) is less than 0 for a particular booking then it returns 0 instead of -ve value. Notice that $$ is the convention to access variables within the aggregation pipeline.

Most of these seem like standard conditional operators but their uses are phenomenal. From MongoDB 3.2 onwards, aggregation framework provides next generation mongodb operators like $lookup (performs left outer join), $switch and others that we shall discuss in my next post.

Any suggestions would be welcome.

Enforcing coding guidelines using CircleCI and Rubocop

Being a rubyist we don’t just write code, we write beautiful code. But sometimes, in a hurry to complete the requirement some developers, especially new comer, might not follow the coding guidelines which would become headache later. Rubocop is here to rescue us from this scenario.

Let’s see how to integrate rubocop:

1. Add rubocop and generate config file:

      # Gemfile
      gem "rubocop", require: false
  
      # .rubocop.yml

      AllCops:
        Exclude:
          # You could specify set of files or dirs while you want to be ignored by rubocop
          - 'vendor/**/*'
          - 'spec/**/*'
          - 'config/**/*'
          - 'db/**/*'
          - 'lib/**/*'
          - 'bin/*'
          - 'Gemfile'
          - 'Gemfile.lock'
          - 'Rakefile'
      Documentation: # Disabling documentation cop
        Enabled: false
    

Here is the file where you can find list of cops that rubocop uses.

2. Run rubocop or through CircleCI:

      $ bundle exec rubocop -R

      Inspecting 47 files
        ...............................................

        47 files inspected, no offenses detected
   

where -R option make rubocop to run rails cops as well.

     #circle.yml

     test:
       post: 
         - bundle exec rubocop -R
   

This would run rubocop upon every commit on CircleCI. Though your test cases have been passed but if rubocop fails then your build fails.

Now let’s see how rubocop enforces you write to better & clean code:

Scenario-1: Using else with unless:

     # Before 
     unless @new_unit.open?
       'Unit is not available!!' 
     else 
       'Remark is empty!!' 
     end 
     
     # Comply to rubocop
     if @new_unit.open?
       'Remark is empty!!'
     else 
       'Unit is not available!!' 
     end
 

As a matter of fact, using else with if would make more sense than using with unless.

Scenario-2: Use next to skip iteration:

     # Before 
     @payment_vouchers.each do |pv|
       if pv.items.any?
         # code here
       end
     end
     
     # Comply to rubocop
     @payment_vouchers.each do |pv|
       next unless pv.items.any?
       # code here
     end
 

As you see after using next, code is more readable than before.

Rubocop not only highlights ruby code related issues but also pin points rails related issues as well.
Scenario-3: Use find_by instead of where.first:

     # Before 
     addr = addresses.where(type: 'primary').first
     
     # Comply to  rubocop
     addr = addresses.find_by(type: 'primary')
 

Scenario-4: Use proc instead of Proc.new:

     # Before 
     validates :address, presence: true, if: Proc.new{ |addr| add.type == 'primary' }
     
     # Comply to rubocop
     validates :address, presence: true, if: -> (addr) { addr.type == 'primary' }
 

Rubocop not only helps in covering all scenarios also insist on coding guidelines. Hope this would be helpful. Any feedback or suggestions are welcome.

What makes rspec3 so interesting?

Being an rspec fan I have been waiting for quite some time¬†for the rspec3 final release. rspec3 has finally been released and it’s ready for production use. There are many interesting changes that has been incorporated in rspec3. Thanks to myron, Andy and david and other contributors. Here are the few changes that makes testing more fun:

Changes in rspec-expectations:

Compound Expectations: Composing a expectation using two or more expectations is called compound expectation. Here is the example:

  # In rspec2
  describe String do
    it "expect to be a instance of String and it should be equal to 'RUBY IS AWESOME'"
      string = "RUBY IS AWESOME"

      expect(string).to be_a(String)
      expect(string).to eql("RUBY IS AWESOME")
    end
  end
  # In rspec3
  RSpec.describe String do
    example "expect to be a instance of String and it should be equal to 'RUBY IS AWESOME'"
      string = "RUBY IS AWESOME"

      expect(string).to be_a(String).and eql("RUBY IS AWESOME")
    end
  end

Note: Not only combined pre-defined matchers you can also combined custom matchers also. Below is the example:

  RSpec::Matchers.define :be_a_first do |expected|
    match do |actual|
      actual.first == expected
    end
  end

  RSpec::Matchers.define :be_a_last do |expected|
    match do |actual|
      actual.last == expected
    end
  end

  RSpec.describe Array do
    describe "#sort" do
      example "expect 'AWESOME' be a first and 'RUBY' at last elements respectively" do
        expect("RUBY IS AWESOME".split(" ")).to be_a_first("AWESOME").and be_a_last("RUBY")
      end
    end
  end

Composing Expectation: Composing a expectation with pre-defined expectations or custom expectations so that you can express exactly what you want. Below is the example:

  RSpec.describe Array do
    describe "#sort" do
      example "expect 'AWESOME' be a first and 'RUBY' at last elements respectively" do
        expect("RUBY IS AWESOME".split(" ").sort).to match(be_a_first("AWESOME")).and match(be_a_last("RUBY"))
      end
    end
  end

Changes in rspec-mocks: The following changes would be interesting if you use mocks in your test suite.

Stub chain of methods: Once in a while you might face an issue how to stub chain of methods? In rspec2 you have done as follows:

  describe "String" do
    it "able to call upcase.split.reverse on a string object" do
      string = "ruby is awesome"

      allow(string).to receive(:upcase).and_return(upcase = "RUBY IS AWESOME")
      allow(upcase).to receive(:split).and_return(string_split = ["RUBY", "IS", "AWESOME"])
      allow(string_split).to receive(:reverse).and_return(["AWESOME", "IS", "RUBY"])

      expect(string.upcase.split(" ").reverse).to eql(["AWESOME", "IS", "RUBY"])
    end
  end

While in rspec3 you can do

   describe "String" do
    it "able to call upcase.split.reverse on a string object" do
      string = "ruby is awesome"

      allow(string).to receive_message_chain(:upcase, :split, :reverse).and_return(["AWESOME", "IS", "RUBY"])
      expect(string.upcase.split(" ").reverse).to eql(["AWESOME", "IS", "RUBY"])
    end
  end

Scopes:
Till now you couldn’t stub methods in before(:all) scope. Although you can do in rspec3, rspec team suggested not to use all the times. Here is the example how to use:

   RSpec.describe String do
      before(:context) do
        RSpec::Mocks.with_temporary_scope do
           string = "ruby is awesome"
           allow(string).to receive(:upcase).and_return("RUBY IS AWESOME")
           @hash = string.upcase
        end
      end 

      example "able to split the string" do
        expect(@hash.split(" ")).to eq(["RUBY", "IS", "AWESOME"])
      end 

      example "able to split and reverse" do
        expect(@hash.split(" ").reverse).to eq(["AWESOME", "IS", "RUBY"])
      end
    end

Changes in rspec-rails

In rspec-rails they have changed the structure a little. From rspec3 onwards, rspec team segregated core rspec configuration from rspec-rails configuration. All core rspec configuration like expectations.syntax = :expect, profile_examples = 10 moved to spec_helper and all rspec-rails configuration like use_transactional_fixtures moved to rails_helper. verify partial doubles: Prior to verifying partial double let see what is partial double. Partial double is a real object which acts as a test double in testing. Using verify_partial_doubles option you can cross check that by unknowingly you mock a method which is not there in the class. Here is the example:

  RSpec.describe Array do
    describe "#to_hash" do
      example "convert array to hash" do
        a = [1,2,3]

        allow(a).to receive(:to_hash).and_return({"1" => 0, "2" => 0, "3" => 0})
        expect(a.to_hash).to eql({"1" => 0, "2" => 0, "3" => 0})
       end
     end
   end

   # spec/spec_helper

   config.mock_with :rspec do |mocks|
     mocks.verify_partial_doubles = true
   end

Here is output when I run the specifications.

Failures:

  1) Array#to_hash convert array to hash
     Failure/Error: allow(a).to receive(:to_hash).and_return({"1" => 0, "2" => 0, "3" => 0})
       [1, 2, 3] does not implement: to_hash
     # ./spec/models/person_spec.rb:63:in `block (3 levels) in '

There is no method called to_hash exists on Array class. But I mock that method on that partial double. As method is not defined and I set verify_partial_doubles to true the above spec has failed. Till now we have seen changes in rspec-rails, rspec-mocks and rspec-expectations. Now we will see changes in rspec-core.

If you have observed carefully, I have not used 'it' throughout the post. In rspec3, it is suggested not to use 'it' to define a test case. Instead use 'example' to define a test case as 'it' might deprecated on future versions. Not just 'it', but a few other aliases have been added that make the test cases more readable. In rspec3, before(:example) is now an alias for before(:each) and before(:context) is an alias of before(:all). Not only can you use pre-defined aliases but also define your own aliases. Here is the example:

   RSpec.configure do |config|
     config.alias_example_to :frun, run: true
   end

Now you can use frun to define a test case with metadata as run:true. Here is the example:

   frun "should adds 'run:true' in example metadata" do
     expect(1+1).to eq(2)
   end

Now you can run the test case using run tag as follows:

  $rspec spec/models/person_spec.rb -t run
   Run options: include {:run=>true}
   .

   Finished in 0.02857 seconds (files took 1.1 seconds to load)
   1 example, 0 failures

Apart from few additions there are few things has been removed from rspec3. From¬†no onwards you can’t do the¬†following:

   RSpec.describe "Array" do
     subject { [1,2,3] }

     its(:length) { is_expected.to eq(3)}

     example "contains three items" do
       expect([1,2,3]).to have_at_least(3).items
     end
   end

To make above example work you have to include rspec-its and rspec-collection_matchers gems in your Gemfile. I hope these notable changes in rspec3 help you. Please help me expand this post by commenting on things I may have missed.

Real Time notifications using slanger and sidekiq

With the increasing expectation of web-applications, everyone wants real time updates or real time notifications to improve the web portals user experience. Understandably, my project required Real time notifications too. I successfully implemented and deployed on production. During my development I found some interesting gems, javascript libraries that I came across. So I thought it would be helpful if I share my experience. To make it more useful I have created a sample repository for demonstrating Real Time notifications for your further reference.

The AIM: Notify the user with a reminder on specific user defined date and time using Web notifications.

I choose slanger gem. (Recently updated by¬†Jiren to make it compatible with rails4). To make it work, add pusher to rails app and for slanger instead of adding it in rails Gemfile, I created a sub directory called ‘slanger’ in my¬†rails project and added¬†slanger. Now we¬†are done with installation. Now let’s see some configuration:

  #config/initializers/pusher.rb

  Pusher.app_id = app-id
  Pusher.key = app-key
  Pusher.secret = app-secret
  Pusher.port   = 4567
  Pusher.host   = 'localhost'
 

where app-id, app-key and secret could be any secure random number for authentication purpose.

Note: Add the IP address of the machine if your running app is in production mode or staging mode.

Now it’s time to add a sidekiq job to push notifications on a particular user defined time. Here is code snippet how to add sidekiq job:


# app/models/reminder.rb

class Reminder
  include Mongoid::Document
  field :text, type: String
  field :notify_at, type: Time

  after_create :add_sidekiq_job
  before_destroy :remove_sidekiq_job

  def add_sidekiq_job
    id = ReminderWorker.perform_at(scheduled_at.second, {id: self.id.to_s})
    self.update_attributes(job_id: id)
  end 

  def remove_sidekiq_job
    queue =  Sidekiq::ScheduledSet.new
    job = queue.find_job(self.job_id)
    job.delete if job
  end 

  def scheduled_at
    notify_at.to_time.to_i - Time.now.to_i
  end
end

The above code snippet adds a sidekiq job after reminder object has been created.

Now let’s trigger some notifications:

  # app/worker/reminder_worker.rb

 class ReminderWorker
  include Sidekiq::Worker

  def perform(args)
    reminder = Reminder.find(args['id'])
    Pusher.trigger("reminder", "reminder_12345", {notification_text: "Reminder: #{reminder.text}"})
  end
end

where trigger takes three arguments. First argument is ‘channel’ and second argument is ‘event’ and third one is data that you want to push to client.

Note: reminder_12345 is a unique way to identify to which user we have to push notification.

Once data has been published on channel the browser should respond and display data on front end. To do this every client should subscribe reminder channel using following javascript snippet:

  #app/assets/javascripts/remainder.js  

  var reminder = pusher.subscribe('reminder');

Prior to subscribe, pusher needs to authenticate. To do this, use following snippet:

  # app/assets/javascripts/remainder.js 

  window.pusher = new Pusher(PUSHER_API_KEY, {authEndpoint: '/pusher/authentication', authTransport: 'ajax',activityTimeout: 120000, disableStats: true});

Where PUSHER_API_KEY would unique key to identify your application. To know more about the remaining options please go through this.

Once you authenticated, client should respond once pusher triggers(nothing but event) a notification. To achieve this use following snippet:

  #app/assets/javascripts/remainder.js  

  reminder.bind('reminder_12345', function(data){
    $.notify(data['notification_text']);
   });

Once you watched above code snippet carefully you will get doubt about what is $.notify and why we need it?

notifyjs is a beautiful javascript library to display notifications on your front end. To know how to customize your notifications please go through the documentation.

So finally coalition of sidekiq, slanger and notifyjs has done job for me. It would be great if you share your thoughts on this coalition. Do play around with the demonstration on Github to see this in action.

Writing unit tests the right way

We all know what unit tests are and which gems we need to use for unit testing. Unit testing means testing a method in isolation. Unit tests are light weight and execute fast when compared to integration tests. You can write unit and integration tests either using rspec or minitest along with any mocking frameworks. Both frameworks have built-in support for mocking frameworks but in rspec they are extracted as a separate gem called rspec-mocks. Here are some pointers

Tests should help in debugging code.

We write tests to check whether we have covered all scenarios for our code. But we forgot to cross-check whether the test cases helps in debugging when an problem raises. Consider the following scenario

  # app/models/project.rb

  class Project
    has_many :buildings
    has_many :units
    has_many :bookings
  end
  # app/models/building.rb

 class Building
   belongs_to :project
   has_many :units
 end
  # app/models/unit.rb

  class Unit
    belongs_to :building
    has_many :bookings
  end
  # app/models/booking.rb

  class Booking
    belongs_to :unit
    belongs_to :project
    has_many :unit_installments
  end

There are 4 kinds of users called as DSGadmin, Partner, Area Sales manager(ASM), Sales Manger(SM) working under same company(technically called as Distributor in my system) are there. Here is the requirement:

  • DSGadmin can see Partner, ASM, SM bookings and his bookings.
  • Partner can see ASM, SM bookings and his bookings
  • ASM can see see SM and his bookings
  • SM can see his bookings only

Now, to test this I wrote some integration tests, thinking that they will be robust and quick.

   describe "Distributor Report", dont_clean: true do
     context "When DSG admin and partner has done a booking each" do
       before(:all) do
         @today = Date.today.to_s

         @project = create(:project)
         @franchise = create(:franchise)
         @dsg_admin = create(:dsg_admin, resource: @franchise)
         @partner = create(:partner, resource: @franchise, parent: @parent)

         @project.distributors = [@franchise]
         @project.save

         create(:distributor_admin_booking, resource: @franchise, sold_by: @dsg_admin, project: @project.id)
         create(:distributor_staff_booking, resource: @franchise, sold_by: @partner, project: @project.id)
       end 

       it "then DSG admin able to see 2 bookings" do
         login(@dsg_admin)

         post sales_reports_path, search: { project_id: @project.id, start_date: @today, end_date: @today, resource_key: "#{@franchise.id}@@Distributor"}

         expect(assigns[:report].first[1]).to eq(2)
       end 

       it "then partner able to see 1 booking" do
         login(@partner)

         post sales_reports_path, search: { project_id: @project.id, start_date: @today, end_date: @today, resource_key: "#{@franchise.id}@@Distributor"}

         expect(assigns[:report].first[1]).to eq(1)
       end
     end
   end
Distributor Report
  When DSG admin and partner has done a booking each
    then DSG admin able to see 2 bookings
    then partner able to see 1 booking (FAILED - 1)

Failures:

  1) Distributor Report When DSG admin and partner has done a booking each then partner able to see 1 booking
     Failure/Error: expect(assigns[:report].first[1]).to eq(1)
       
       expected: 1
            got: 2
       
       (compared using ==)
     # ./spec/requests/distributor_reports_spec.rb:172:in `block (3 levels) in '

Finished in 2.17 seconds
2 examples, 1 failure

As we can see, when I ran tests then one of my integration tests failed. However, it did not tell me where things failed. Now I realized the need to write some good unit tests and got down to it.

  describe "#b_match" do
    context "should return criteria as" do
      before do
        @sales.stub(:c_project).and_return(nil)
        @sales.stub(:resource).and_return(nil)
        @sales.stub(:default_match).and_return({'deleted_at' => nil})
      end 

      it "returns {'deleted_at' => nil} if no project or resource has selected" do
        @sales.b_match
        expect(@sales).to have_received(:default_match)
      end 

      it "returns {{'deleted_at' => nil, sold_by_id: {'$in' => @user.assigned_users_ids}} if selected resource is distributor" do
        expect(@sales.b_match).to eq('$match' => {'deleted_at' => nil, sold_by_id: {'$in' => @partner.assigned_users_ids}})
      end
    end
  end
  Sales 
    #b_match 
      returns criteria as {{'deleted_at' => nil, sold_by_id: {'$in' => @user.assigned_users_ids}} if selected resource is distributor


Failures:

  1) Sales#b_match should return criteria as returns {{'deleted_at' => nil, sold_by_id: {'$in' => @user.assigned_users_ids}} if selected resource is distributor
     Failure/Error: expect(@sales.b_match).to eq('$match' => {'deleted_at' => nil, sold_by_id: {'$in' => @partner.assigned_users_ids}})
       
       expected: {"$match"=>{"deleted_at"=>nil, :sold_by_id=>{"$in"=>["5378c281472d208bc100000d"]}}}
            got: {"$match"=>{"deleted_at"=>nil}}
       
       (compared using ==)
       
       Diff:
       @@ -1,2 +1,2 @@
       -"$match" => {"deleted_at"=>nil, :sold_by_id=>{"$in"=>["5378c281472d208bc100000d"]}}
       +"$match" => {"deleted_at"=>nil}
       
     # ./spec/reports/sales_spec.rb:75:in `block (4 levels) in '

Finished in 0.45649 seconds
9 examples, 1 failure

Now, with this failure, I found the issue in b_match method. In the fix below, it was obvious that the I required the `merge!` instead of the `merge` method. However, the integration tests never pointed to this particular failure – the unit test did!

  def b_match
     match = default_match
     if @current_resource.class == Distributor
+      match.merge!(sold_by_id: {'$in' => @user.assigned_users_ids})
-      match.merge(sold_by_id: {'$in' => @user.assigned_users_ids})
     end
     {'$match' => match}
   end

Unit tests should be independent

Unit tests should be¬†less tightly coupled to avoid dependency. Consider the scenario where we need to calculate the compound interest if a customer didn’t pay their instalment¬†on time. Here is a code snippet

  # app/models/unit_installment.rb

  class UnitInstallment
    field :percentage
    field :amount, type: Float, default: 0
    field :order, type: Integer
    field :received_amount, type: Float, default: 0
    field :interest_amount, type: Integer, default: 0
    field :is_due, type: Boolean, default: false

    belongs_to :booking

    before_create :initialize_is_due

    def initialize_is_due
      self.is_due = unit.building.building_schemes.where(payment_schedule_id: payment_schedule_id, scheme_id: scheme_id).first.try(:due)
      make_due
    end
  end

Now I have written the code to calculate the interest for a installment amount. Here is my test case to check interest calculation:

require 'spec_helper'

describe InterestAmount do
  before do
    UnitInstallment.any_instance.stub(:initialize_is_due).and_return(true)
    UnitInstallment.any_instance.stub(:update_booking).and_return(true)
    UnitInstallment.any_instance.stub(:valid?).and_return(true)
  end 

  let!(:booking) { create(:org_booking) }
  let!(:installment) { create(:unit_installment, booking: booking) }
  let!(:project) { create(:project) }

  context "#calculate" do
    it "should update interest amount on installment" do
      interest = InterestAmount.new(installment: installment, project: project, booking: booking)
      interest.stub_chain(:interest, :round).and_return(100)
      interest.calculate

      expect(installment.reload.interest_amount).to eq(100)
      expect(interest).to have_received(:interest)
    end
  end

Notice that, I have stubbed before_create callback which deals with unit, payment_schedule, scheme objects. So if either unit, payment_schedule or scheme are invalid then I can’t create unit installment object which eventually can’t check interest calculation.

I hope it helps you understanding how to write unit tests that help in debugging.