Optimize Your Rails Fragment Cache By Lazily Loading your Objects

I was looking into the performance of our my portfolio page the other day which was supposedly cached.  Here's what I noticed within the controller.





And here was our view:

This being a page that show properties and spaces, we are attempting to cache the entire page based on some keys that get updated whenever any of the spaces or properties within the user's portfolio is updated.  But here's the problem, our queries are doing some eager loading from the database.  Even though we were fragment caching the entire page, each cached hit was still hitting the database, creating a bunch of active record objects and then never actually using them. Even on a cached hit, this is what I saw in the logs:

To fix this, we moved all of the query logic out of the controller show method and into a our query object where it is safely memoized

Now instead of accessing instance variables created within the controller show method, our view accesses helper methods lazily from within the cache fragment.

Our cached my portfolio page no longer needlessly loads dozens of active record objects.

Testing SSL under capybara

If your website runs under ssl than your integration tests should too.  This post outlines how to set up testing over https with capybara 2.1 and a few popular capybara drivers.  The majority of this info was borrowed from this capybara github issue.

In order to run your capybara test server under ssl, you will need to generate a self signed certificate.
You'll want to check the certificate and private key into your project.  I added them to my spec/support directory.

Now that the certificate is all set, start capybara with ssl enabled using your certificates.  Hint: If you are using rspec, the following snippets can be pasted into your spec_helper.rb.



Next tell capybara, to make requests using https:


Now you'll need to instruct your web driver to automatically accepted your self signed certificate even though you are running from localhost.



Finally, add the following monkey patch capybara to make http with ssl enabled:



I am not proud of the last step, and jnicklas indicated in this closed issue that this should not be needed, but not sure how else to get it done.  Plan on following up with him and updating the post.

Scaling Heavy Web Requests with Sidekiq and Pusher

Long Running Web Requests

At View The Space, many of our pages deliver real time data over extended time periods.  The database queries producing this data can take significant time to run and since the reporting is meant to be real time, caching is not an option.  Our application servers consist of a bunch of single threaded unicorn processes sitting on top of Heroku.  Tying up our web requests with long running heavy queries would be a bad idea as all other web requests would queue up causing bad performance for our entire app.

Push it in the Background with Sidekiq

Instead of performing long queries during the course of the rails web request, we push that heavy lifting to the background with Sidekiq.  Sidekiq is a multi-threaded background job queueing library built on top of redis and by default each Sidekiq process leverages 25 threads.  When a request is made to our building traffic dashboard, a simple html page is quickly rendered to the user followed by an ajax json request.  That second ajax json request simply pushes a critical job onto a sidekiq queue:


That first job pushes another critical query job for each query to be performed for the dashboard.  Since sidekiq is multi-threaded, each query should be performed in parallel.



Once the query is complete, the job pushes the result to the browser using Pusher and the pusher-gem. As a side note, Pusher is a service that makes using websockets easy and it supports all major browsers including Internet Explorer.

Pusher publishes the result to a unique user channel based on the user's database id and rails session id.  The query result should be some simple raw data within a hash or an array that can easily be converted into json.

Rendering the Data within the Browser

Now that the data has been pushed from the server side sidekiq job, we need some client side pusher javascript code listening for the results from the browser. The following coffee script initializes pusher with the unique channel we published to on the server side and listens for each event. Once we know the pusher subscription has succeeded, we submit the ajax form which sets the whole sidekiq to pusher process into motion.

Summary

Heavy lifting such as real time data reporting is not something you want to execute during your rails web requests.  Instead perform that expensive data crunching in the background with Sidekiq and push the results to your users' browsers with Pusher.

What's Missing from Heroku Add-ons

Today I got burned by my redis service for the last time.  I am not going to name names, but they are listed within the heroku add-ons.  Everybody needs a redis instance so I had to pick a new service to replace my existing, but there are currently 5 redis providers listed as add-ons.

I had no way of deciding which one to pick.   Yeah yeah, i could have done some research, but my existing redis instance was just flat out not working and parts of our app was down.  I was in a hurry.  In the end, like  the seasoned engineer I am, I picked the one with the prettiest logo.  My point is, why doesn't Heroku have some type of transparency within their add ons section?  It should look like the itunes store.  When choosing between heroku add-ons, I should be able to see how many other projects are using the add-on and some reviews would also be nice.   Can anyone think of a reason why it doesn't work this way today?

Setting the Rails Timezone to the User's Browser Timezone

Timezones confuse me.

At VTS we have lots of reports and charts that group data by the day. These reports should look different based on the user's configured timezone. I did not want to force all of our users to configure their timezone within our app when they already configured it within their operating system and hence their browser. The only problem... the browser does not pass the timezone to the server by default. You have to take care of it yourself.

The solution is pretty straight forward:

1. Set each users timezone as a cookie
2. Each request sets the Rails timezone to the value of that cookie

The above coffee script code uses jsTimezoneDetect to set the browsers timezone within a cookie that will live for the next 365 days. All that is left is setting the timezone during each request.


The above code would go into your ApplicationController. Simple around filter for each request that sets the timezone if it's present. Notice that it sets the Time.zone back to the original default timezone after each request and, yes, the Time.zone method is actually thread safe.

Slight Caveat

The timezone is set after each page request on document ready. That means the very first request will not have the cookie set for the server side around_filter and will not set the timezone. I have seen some people out there getting fancy with ajax redirects but this is probably a bit overkill for most use cases.  I start to have nightmares involving infinite redirects.

Since I extracted all of the above into a Rails Engine, you can ignore all of my rambling and just install the browser-timezone-rails gem.

Your app ain't working and you don't even know it.

I am a nerd.  This and my kids is what keeps me up at night.  This post is about how I find a way to get to sleep without over-medicating.

Airbrake

This probably isn't that enlightening, but when an error happens in your app, you should probably know about it.  Airbrake takes care of this for you in a smart way.

Still Alive

There are plenty of services out there that ping your site but what does that prove?  Is your site doing what it's supposed to be doing?  Still Alive is like Cucumber for your live app.  Here's what we have running every minute:

Still not satisfied?  This is what we have running every 15 minutes:

Notice this is actually shares stuff which sends emails, but are we sure our emails are actually being delivered?

Email Monitoring Job

Every 15 minutes we run a resque job that checks our test email account to see if it's received an email in the last 30 minutes.

If no recent email is found, it raises an error.  That error gets picked up by airbrake since we have this within our resque initializer:

How did those emails get there you ask? Remember our Still Alive script sends email every 15 minutes during it's test run.

I'm getting alerts, what's going on?

I'm not going to pretend that 99% of you reading this post don't know about newrelic, but this is our go to tool to diagnose any kind of performance related issues. It also pings your website every minute which is a nice addition to stillalive. After all, if two alerting systems are reporting errors, it's a real sign that there is definitely an issue.

You have invalid ActiveRecord objects lurking in your database

Yes I am trying to scare you into reading this, but let's face it, you probably do.  There are a bunch of ways it can happen.

• That cowboy sitting next to you saved a bunch of objects with validation turned off
• Code changes - new validations that haven't always been there
• Faulty logic in your code that let's invalid stuff through

So you be saying, is this a big deal?  The answer in many cases is yes.  Suppose you add some new code that validates a new presence of a new field on your user object but you forget to update all of your existing user profiles.   If any page within your app updates a user object,  your customers may be in for a 500 page.
This is where active_sanity come in which checks all of the objects in your database for validity.  For each object that is invalid, it inserts an entry into an InvalidRecord table.  Again, we have a resque job that runs active sanity once a night:

You're probably noticing a pattern here.  If there are any invalid records, an error is raised, which fires off an airbrake alert.  This alert has a link to the rails_admin page which shows a list of all invalid records:

Conclusion

It's important for your sanity and general well being to be the first one to know that your app isn't functioning properly. At View the Space, we are constantly trying to think of new ways to improve upon our monitoring. Let us know what you do? Drop us a comment.

Cut down on those objects to speed up your cucumber tests

Chances are that a big reason your tests are so slow is all of those extra objects you are creating. Recently I cut down on the objects in my tests and I was able to speed up by cucumber runs by 30-40%. I achieved most of those gains by making some small changes to factory girl. I don't want to bore you with the detail of our object model but it goes something like this. Our main object is a space and our tests frequently create several of them. The problem is that a space has a reference to a user, property, an contacts. Each of those actually have references to other objects. Just by creating a few spaces we end up creating a web of objects but our tests didn't need all of these extra objects. Enter the default objects within our factory_girl definitions:

def default_user default_email = 'default@test.com' User.find_by_email(default_email) || Factory(:user, :email=>default_email) end def default_property street_address = '5555 5th Ave' Property.find_by_street_address(street_address) || Factory(:property, :street_address=>street_address) end def default_space floor = "99th-98th floor" Space.find_by_floor(floor) || Factory(:space, :floor=>floor) end factory :space do property { default_property } user { default_user } suite { "##{Forgery(:basic).number.to_i}" } floor { Forgery(:basic).number.to_i } rental_rate { Forgery(:basic).number * 100 } description { Forgery(:lorem_ipsum).words(rand 100) } space_available "25,000 sq ft" availability_type "arranged" status 'active' end

Instead of creating new properties, users, and contacts each time a space is created, by default all spaces share the same nested objects. Most often this is not at all a problem in any of our tests. If we do require spaces do not share the same property, we specify it the Given section of our test.