in Ruby on Rails

Background Tasks & Rails – A Sidekiq Example

This post involves a little bit of everything–queuing (Sidekiq), JSON, API calls, redis, database manipulation. Enjoy!

The latest feature I am pushing to Dotawithme is listing the players by who is currently online. This is an oft requested feature (with good reason – DOTA players are impatient) but I wasn’t entirely sure what the best way to go about this was.

Of course I couldn’t just hammer Valve’s Steam API or I would quickly hit my daily limit. This means I need to create a background process to run at set intervals and update the database accordingly. There are many different gems to help your Rails app out with this and I settled on Sidekiq (‘Sidekick’) because it’s fast and (supposedly) simple to use.

Setting it up is relatively straightforward. Put the gem ‘sidekiq’ in your Gemfile, run bundle install and you are good to go (if you haven’t already, be sure to install redis).

Now you are going to want to create a new file in app/workers/ called stuffs_worker.rb (or whatever is appropriate). Mine is actually called steams_worker.rb as I’m using Steam’s API.

Let’s jump to the code of what that looks like:

Ok this is great, but now where do we make a call to this function? Oh I should note: the one function you need in this file is a def perform – and put whatever code in there that you need. Now, to call this function we make a call to it in our controller. In my players_controller in the def index I have this:

This took me a while to figure out. The first part SteamsWorker is our class name. The perform_in(1.minute – is how often this task will be performed. You could do 2.hours, 1.day, whatever fits your needs. The second argument is how I passed the player’s ids. I was able to successful pass one id no problem, but has having a hard time finding a way to pass all the id’s without their other fields. This is where (:select -> :id).collect comes in. It puts them all in an array by ID number which I’m then able to call on in the Sidekiq function. Great stuff.

That’s all there is to it. Now just remember to run your sidekiq (bundle exec sidekiq) in another terminal window and you are good to go!

One last thing that I want to point out because I didn’t realize it for a long while. If the job fails it will keep repeating it hoping that it will get it right. This is great and useful for certain tasks, HOWEVER these kept building up in my queue and would keep running even after I had CTRL+C the server and changed some code. This turned debugging into a nightmare so don’t forget to use this great trick to not have that happen to you.

This turned out to be key in my getting this solution correct. Hope this is useful for someone as all the examples I found of Sidekiq were only sending one id for a task rather than having it iterate over all subjects in a database.

Write a Comment

Comment