Thursday, 6 December 2012

Delayed job, a real pain in production

There are so many ways to handle background jobs in Ruby on Rails. But the most simplest one is delayed job (DJ) as I showed you in the following post Handling delayed jobs. Simple in the sense that it requires very less effort to set up and in uses single table to handle the jobs.

It's a very good solution for development environment, but in production people starts complaining of problems such as:
  • DJ is eating up my server resources.
  • DJ is firing new daemon processes every now and then.
  • Many DJ instances are running at a time.
And many more similar kind of issues.
To run delayed job daemon along with rails server, people often end up creating config/initializers/start_worker.rb, and put codes something like do
  system("rake jobs:work")

I don't encourage this process.
Better solution will be to start delayed job using cron jobs.
There's a gem available for this purpose: whenever.

  • Add this line to your gemfile: gem "whenever", :require => false
  • $ bundle install
  • $ wheneverize .
  • This will create config/schedule.rb.
  • Open up that file and put the following codes in it:
       set :output, "/path_to_your_project/log/cron_log.log"
       every :reboot do
         command "cd /path_to_your_project && RAILS_ENV=production script/delayed_job start"
  • This will start delayed job on every server boot. You may also write 
       every 10.hours do
         command "cd /path_to_your_project && RAILS_ENV=production script/delayed_job restart"
          This will restart delayed job after every 10 hours. Although point number 6 is not a very feasible    
          solution, but it surely helps in some cases.
  • whenever --update-crontab your_project_name.
  • $ crontab -l
          To check whether the jobs you just made are listed.