Because sidekiq is running our background jobs, and some of these are time-sensitive, we need to ensure sidekiq is always running, particularly after system reboots.
In order to start at boot time, Sidekiq needs to be managed as a system service, instead of a process that gets started and stopped only via a rails deploy process. Here’s how we set that up. Most of this should be automated via ansible, but this guide captures the manual process that guides our ansible configuration and provides additional context and refernce. We typically run our projects on Ubuntu based servers that use Systemd as their init system and service manager and use Capistrano to manage our code deployments.This guide assumes Ubuntu 18.04.
First, add Sidekiq to your application following the steps in Using Sidekiq for background jobs. Once you’ve completed those steps and can start up Sidekiq in your development environment, come back here and continue by configuring Capistrano (Step 2).
We use the Capistrano::Sidekiq gem to integrate Sidekiq into our Capistrano-based deployment process. The most up-to-date guide for how to configure capistrano so it can deploy to DCE built servers is Capify a Rails app the ansible-samvera way
We manage this process via ansible, in the sidekiq role in ansible-samvera.
Connect to your server as the deploy user via ssh and issue the command
sudo systemctl restart sidekiq.
Ensure your database pool has enough connections to handle what sidekiq will throw at it. By default sidekiq starts 25
threads, although this is configurable. Update your
database.yml file to match. See the sidekiq Concurrency
documentation for more details.
production: adapter: mysql2 database: foo_production pool: 25
Once you’ve all your configuration files set, you can test your deployment. Just run the deploy command for each of your defined stages, replacing $STAGE_NAME in the command below accordingly:
bundle exec cap $STAGE_NAME deploy
Congratulations! You should now be able to visit the
/sidekiq/busy route on your server and see a page that
shows your running queues: