Sidekiq in Ruby on Rails 7

In today’s fast-paced digital landscape, building web applications that can handle high traffic and complex tasks efficiently is crucial for success. As developers, we’re constantly seeking ways to optimize performance, streamline processes, and enhance the user experience. 

One powerful tool in our arsenal for achieving these goals is Sidekiq within Ruby on Rails.

What is Sidekiq?

Sidekiq is a popular background processing framework for Ruby on Rails applications. It allows developers to offload time-consuming or resource intensive tasks from the main web request/response cycle to background workers, improving the responsiveness and scalability of the application.

What does Sidekiq do?

Sidekiq excels at handling a wide range of background processing tasks in Ruby on Rails applications.

• Sending Emails

  When sending emails in response to user actions, like account registration or password resets, Sidekiq can be used to perform email delivery asynchronously, ensuring that the main web server remains responsive and users aren’t kept waiting.

• Processing File Uploads

  Uploading and processing large files can be resource-intensive and time consuming. Sidekiq can handle file processing tasks in the background, freeing up the main web server to continue serving requests while files are being uploaded, processed or converted to different formats.

• Scheduling Jobs

  Sidekiq can schedule jobs to run at specific times or intervals using its built-in support for cron-like scheduling. This allows developers to perform periodic tasks such as data backups, report generation, or database maintenance without manual intervention.

• Batch Processing

  When dealing with large datasets or performing bulk operations, such as importing/exporting data or batch updates, Sidekiq can divide the work into smaller chunks and process them asynchronously in the background, improving performance and reducing the risk of timeouts or resource exhaustion.

• API Integrations

  Integrating with third-party APIs often involves making network requests and processing responses, which can introduce latency and potential failures. Sidekiq can handle API integration tasks in the background, retrying failed requests, and ensuring reliable communication with external services.

• Image Processing

  Manipulating images, such as resizing, cropping, or applying filters can be CPU-intensive and slow down the main web server. Sidekiq can perform image processing tasks asynchronously, allowing the main web server to continue serving requests while images are being processed in the background.

• Data Cleanup and Maintenance

  Performing data cleanup tasks, such as deleting expired records, archiving old data, or optimizing database indexes, can improve the performance and reliability of Rails applications. Sidekiq can handle these maintenance tasks in the background, ensuring that they don’t impact the responsiveness of the application.

How does Sidekiq Operate?

Sidekiq is responsible for managing the queuing, scheduling, and execution of background jobs within a Rails application. It operates through these three elements: Redis, client, and server.

• 1. Redis

  Redis is an in-memory data store that serves as the backend for Sidekiq. Sidekiq uses Redis to store information about enqueued jobs, such as metadata, queue names, and job parameters. Redis acts like a high-performance message queue, allowing Sidekiq to efficiently process background jobs asynchronously.

• 2. Client

  In the context of Sidekiq, the client refers to the component of the Sidekiq framework responsible for enqueuing jobs into Redis. When a job is enqueued using Sidekiq’s API, the client serializes the job data and pushes it onto a Redis queue for processing by Sidekiq workers.

• 3. Server

  The server refers to the Sidekiq worker processes that retrieve and process jobs from Redis Queues. Sidekiq workers continuously poll Redis for new jobs, and when a job is found in a queue, the worker retrieves the job data, deserializes it, and executes the corresponding Ruby code to perform the background task.

Getting Started with Sidekiq in Ruby on Rails 7

1. Create a new project or skip this step if you already have one set up.

$ rails new my_project

2. Add Sidekiq to your project.

$ bundle add sidekiq

3. Create a new job.

$ rails generate sidekiq:job thank_you_job

This command will generate the file app/sidekiq/thank_you_job.rb and a corresponding test file. We can customize the job file:

class ThankYouJob
  queue_as :default

  def perform(name, product)
    text = "Hello #{name}, thank you for buying #{product}!"
    Notifications::Email.call(text)
    # any other valid Ruby/Rails can be here!
  end
end

As previously mentioned, Sidekiq relies on Redis to enqueue background jobs. So, make sure you have a Redis server running.

Redis Configurations

There are two ways to configure Sidekiq. First, you can make use of the REDIS_URL environment variable, which is managed by the dotenv-rails gem. Alternatively, if your setup requires additional customization, you can create a sidekiq.rb file within the config/initializers directory. 

Using .env file

REDIS_URL=redis://redis.example.com:6379/1

Using config/initializers/sidekiq.rb

Sidekiq.configure_server do |config|
 config.redis = { url: 'redis://redis.example.com:6379/1' }
end

Sidekiq.configure_client do |config|
 config.redis = { url: 'redis://redis.example.com:6379/1' }
end

In Docker, you can use a simple Redis container to run Redis and connect your app with it.

Queueing the Jobs

Here we are enqueuing a background job using Sidekiq within a rails application, executing the job in the Rails console, and confirming its successful creation and queuing for processing.

$ rails c

3.0.0 :001 > SomeJob.perform_async "John", "Laptop"
=> "5a4c435ddd2295a6104cd5l"

Running the Jobs

Once the job is in the queue, we need to run the Sidekiq worker to execute it. Open another terminal window and execute:

$ bundle exec sidekiq

If everything fine you will see, output similar to this on your screen:

Using Sidekiq in Controller

Let’s assume we have an Order controller where we need to send a “Thank you!” email to our buyer:

class OrdersController < ApplicationController
  def create
    ThankYouJob.perform_async('John', 'Laptop')

    # any other valid Ruby/Rails code can be here!

    render json: { message: 'Created' }, status: :created
  end
end

We also have the capability to use the perform_in and perform_at methods to schedule jobs:

class OrdersController < ApplicationController
  def create
    ThankYouJob.perform_async('John', 'Laptop')
    ThankYouJob.perform_in(10.seconds, 'John', 'Laptop')
    ThankYouJob.perform_at(20.seconds.from_now, 'John', 'Laptop')
    render json: { message: 'Created' }, status: :created
  end
end

Using Sidekiq with Cron

To enable scheduled jobs with Sidekiq, we’ll integrate the sidekiq-cron gem file into our Gemfile:

gem 'sidekiq-cron'

Next, let’s define a job for our scheduled task:

class ReturnJob
  queue_as :default

  def perform(name)
    text = "Hello #{name}, we have some products for you!"
    Notifications::Email.call(text)
    # any other valid Ruby/Rails can be here!
  end
end

We’ll then set up a schedule for this job in the config/schedule.yml file:

return_job:
  cron: "0 0 * * 0"
  class: "ReturnJob"
  queue: default

Lastly, we need to configure Sidekiq to read from schedule.yml file:

schedule_file = "config/schedule.yml"
if File.exist?(schedule_file) && Sidekiq.server?
 Sidekiq::Cron::Job.load_from_hash YAML.load_file(schedule_file)
end

This setup allows us to schedule and execute recurring jobs using Sidekiq with the help of cron expressions.

Sidekiq vs. Sidekick Pro

Sidekiq and Sidekiq Pro are both popular background processing frameworks for Ruby applications. While Sidekiq provides essential features for job processing and is available as an open-source solution, Sidekiq Pro offers additional advanced features, scalability options, and support services through a commercial licensing model.

Sidekiq Pro enhances the capabilities of regular Sidekiq, with advanced features such as batch jobs, job tags, and multi-process currency mode. It also provides more robust monitoring and reliability features  for mission-critical applications.

Sidekick Pro, however, comes with a cost associated with licensing and support, whereas regular Sidekiq remains free and open-source. 

The choice between the two depends on the specific requirements, scalability needs, and budget constraints on the project.

Embracing Efficiency: The Sidekiq Solution for Ruby on Rails 7

Sidekiq proves indispensable in modern web development, offering efficient background processing for Ruby on Rails applications. From email delivery to API integrations, its versatility streamlines tasks and enhances user experiences. 

Sidekiq empowers developers to optimize performance and elevate application quality in today’s digital landscape.