Discourse Onboarding LabChapter VI — Sidekiq Background Jobs✓ 0/6Contents
Chapter VI

Sidekiq Background Jobs

Jobs.enqueue is how Discourse offloads slow work (image processing, emails, rebaking, notifications) from the request cycle onto Sidekiq, so controllers and model callbacks stay fast. Jobs::Base wraps every job's perform method with instrumentation, multisite database iteration, and exception handling, while a run_immediately/run_later switch lets the exact same enqueue call push to Sidekiq in production or execute synchronously in development/tests.

From a model callback to a background job
User#refresh_avatar (after_save)app/models/user.rb
Jobs.enqueueapp/jobs/base.rb
Sidekiq queue (DB.after_commit + client_push)app/jobs/base.rb
Jobs::Base#performapp/jobs/base.rb
execute(opts) subclass logic…r/rebake_quoted_posts_for_user.rb
Figure I · Click a node to see what it does, where it lives in the code, and its labeled connections — the annotation opens beside it.
walkthrough

Jobs.enqueue: resolving, sanitizing, and dispatching

app/jobs/base.rb · lines 379–449
lines 379–384

Callers can pass either a job class or a name like :rebake_quoted_posts_for_user; either way it's resolved down to a concrete Jobs:: class.

lines 386–389

Unless the caller explicitly opts to run on all sites, the job is pinned to the current multisite database so it re-connects to the right db when it runs.

lines 394–407

Sidekiq stores job args as JSON, so enqueue round-trips opts through JSON.dump/JSON.parse and warns (in a deprecation) if that changes the values — this catches things like symbol keys or non-primitive values before they silently get mangled.

lines 409–418

In the normal (run_later?) case, enqueue builds the Sidekiq job hash and pushes it via DB.after_commit — deferring the actual Sidekiq push until any open transaction has committed.

lines 419–421

When jobs aren't meant to run later (e.g. Jobs.run_immediately! in tests), development mode still runs the job off-thread via Scheduler::Defer rather than fully synchronously.

def self.enqueue(job, opts = {})    if job.instance_of?(Class)      klass = job    else      klass = "::Jobs::#{job.to_s.camelcase}".constantize    end    # Unless we want to work on all sites    unless opts.delete(:all_sites)      opts[:current_site_id] ||= RailsMultisite::ConnectionManagement.current_db    end    delay = opts.delete(:delay_for)    queue = opts.delete(:queue)    # Only string keys are allowed in JSON. We call `.with_indifferent_access`    # in Jobs::Base#perform, so this is invisible to developers    opts = opts.deep_stringify_keys    # Simulate the args being dumped/parsed through JSON    parsed_opts = JSON.parse(JSON.dump(opts))    if opts != parsed_opts      Discourse.deprecate(<<~TEXT.squish, since: "2.9", drop_from: "3.0", output_in_test: true)        #{klass.name} was enqueued with argument values which do not cleanly serialize to/from JSON.        This means that the job will be run with slightly different values than the ones supplied to `enqueue`.        Argument values should be strings, booleans, numbers, or nil (or arrays/hashes of those value types).      TEXT    end    opts = parsed_opts    if ::Jobs.run_later?      hash = { "class" => klass, "args" => [opts] }      if delay        hash["at"] = Time.now.to_f + delay.to_f if delay.to_f > 0      end      hash["queue"] = queue if queue      DB.after_commit { klass.client_push(hash) }    else      if Rails.env.development?        Scheduler::Defer.later("job") { klass.new.perform(opts) }
step 1 of 5
      dbs =        if opts[:current_site_id]          [opts[:current_site_id]]        else          RailsMultisite::ConnectionManagement.all_dbs        end      exceptions = []      dbs.each do |db|        exception = {}        RailsMultisite::ConnectionManagement.with_connection(db) do          job_instrumenter =            JobInstrumenter.new(job_class: self.class, opts: opts, db: db, jid: jid)          begin            I18n.locale =              SiteSetting.default_locale || SiteSettings::DefaultsProvider::DEFAULT_LOCALE            I18n.ensure_all_loaded!            begin              logster_env = {}              Logster.add_to_env(logster_env, :job, self.class.to_s)              Logster.add_to_env(logster_env, :db, db)              Thread.current[Logster::Logger::LOGSTER_ENV] = logster_env              execute(opts)            rescue => e              exception[:ex] = e              exception[:other] = { problem_db: db }            end          rescue => e            exception[:ex] = e            exception[:message] = "While establishing database connection to #{db}"            exception[:other] = { problem_db: db }          ensure            job_instrumenter.stop(exception: exception)          end        end        exceptions << exception unless exception.empty?      end
app/jobs/base.rb · lines 289–328
Figure II · Base#perform loops over every relevant multisite database, and for each one wraps the actual execute(opts) call between JobInstrumenter.new (start timing/GC/profiler capture) and job_instrumenter.stop (record duration, SQL/Redis stats, and any exception) — this is why every job's timing and failure shows up in sidekiq.log regardless of what the job itself does.
Figure III · state machineJobInstrumenter status lifecycle
JobInstrumenter.new finishes setupjob_instrumenter.stop(exception: {}) — execute(opts) returned normallyjob_instrumenter.stop(exception: {...}) — execute(opts) raisedstartingpendingsuccessfailed
Click a state above.
checkpoint

Check your understanding

Answered in place — nothing is graded, everything is explained. 0 / 2 passed

Why does Jobs.enqueue push the Sidekiq job hash via DB.after_commit instead of pushing it directly?

How can a spec make Jobs.enqueue run a job's logic synchronously instead of pushing it to Sidekiq?