Background jobs

Durable multi-step jobs as Edge Functions: each step runs once and is saved, failing steps are retried, runs can sleep for days, with idempotency keys and a run history. Job mode runs one long call (up to 60 minutes) with progress, a log and checkpoints.

A background job is an Edge Function whose handler is wrapped in job(). Its work is split into named steps. StitchDB saves the result of every finished step in your database and calls the function again for the next one, so a run survives crashes, deploys and server restarts, a failing step is retried with backoff without repeating the steps before it, and a run can sleep for minutes or days without holding anything open.

#Turn it on

In Edge Functions › Background jobs (project owners and admins) click Turn on background jobs. This creates the schema _stitch_jobs in your database and starts the job worker on your project's server.

#Write a job

// functions/onboarding/index.ts
import { job } from 'https://stitchdb.com/sdk/stitchdb-jobs.mjs'
import { createClient } from 'https://stitchdb.com/sdk/stitchdb-js.mjs'

const admin = createClient(Deno.env.get('STITCHDB_URL')!, Deno.env.get('STITCHDB_SERVICE_ROLE_KEY')!)

Deno.serve(job(async (payload, ctx) => {
  const user = await ctx.step('load user', async () => {
    const { data, error } = await admin.from('profiles').select('id, email').eq('id', payload.user_id).single()
    if (error) throw error
    return data
  })
  await ctx.step('welcome e-mail', () => sendEmail(user.email, 'Welcome!'))
  await ctx.sleep('wait a day', '1d')
  await ctx.step('tips e-mail', () => sendEmail(user.email, 'Three tips'))
  return { sent: 2 }
}))

Deploy it like any function (stitchdb functions deploy onboarding). The function only answers calls from the job worker, which signs every call with your project's service key; other requests get 401.

#Start a run

From your server or another Edge Function, with the service key:

const admin = createClient(STITCHDB_URL, STITCHDB_SERVICE_ROLE_KEY)

const { data, error } = await admin.jobs.start('onboarding', { user_id }, {
  idempotencyKey: `onboarding:${user_id}`,   // the same job + key starts one run only
  delay: 60,                                    // seconds, or runAt: new Date(...)
  maxAttempts: 5,
})

await admin.jobs.get(data.id)                   // status, result or error, and every step
await admin.jobs.list({ job: 'onboarding', status: 'failed' })
await admin.jobs.cancel(data.id)
await admin.jobs.retry(data.id)                 // a failed run continues from the step that failed

From SQL, for example in a trigger or on a schedule with pg_cron:

select stitch_jobs_enqueue('onboarding', jsonb_build_object('user_id', new.id), 'onboarding:' || new.id);

select cron.schedule('nightly-report', '0 3 * * *', $$ select stitch_jobs_enqueue('nightly-report') $$);

stitch_jobs_enqueue(job, payload, idempotency_key, run_at) returns the run id and can be called by the project owner role and service_role. Or start a run in Studio with Start a run. Over HTTP the same API is at https://<ref>.p.stitchdb.com/jobs/v1:

Send the service key as apikey and Authorization: Bearer. Keys of signed-in users are refused, because a job runs with full access; to let users start jobs, call jobs.start() from an Edge Function that checks what they may do.

#How a run moves

Retries wait 5 s, 10 s, 20 s and so on, doubling up to an hour. If the function crashes, times out or is not deployed, that counts as a failed attempt of the current step. If the server stops while a step runs, the run is picked up again after about eight minutes.

#Job mode: long runs

Job mode is included from the Pro plan. Some work does not split into short steps: importing a large CSV, sending a batch of e-mails, calling an AI API for each of thousands of documents, syncing with another service's API. Job mode runs such a function in one call for up to 15 minutes on Pro and 60 minutes on Scale, in an isolate of its own, and shows its progress and log while it runs. Wrap the handler in longJob():

// functions/import-csv/index.ts
import { longJob, FatalJobError } from 'https://stitchdb.com/sdk/stitchdb-jobs.mjs'
import { createClient } from 'https://stitchdb.com/sdk/stitchdb-js.mjs'

const admin = createClient(Deno.env.get('STITCHDB_URL')!, Deno.env.get('STITCHDB_SERVICE_ROLE_KEY')!)

Deno.serve(longJob(async (payload, ctx) => {
  const res = await fetch(payload.url, { signal: ctx.signal })
  if (!res.ok) throw new FatalJobError(`Could not download the file: ${res.status}`)
  const rows = parseCsv(await res.text())

  let i = ctx.checkpoint?.next ?? 0                     // after an interruption, continue here
  for (; i < rows.length; i += 500) {
    if (ctx.signal.aborted) return { stopped: i }        // cancelled in Studio or with jobs.cancel()
    const { error } = await admin.from('contacts').upsert(rows.slice(i, i + 500))
    if (error) throw error
    ctx.progress(Math.min(1, (i + 500) / rows.length), `${Math.min(i + 500, rows.length)} of ${rows.length} rows`)
    ctx.save({ next: i + 500 })
  }
  ctx.log('imported', rows.length, 'rows')
  return { imported: rows.length }
}))

await admin.jobs.start('import-csv', { url }, { mode: 'long', maxAttempts: 2 })

const { data } = await admin.jobs.get(id)     // status, progress (0 to 1), message, seconds, checkpoint, result
const { data: log } = await admin.jobs.logs(id, { after: -1 })   // { lines, next, more }

From SQL: select stitch_jobs_enqueue_long('import-csv', jsonb_build_object('url', 'https://…')). Over HTTP: POST /runs with "mode": "long", and GET /runs/<id>/logs?after= for the log.

Time is counted from the start to the end of each run, in seconds, against the job mode minutes of your plan; when they are used up, new long runs fail until the 1st of the next month. Each run also counts as one job run. Cancelling stops the function within about five seconds. If the server stops during a run, it is started again from its last checkpoint (when maxAttempts allows) within about two minutes.

#Good to know

• Each step is one Edge Function invocation and counts toward your function invocations and limits (memory, CPU and time per invocation).
• A run has at most 1000 steps; payloads, step results and run results are JSON up to 256 KB. Store larger data in a table or in Storage and pass its id.
• Finished runs are kept for the history days of your plan, then deleted. Runs and steps count toward your database size.
• Turning jobs off stops the worker; runs keep their state and continue when you turn it on again.