Getting Started

In this section, we will build in three parts.

• Job Manager (Kiribi worker): The center of the overview diagram
  • Receives and manages job queues
• Performer (Job processing worker): The lower part of the overview diagram
  • Processes job queues
• Enqueuer (Job enqueuing worker): The upper part of the overview diagram
  • Enqueues job queues

1. Create a Job Manager (Kiribi worker)

Use create-cloudflare-cli (C3) to build the base worker.
You will be asked for the project name, so feel free to set it as you like.

npm create cloudflare@latest -- --ts --type=hello-world

Install kiribi in the directory of the created project.

npm install kiribi

Create a D1 database and a queue for Kiribi.
You can set the database name and queue name as you like, but please copy the database_id displayed as it will be used in later steps.

npx wrangler d1 create kiribi-db
# copy the database_id

npx wrangler queues create kiribi-queue

Update wrangler.toml as follows:

name = "my-kiribi" # set the name of your worker freely
compatibility_date = "2024-04-03"
main = "src/index.ts"

[[d1_databases]]
binding = "KIRIBI_DB" # Be sure to set KIRIBI_DB for the d1 database binding
database_name = "kiribi-db" # The name of the d1 database you created
database_id = "xxxxxxxxxxxxxxxx" # The database_id of the d1 database you created
migrations_dir = './node_modules/kiribi/migrations'

[[queues.producers]]
binding = "KIRIBI_QUEUE" # Be sure to set KIRIBI_QUEUE for the queue binding
queue = "kiribi-queue" # The name of the queue you created

[[queues.consumers]]
queue = "kiribi-queue" # The name of the queue you created
max_retries = 5

[[services]]
binding = "KIRIBI" # Be sure to set KIRIBI for the service binding
service = "my-kiribi" # same as `name`

INFO

max_retries is a hard limit for the job retry setting. Please set it with some margin.
For example, if max_retries is 5, the number of retries will be 4.

Migrate the database.

If you want to run it locally:

npx wrangler d1 migrations apply kiribi-db --local

If you want to run it remotely:

npx wrangler d1 migrations apply kiribi-db --remote

Update the worker code in src/index.ts.

import { Kiribi } from 'kiribi'

export default class extends Kiribi {}

Deploy the worker.

npx wrangler deploy

INFO

If you deploy the worker for the first time, you may encounter an error like the following.

✘ [ERROR] A request to the Cloudflare API (/accounts/xxxxxxxxx/workers/scripts/my-kiribi) failed.

  workers.api.error.service_binding_error: could not resolve binding "KIRIBI": script
  "my-kiribi" not found [code: 10143]

If you encounter this error, comment out [[services]] in wrangler.toml, deploy, and then uncomment and deploy again.

[[services]]                                                        
binding = "KIRIBI" # Be sure to set KIRIBI for the service binding
service = "my-kiribi"
# [[services]]
# binding = "KIRIBI"
# service = "my-kiribi"

2. Create Performers (Job processing workers)

Create a worker to process jobs enqueued in Kiribi.
You can add this worker to the Kiribi worker created in the previous step, or you can create it as a separate worker.

Case of adding to the Kiribi worker

Add the following to src/index.ts.

import { Kiribi } from 'kiribi'
import { KiribiPerformer } from 'kiribi/performer'

export default class extends Kiribi {}

export class MyPerformer extends KiribiPerformer { 
  async perform(payload) {                         
    // Do something with the payload
    console.log('perform', payload)                
  }                                                
}                                                  

For more information on perform, click here.

Add the following to wrangler.toml.

name = "my-kiribi"
compatibility_date = "2024-04-03"
main = "src/index.ts"

# ...Omitted...

[[services]]
binding = "KIRIBI"
service = "my-kiribi"

[[services]]                                                    
binding = "MY_JOB" # You can name the binding whatever you want
service = "my-kiribi" # same as `name`
entrypoint = "MyPerformer" # the name of exported class

Case of creating a separate worker

If you want to separate the processing system from the Kiribi worker, do the following.
For example, create a worker named my-performing-worker. (The method of creating a worker is omitted.)

Update the src/index.ts of my-performer-worker as follows.

import { KiribiPerformer } from 'kiribi/performer'

// There must be at least one default export
export default class extends KiribiPerformer {
  async perform(payload) {
    // Do something with the payload
    console.log('perform', payload)
  }
}

// Add other workers as needed
export class FooPerformer extends KiribiPerformer {
  async perform(payload) {
    // Do something with the payload
    console.log('perform', payload)
  }
}

Add the following to the wrangler.toml of the Kiribi worker (my-kiribi).

name = "my-kiribi"
compatibility_date = "2024-04-03"
main = "src/index.ts"

# ...Omitted...

[[services]]
binding = "KIRIBI"
service = "my-kiribi"

[[services]]                                                        
binding = "MY_JOB" # You can name the binding whatever you want
service = "my-performing-worker" # The name of the performer worker

[[services]]                                                        
binding = "FOO_JOB"
service = "my-performing-worker"
entrypoint = "FooPerformer" # the name of exported class

3. Create Enqueuers (Job enqueuing workers)

Create a worker to enqueue jobs in Kiribi.
The method of creating a worker is omitted.

Update the wrangler.toml of the worker as follows.

name = "my-enqueuing-worker"
compatibility_date = "2024-04-03"
main = "src/index.ts"

[[services]]
binding = "KIRIBI" # Be sure to set KIRIBI for the service binding
service = "my-kiribi" # The name of the Kiribi worker

import { Kiribi } from 'kiribi'

interface Env {
  KIRIBI: Service<Kiribi>
}

export default {
  async fetch(req: Request, env: Env, ctx: ExecutionContext) {
    // Enqueue a job
    await env.KIRIBI.enqueue('MY_JOB', { key: 'value' })

    return new Response('OK')
  }
}

For more information on how to use it, click here.