Inngest - Patterns: Async + Event-Driven

Modern products have increased in both customization, notifications, and customized notifications. Let's take a common use case: X product adds a feature that enables their users to select when to post a status report to a connected Slack channel via the product's Slackbot. Imagine a sales product where some teams prefer their report on Monday or Friday and at 9am ET or 9am PT.

To start, a simple system may run a long, complex cron job that checks what is configured to scheduled at that given time slot, then runs all of the code in a loop. Individual errors will have to be caught and logged for future auditing and retries.

As volume increases, the long running job may be more prone to fail for whatever reason (e.g. memory leaks). Each individual task that is scheduled should be run independently, allowing for individual retries and parallelization. After all, this is a scheduling feature — if a scheduled post is 15 minutes late, is that acceptable for your user?

The key is combining a reliable scheduling system with a reliable system that can process tasks in parallel.

How to implement this pattern

At a basic level, you're combining a cron job with a queue and workers. You define 1 or more cron jobs then have each cron job publish n number of messages to a queue for each task that has been scheduled to run at that given time.

Then a worker will need to be set up to poll for messages and complete each task, all while handling failures, retries, and logging well.

How to implement with Inngest

Inngest supports scheduled functions and event-triggered functions. Combining the two enables you to fan-out functions to run in parallel. We'll define two these two functions:

typescript
import { createScheduledFunction, createFunction } from "inngest";
const inngest = new Inngest("Scheduling Backend");
// A scheduled function uses the current time to find notifications to send
const slackCron = createScheduledFunction("Slack Notification Cron",
  "0 9,12 * * MON,FRI",
  async () => {
    const day = new Date().getDay();
    const hour = new Date().getHours();
    const notifications = await getNotificationsToRun(day, hour);
    const payloads = notifications.map((notification) => ({
      name: "app/notification.dispatched",
      data: { notification },
    }));
    // Inngest can accept individual event payloads or an array for batching
    // up to 512kb in size
    await inngest.send(payloads);
    return `${notifications.length} notifications dispatched`;
  }
);
// A function runs for every app/notification.dispatched event to
// post the notification to Slack
const postSlackNotification = createFunction("Send Slack Notification",
  "app/notification.dispatched",
  async ({ event }) => {
    const reportData = getAccountReportData(event.data.notification.accountId);
    const result = await app.client.chat.postMessage({
      channel: event.data.notification.slackChannelId,
      blocks: generateReportSlackBlocks(reportData),
      // ...
    });
    return result;
  }
);

This is the system — both functions can even be defined in the same file to keep things simple and maintainable. This approach works well for systems that have commonly scheduled times, but for more flexible systems that are scheduled as one-off, non-repeated tasks you should review Patterns: Running at specific times.

Additional Resources

← View All Patterns