Orchestrating Complex Logic in Base44: Multi-Agent Handoffs Without Rate-Limit Chaos

The first version of a Base44 agent workflow feels magical: one prompt researches a lead, another drafts an email, and a third updates the CRM. Then a webhook arrives twice, the email API answers with 429, two workers grab the same job, and your “autonomous team” sends Marta three identical intros before lunch.

This is not an AI intelligence problem. It is a distributed-systems problem wearing a very fashionable hoodie. The fix is to stop passing invisible context from prompt to prompt and make every transition durable, inspectable, and safe to repeat. Base44 gives you the right primitives: entities for state, Deno-powered backend functions for protected logic, automations for triggers, and SDK modules for agents and integrations. You supply the orchestration discipline.

Why do Base44 multi-agent handoffs break in production?

They break when transient agent output is mistaken for workflow state; persist every transition before triggering the next agent.

A chat transcript is useful evidence, but it is a terrible scheduler. A robust workflow has a single durable record with a workflow ID, current step, status, version, attempt count, and idempotency key. Each specialist receives a compact input contract and returns a compact output contract. The coordinator—not the model—decides whether the contract is valid and which state transition is allowed.

Base44’s official developer documentation says its agent module manages conversations and messages, while backend functions can call app data and integrations under user or service-role permissions. That separation matters. Agents reason; functions enforce invariants. If you want a refresher on separating generated UI from guarded backend behavior, see our agent orchestration architecture deep dive.

The architecture: ledger, coordinator, workers

Use three layers. The ledger is a Base44 entity such as WorkflowRun plus a WorkflowStep entity. The coordinator validates transitions and creates work. Workers claim a step, call one agent or provider, store the result, and exit. A scheduled automation rescues queued or expired work. This is deliberately boring. Boring is what lets you sleep after Product Hunt day.

• WorkflowRun: id, status, currentStep, input, result, version, createdBy, timestamps.
• WorkflowStep: workflowId, kind, status, payload, output, attempt, idempotencyKey, leaseUntil, error.
• Coordinator: the only code allowed to advance the workflow state machine.
• Worker: performs one bounded side effect and reports a typed result.

Keep the state machine explicit: queued → running → succeeded, with retry_wait, failed, and cancelled as real states. Never overload “pending” to mean queued, sleeping, blocked, and maybe finished. Ambiguous states create ambiguous recovery.

Example 1: accept work once, even when the webhook repeats

Create WorkflowRun and WorkflowStep entities with the fields above and a unique idempotencyKey. This Base44 backend function authenticates the caller, validates the body, checks replay safety, and creates the first step. Direct webhooks have no signed-in user, so production webhook routes should also verify the provider signature before using service-role access.

// base44/functions/startLeadWorkflow/entry.ts
import { createClientFromRequest } from "npm:@base44/sdk";

type StartBody = {
  leadId: string;
  idempotencyKey: string;
  objective?: string;
};

function isStartBody(value: unknown): value is StartBody {
  if (!value || typeof value !== "object") return false;
  const v = value as Record<string, unknown>;
  return typeof v.leadId === "string" && v.leadId.length > 0 &&
    typeof v.idempotencyKey === "string" && v.idempotencyKey.length >= 16 &&
    (v.objective === undefined || typeof v.objective === "string");
}

Deno.serve(async (req) => {
  if (req.method !== "POST") {
    return Response.json({ error: "Method not allowed" }, {
      status: 405,
      headers: { Allow: "POST" },
    });
  }

  try {
    const base44 = createClientFromRequest(req);
    const user = await base44.auth.me();
    if (!user) return Response.json({ error: "Unauthorized" }, { status: 401 });

    let body: unknown;
    try { body = await req.json(); }
    catch { return Response.json({ error: "Invalid JSON" }, { status: 400 }); }
    if (!isStartBody(body)) {
      return Response.json({ error: "leadId and a 16+ char idempotencyKey are required" }, { status: 422 });
    }

    const db = base44.asServiceRole.entities;
    const existing = await db.WorkflowRun.filter({ idempotencyKey: body.idempotencyKey });
    if (existing.length > 0) {
      return Response.json({ workflowId: existing[0].id, replayed: true }, { status: 200 });
    }

    const run = await db.WorkflowRun.create({
      status: "queued",
      currentStep: "research",
      leadId: body.leadId,
      objective: body.objective?.slice(0, 1000) ?? "Qualify lead",
      idempotencyKey: body.idempotencyKey,
      version: 1,
      createdBy: user.id,
    });

    await db.WorkflowStep.create({
      workflowId: run.id,
      kind: "research",
      status: "queued",
      attempt: 0,
      idempotencyKey: body.idempotencyKey + ":research",
    });

    return Response.json({ workflowId: run.id, replayed: false }, { status: 202 });
  } catch (error) {
    console.error("startLeadWorkflow failed", error);
    return Response.json({ error: "Unable to start workflow" }, { status: 500 });
  }
});

Edge case: the read-then-create check can still race if two identical requests arrive together. The entity schema must enforce uniqueness on idempotencyKey; catch that unique violation and fetch the winner. Application checks improve the message. Database constraints provide correctness.

What is the safest way to handle API rate limits?

Queue the call, honor Retry-After, use capped exponential backoff with jitter, and retry only demonstrably idempotent operations.

A 429 is feedback, not a challenge. The provider is telling you capacity is unavailable. Retrying instantly multiplies the load and aligns every worker into a retry storm. AWS’s published guidance recommends progressively longer waits, random jitter, and a maximum attempt or elapsed-time limit. The deeper rule is equally important: retry at one layer. If the SDK, worker, coordinator, and UI all retry, three attempts per layer can quietly become dozens of calls.

Code comparison: the tempting retry versus the production retry

Example 2: a bounded provider client with Retry-After and jitter

This helper is suitable inside a Base44 backend worker. It times out each request, refuses to retry permanent client errors, supports both forms of Retry-After, and sends a stable provider idempotency key. The workflow step persists between invocations; the helper only handles short transient retries that fit comfortably inside the function budget.

type CallOptions = {
  url: string;
  token: string;
  body: unknown;
  idempotencyKey: string;
  maxAttempts?: number;
};

function retryAfterMs(value: string | null): number | null {
  if (!value) return null;
  const seconds = Number(value);
  if (Number.isFinite(seconds) && seconds >= 0) return seconds * 1000;
  const date = Date.parse(value);
  return Number.isNaN(date) ? null : Math.max(0, date - Date.now());
}

export async function callProvider<T>(options: CallOptions): Promise<T> {
  const maxAttempts = Math.min(Math.max(options.maxAttempts ?? 4, 1), 6);
  if (!options.url.startsWith("https://")) throw new Error("HTTPS endpoint required");
  if (options.idempotencyKey.length < 16) throw new Error("Weak idempotency key");

  let lastError: Error = new Error("Provider call did not run");
  for (let attempt = 0; attempt < maxAttempts; attempt++) {
    try {
      const response = await fetch(options.url, {
        method: "POST",
        headers: {
          Authorization: "Bearer " + options.token,
          "Content-Type": "application/json",
          "Idempotency-Key": options.idempotencyKey,
        },
        body: JSON.stringify(options.body),
        signal: AbortSignal.timeout(15_000),
      });

      if (response.ok) {
        const text = await response.text();
        if (!text) throw new Error("Provider returned an empty success body");
        try { return JSON.parse(text) as T; }
        catch { throw new Error("Provider returned invalid JSON"); }
      }

      const excerpt = (await response.text()).slice(0, 500);
      const retryable = response.status === 408 || response.status === 429 || response.status >= 500;
      if (!retryable) throw new Error("Permanent HTTP " + response.status + ": " + excerpt);

      lastError = new Error("Transient HTTP " + response.status + ": " + excerpt);
      if (attempt === maxAttempts - 1) break;
      const cap = Math.min(1000 * 2 ** attempt, 20_000);
      const hinted = retryAfterMs(response.headers.get("retry-after"));
      const delay = hinted === null ? Math.floor(Math.random() * cap) : Math.min(hinted, 30_000);
      await new Promise((resolve) => setTimeout(resolve, delay));
    } catch (error) {
      lastError = error instanceof Error ? error : new Error(String(error));
      if (lastError.message.startsWith("Permanent HTTP")) throw lastError;
      if (attempt === maxAttempts - 1) break;
      const delay = Math.floor(Math.random() * Math.min(1000 * 2 ** attempt, 20_000));
      await new Promise((resolve) => setTimeout(resolve, delay));
    }
  }
  throw new Error("Provider exhausted retries: " + lastError.message);
}

Gotcha: some APIs accept an idempotency header but retain keys only for a limited window. Persist the provider response ID as well. Also, a timeout is an unknown outcome—not proof of failure. The provider may have completed the side effect after your client disconnected. Query by idempotency key before repeating a charge, email, or publish action.

Design the handoff as a versioned contract

Do not hand the copywriter agent “whatever the researcher said.” Give it a schema: summary, evidence URLs, confidence, blocked reason, and requested next action. Bound string and array sizes because model output is untrusted input. Include a schemaVersion so you can reject old workers after a deployment instead of quietly interpreting their fields incorrectly.

A handoff should also be causally linked. The worker reports the workflowId, stepId, and the workflow version it read. If the coordinator now sees a different version, another transition won. Store the late result for audit, but do not advance the workflow from stale state.

Example 3: validate a handoff and reject stale workers

// base44/functions/completeHandoff/entry.ts
import { createClientFromRequest } from "npm:@base44/sdk";

type Handoff = {
  workflowId: string;
  stepId: string;
  expectedVersion: number;
  schemaVersion: 1;
  summary: string;
  evidence: string[];
  nextAction: "draft" | "human_review" | "stop";
};

function parseHandoff(value: unknown): Handoff | null {
  if (!value || typeof value !== "object") return null;
  const v = value as Record<string, unknown>;
  const evidence = Array.isArray(v.evidence) ? v.evidence : [];
  if (typeof v.workflowId !== "string" || typeof v.stepId !== "string" ||
      !Number.isInteger(v.expectedVersion) || v.schemaVersion !== 1 ||
      typeof v.summary !== "string" || v.summary.length === 0 || v.summary.length > 4000 ||
      evidence.length > 10 || !evidence.every((x) => typeof x === "string" && x.length <= 500) ||
      !["draft", "human_review", "stop"].includes(String(v.nextAction))) return null;
  return v as unknown as Handoff;
}

Deno.serve(async (req) => {
  if (req.method !== "POST") return Response.json({ error: "Method not allowed" }, { status: 405 });
  try {
    const base44 = createClientFromRequest(req);
    const user = await base44.auth.me();
    if (!user) return Response.json({ error: "Unauthorized" }, { status: 401 });
    const body = parseHandoff(await req.json().catch(() => null));
    if (!body) return Response.json({ error: "Invalid handoff contract" }, { status: 422 });

    const db = base44.asServiceRole.entities;
    const run = await db.WorkflowRun.get(body.workflowId);
    const step = await db.WorkflowStep.get(body.stepId);
    if (!run || !step || step.workflowId !== run.id) {
      return Response.json({ error: "Workflow or step not found" }, { status: 404 });
    }
    if (run.createdBy !== user.id) {
      return Response.json({ error: "Forbidden" }, { status: 403 });
    }
    if (step.status === "succeeded") {
      return Response.json({ workflowId: run.id, replayed: true }, { status: 200 });
    }
    if (run.version !== body.expectedVersion || run.currentStep !== step.kind) {
      return Response.json({ error: "Stale handoff", actualVersion: run.version }, { status: 409 });
    }

    await db.WorkflowStep.update(step.id, {
      status: "succeeded",
      output: { summary: body.summary, evidence: body.evidence },
      completedAt: new Date().toISOString(),
    });

    const terminal = body.nextAction === "stop";
    await db.WorkflowRun.update(run.id, {
      status: terminal ? "succeeded" : "queued",
      currentStep: body.nextAction,
      version: run.version + 1,
    });
    if (!terminal) {
      await db.WorkflowStep.create({
        workflowId: run.id,
        kind: body.nextAction,
        status: "queued",
        attempt: 0,
        idempotencyKey: run.id + ":" + (run.version + 1) + ":" + body.nextAction,
      });
    }
    return Response.json({ workflowId: run.id, next: body.nextAction });
  } catch (error) {
    console.error("completeHandoff failed", error);
    return Response.json({ error: "Could not complete handoff" }, { status: 500 });
  }
});

The example demonstrates optimistic concurrency, but the final protection must be atomic. If your entity API cannot conditionally update “where version equals expectedVersion,” move the claim/transition into the smallest supported atomic operation or add a unique transition key. Two separate updates can leave a succeeded step beside an unadvanced run after a crash. Recovery code should detect and reconcile that exact split-brain state.

Automations are triggers, not long-running workers

Base44 supports scheduled, entity-event, and connector automations. Each run is logged, which makes them a clean way to wake the worker when a step appears. But the documented three-minute ceiling means you should process a bounded batch, persist the cursor, and return. Sleeping for minutes inside a function wastes the budget and makes deploys painful.

Prefer nextRunAt over an in-memory sleep for long backoff. A scheduled sweeper queries retryable steps whose time has arrived, claims a small batch, and invokes the worker. Because the minimum schedule interval is five minutes, latency-sensitive workflows may use entity events for the first attempt and the scheduled sweeper as recovery. Read our resilient automation guide for the wider queue-and-recovery pattern.

Troubleshooting: debug state before prompts

When a workflow stalls, start with the ledger and function logs. Base44 records automation runs, and deployed backend-function logs include output, errors, and timing. Search by workflow ID, not by a human-readable lead name.

• Step stays queued: verify the automation is active, the event filter matches, and the worker’s last invocation is recent. Zero errors can mean the worker never ran.
• Duplicate emails or charges: confirm the same idempotency key reached the provider. If keys differ per retry, the mechanism is decorative.
• 429 loop: log status, parsed Retry-After, attempt, nextRunAt, and provider request ID. Check for retries at several layers.
• Random 401/403: distinguish user context from service role. SDK-invoked functions inherit the user; direct HTTP webhooks do not.
• Workflow advances twice: inspect version conflicts and unique transition constraints. A read followed by an unconditional update is not a lock.
• Success with empty output: validate response bodies. HTTP 200 proves transport success, not semantic completeness.
• Runs die near three minutes: reduce batch size, persist a cursor, and let the next invocation resume.

Edge cases that ambush otherwise good workflows

• Out-of-order completion: a slow research agent returns after a human already cancelled the run. Version checks must win over enthusiasm.
• Poison work: malformed input always fails. After a small attempt cap, move it to dead_letter with a safe diagnostic.
• Partial fan-out: four of five subtasks finish. Define whether the join requires all, a quorum, or a deadline plus partial result.
• Prompt injection through evidence: scraped pages are data, not instructions. Delimit them and restrict the agent’s tools and permissions.
• Cancellation during a call: you may not stop the provider. Mark the late response ignored and run compensation only when safe.
• Clock skew: prefer server timestamps and tolerate small differences when comparing leases or Retry-After dates.
• Schema deployment: old queued steps survive new code. Version contracts and support at least one migration path.

The indie-hacker version of “production-grade”

You do not need Kafka, Kubernetes, or a platform team. You need one source of truth, small functions, safe retries, and a screen that shows stuck work. Start with a linear workflow. Add fan-out only when independent steps genuinely save time. Add a human-review state before irreversible side effects. Cap spend and attempts at the workflow level, not merely per agent call.

The vibe-coding move is not avoiding engineering. It is spending engineering effort where it compounds. A polished prompt may improve one response. A durable handoff contract improves every run, makes failures explainable, and lets you swap models or providers without rebuilding the whole product.

Sources and implementation notes

Platform behavior and limits are based on the official Base44 backend functions documentation, automation reference, and agents SDK reference. Retry guidance follows the AWS Well-Architected Framework. Confirm current Base44 limits and SDK types before deployment because hosted platform details can change.