Why AI Agent Development Is Stalling: A Senior Architect’s Guide to Testing Through the Plateau

Your agent passes the demo. It answers the golden questions, calls the weather tool, and completes the happy path in staging. Then a customer asks the same thing with an empty account, the tool returns HTTP 429, or a previous turn leaves stale state in memory. The agent loops, performs a duplicate side effect, or confidently reports work it never completed.

Teams often call this an intelligence plateau and reach for a newer model, a longer system prompt, or another orchestration framework. That diagnosis is incomplete. The agent is no longer a function that maps input to output. It is a stateful control loop spanning a probabilistic planner, tool protocols, external services, memory, retries, and user-visible effects. A prompt-only test observes one frame of a film.

The uncomfortable evidence is already visible. A METR randomized controlled trial had 16 experienced open-source developers complete 246 real tasks; with early-2025 AI tools they took 19% longer, even though they predicted a 24% speedup. METR explicitly warns that coding benchmarks trade realism for scale. Read the METR study. The lesson is not that AI cannot help. It is that capability scores do not automatically become reliable system throughput.

Why do agent tests pass while the product still fails?

Because deterministic assertions observe one output, while agent failures emerge across tool calls, state, timing, and recovery paths.

Traditional browser tests assume that the same state and action should produce a narrow, predictable result. Agent systems add at least four sources of variation: model sampling, retrieved context, tool availability, and asynchronous timing. Exact-text assertions turn valid paraphrases into false failures, while a permissive “response exists” assertion lets fabricated success pass.

Evaluation design can also create a false plateau. OpenAI originally created SWE-bench Verified by reviewing 1,699 problems and selecting 500. A later audit of 138 problems that a model did not consistently solve found material test-design or problem-description issues in 59.4% of that audited subset. The published audit is a sharp reminder: when progress stalls, audit the oracle before blaming the system under test.

What should an AI agent test suite measure?

Measure outcome invariants, tool contracts, trace quality, safety boundaries, and recovery behavior—not exact wording from one model run.

Start with the user-visible outcome, then work backward through the trace. If the request is “cancel order 4821,” the invariant is not that the final message contains “cancelled.” The order must transition once, only an authorized account may trigger it, an already-cancelled order must remain unchanged, and the response must not claim success after a tool error. Those assertions survive model upgrades because they describe the product contract.

A useful pyramid has four layers. At the base, unit-test parsers, policy functions, and state reducers. Next, contract-test every tool boundary with recorded or synthetic responses. Above that, replay complete traces without contacting the model. At the top, run a small number of live-model journeys and an offline evaluation set. If every commit starts with hundreds of paid, nondeterministic model calls, failures will be slow and hard to reproduce.

Example 1: test the tool contract before testing the model

The first example is a complete Vitest file for an order-cancellation tool adapter. It validates authorization, input shape, upstream errors, empty-body responses, and duplicate requests. The model is deliberately absent: no planner can compensate for an adapter that converts a 503 into “success.”

// tests/cancel-order.contract.test.ts
// Run: npm i -D vitest && npx vitest run
import { describe, expect, it, vi } from 'vitest';

type CancelResult =
  | { ok: true; orderId: string; status: 'cancelled' | 'already_cancelled' }
  | { ok: false; code: string; retryable: boolean };

async function cancelOrder(
  input: { orderId?: string; accountId?: string; idempotencyKey?: string },
  fetcher: typeof fetch = fetch,
): Promise<CancelResult> {
  if (!input.orderId || !/^[0-9]+$/.test(input.orderId)) {
    return { ok: false, code: 'INVALID_ORDER_ID', retryable: false };
  }
  if (!input.accountId || !input.idempotencyKey) {
    return { ok: false, code: 'MISSING_AUTH_CONTEXT', retryable: false };
  }

  try {
    const response = await fetcher(
      'https://orders.internal/v1/orders/' + input.orderId + '/cancel',
      {
        method: 'POST',
        headers: {
          'x-account-id': input.accountId,
          'idempotency-key': input.idempotencyKey,
        },
      },
    );

    if (response.status === 409) {
      return { ok: true, orderId: input.orderId, status: 'already_cancelled' };
    }
    if (!response.ok) {
      return {
        ok: false,
        code: 'UPSTREAM_' + response.status,
        retryable: response.status === 429 || response.status >= 500,
      };
    }

    // Edge case: a proxy may return 204 instead of the documented JSON body.
    if (response.status === 204) {
      return { ok: true, orderId: input.orderId, status: 'cancelled' };
    }
    const body = (await response.json()) as { status?: string };
    if (body.status !== 'cancelled') {
      return { ok: false, code: 'INVALID_UPSTREAM_BODY', retryable: false };
    }
    return { ok: true, orderId: input.orderId, status: 'cancelled' };
  } catch (error) {
    const code = error instanceof Error ? error.name : 'UNKNOWN_NETWORK_ERROR';
    return { ok: false, code, retryable: true };
  }
}

describe('cancelOrder tool contract', () => {
  it('preserves idempotent already-cancelled state', async () => {
    const fetcher = vi.fn().mockResolvedValue(new Response(null, { status: 409 }));
    const result = await cancelOrder(
      { orderId: '4821', accountId: 'acct-7', idempotencyKey: 'run-123' },
      fetcher,
    );
    expect(result).toEqual({ ok: true, orderId: '4821', status: 'already_cancelled' });
  });

  it('marks rate limits retryable without claiming success', async () => {
    const fetcher = vi.fn().mockResolvedValue(new Response('busy', { status: 429 }));
    const result = await cancelOrder(
      { orderId: '4821', accountId: 'acct-7', idempotencyKey: 'run-124' },
      fetcher,
    );
    expect(result).toEqual({ ok: false, code: 'UPSTREAM_429', retryable: true });
  });

  it('rejects malformed IDs before any network side effect', async () => {
    const fetcher = vi.fn();
    expect(await cancelOrder({ orderId: '../admin' }, fetcher)).toMatchObject({
      ok: false,
      code: 'INVALID_ORDER_ID',
    });
    expect(fetcher).not.toHaveBeenCalled();
  });
});

This is where Selenium-era instincts still pay off: control the boundary and make the failure observable. The gotcha is mocking too high. If you mock cancelOrderitself, you never exercise HTTP status mapping, header propagation, or JSON parsing—the exact seams that break when an API evolves.

Example 2: replay traces and assert causal invariants

A trace is an event log, not a screenshot. Give every run a correlation ID and record the model request hash, retrieved document IDs, tool name, redacted arguments, result status, latency, state transition, and final claim. Never store raw credentials or unrestricted customer data. This standalone TypeScript program validates JSONL traces in CI.

// scripts/validate-agent-trace.ts
// Run: npm i -D tsx && npx tsx scripts/validate-agent-trace.ts traces/run.jsonl
import { readFile } from 'node:fs/promises';

type Event = {
  runId: string;
  seq: number;
  type: 'tool_call' | 'tool_result' | 'final';
  tool?: string;
  callId?: string;
  ok?: boolean;
  claim?: 'completed' | 'failed' | 'needs_input';
};

function parseEvents(text: string): Event[] {
  return text.split(/\r?\n/).flatMap((line, index) => {
    if (!line.trim()) return []; // Edge case: trailing or blank JSONL lines.
    try {
      const value = JSON.parse(line) as Partial<Event>;
      if (!value.runId || !Number.isInteger(value.seq) || !value.type) {
        throw new Error('missing runId, integer seq, or type');
      }
      return [value as Event];
    } catch (error) {
      const message = error instanceof Error ? error.message : String(error);
      throw new Error('Invalid trace line ' + (index + 1) + ': ' + message);
    }
  });
}

export function validateTrace(events: Event[]): string[] {
  const errors: string[] = [];
  if (events.length === 0) return ['trace is empty'];

  const runIds = new Set(events.map((event) => event.runId));
  if (runIds.size !== 1) errors.push('trace mixes multiple run IDs');

  const seenSeq = new Set<number>();
  const calls = new Map<string, Event>();
  const results = new Map<string, Event>();

  for (const event of events) {
    if (seenSeq.has(event.seq)) errors.push('duplicate sequence ' + event.seq);
    seenSeq.add(event.seq);

    if (event.type === 'tool_call') {
      if (!event.callId || !event.tool) errors.push('tool_call missing callId or tool');
      else if (calls.has(event.callId)) errors.push('duplicate side-effect call ' + event.callId);
      else calls.set(event.callId, event);
    }
    if (event.type === 'tool_result') {
      if (!event.callId) errors.push('tool_result missing callId');
      else results.set(event.callId, event);
    }
  }

  for (const callId of calls.keys()) {
    if (!results.has(callId)) errors.push('tool call has no result: ' + callId);
  }

  const final = events.findLast((event) => event.type === 'final');
  if (!final) errors.push('trace has no final event');
  const failedTool = [...results.values()].some((event) => event.ok === false);
  if (final?.claim === 'completed' && failedTool) {
    errors.push('agent claimed completion after a failed tool');
  }
  return errors;
}

async function main(): Promise<void> {
  const path = process.argv[2];
  if (!path) throw new Error('Usage: validate-agent-trace <trace.jsonl>');
  const events = parseEvents(await readFile(path, 'utf8'));
  const errors = validateTrace(events);
  if (errors.length) {
    console.error(errors.join('\n'));
    process.exitCode = 1;
  } else {
    console.log('Trace is causally valid: ' + path);
  }
}

main().catch((error: unknown) => {
  console.error(error instanceof Error ? error.stack : error);
  process.exitCode = 1;
});

Notice what this does not grade: eloquence. It proves causal consistency. A completed claim must be backed by successful effects; every call must terminate; duplicate call IDs are rejected. Extend the validator with a per-run tool-call budget, allowed transition graph, and policy decision IDs. For a deeper browser-observability pattern, see our flaky test debugging guide.

Example 3: inject faults at the browser boundary

Happy-path mocks teach the agent UI that infrastructure never fails. Playwright routing can force the cases that are expensive or unsafe to create in production: 429 responses, dropped connections, delayed results, and duplicated submissions. The following complete spec assumes a chat page with accessible labels and a cancellation workflow.

// tests/agent-recovery.spec.ts
// Run against your app: BASE_URL=http://localhost:3000 npx playwright test
import { expect, test } from '@playwright/test';

const baseURL = process.env.BASE_URL ?? 'http://localhost:3000';

test('does not fabricate success after a retryable tool failure', async ({ page }) => {
  const pageErrors: string[] = [];
  page.on('pageerror', (error) => pageErrors.push(error.message));

  await page.route('**/api/tools/cancel-order', async (route) => {
    try {
      const request = route.request();
      const body = request.postDataJSON() as { orderId?: string };
      if (body.orderId !== '4821') {
        // Edge case: fail loudly when the planner mutates a validated identifier.
        await route.fulfill({
          status: 400,
          contentType: 'application/json',
          body: JSON.stringify({ code: 'UNEXPECTED_ORDER_ID' }),
        });
        return;
      }
      await route.fulfill({
        status: 429,
        headers: { 'retry-after': '30' },
        contentType: 'application/json',
        body: JSON.stringify({ code: 'RATE_LIMITED' }),
      });
    } catch (error) {
      await route.abort('failed');
      throw error;
    }
  });

  await page.goto(baseURL + '/agent');
  await page.getByLabel('Message').fill('Cancel order 4821');
  await page.getByRole('button', { name: 'Send' }).click();

  await expect(page.getByRole('status')).toContainText(/try again|temporarily unavailable/i);
  await expect(page.getByText(/order 4821 (was|has been) cancelled/i)).toHaveCount(0);
  await expect(page.getByRole('button', { name: /retry/i })).toBeVisible();
  expect(pageErrors).toEqual([]);
});

test('deduplicates a double submit before the side effect', async ({ page }) => {
  let calls = 0;
  await page.route('**/api/tools/cancel-order', async (route) => {
    calls += 1;
    await new Promise((resolve) => setTimeout(resolve, 150));
    await route.fulfill({
      status: 200,
      contentType: 'application/json',
      body: JSON.stringify({ orderId: '4821', status: 'cancelled' }),
    });
  });

  await page.goto(baseURL + '/agent');
  await page.getByLabel('Message').fill('Cancel order 4821');
  const send = page.getByRole('button', { name: 'Send' });
  await send.dblclick(); // Edge case: impatient user or duplicated UI event.

  await expect(page.getByText(/order 4821 (was|has been) cancelled/i)).toBeVisible();
  expect(calls).toBe(1);
});

Keep trace capture on for failure diagnosis. Playwright recommends recording traces on the first CI retry, or retaining them on failure when retries are disabled, because always-on tracing is performance-heavy. For agent tests, be careful: a retry can conceal model variance or repeat a side effect. Prefer no automatic retry for live-effect tests, and use a sandbox plus idempotency keys if a retry is unavoidable. Our agent workflow testing checklist maps these controls to a release gate.

Build an evaluation set that survives model upgrades

Treat evaluation cases like test assets, not a spreadsheet of clever prompts. Each case needs a version, initial state, permissions, input, available tools, injected conditions, required invariants, forbidden outcomes, and a reason it exists. Store the exact model and prompt versions with the result. Otherwise, a score change cannot be attributed.

• Representative cases: common user journeys drawn from sanitized production patterns, including multilingual and ambiguous requests.
• Boundary cases: empty retrieval, maximum context, zero tool results, Unicode identifiers, expired credentials, and already-completed work.
• Adversarial cases: prompt injection in retrieved text, attempts to cross tenant boundaries, and instructions hidden in tool output.
• Recovery cases: 429, 500, timeout, malformed JSON, partial stream, duplicate event delivery, and process restart after a side effect.

Split release gates into “must never regress” and “quality trend.” Authorization bypass, duplicate payment, secret disclosure, or a false completion claim is a hard failure on one occurrence. Tone and helpfulness are distributions; compare them across repeated runs with confidence intervals rather than declaring victory from one sample. If you use an LLM judge, calibrate it against blinded human labels, randomize answer order, and keep a deterministic rubric. A judge is another probabilistic component, not ground truth.

Troubleshooting: diagnose the plateau instead of tuning blindly

When a suite becomes noisy, freeze prompt changes and classify the failure at the first divergence in the trace. The final bad sentence is usually downstream evidence, not the root cause.

• Passes locally, fails in CI: compare locale, timezone, model alias resolution, secret availability, network policy, dependency lock, and seeded fixtures. Record environment fingerprints; do not “fix” this with more retries.
• Agent claims success after a tool error: inspect whether the adapter returned an error as plain text instead of a typed failure. Enforce a result union and block the “completed” state unless the required effect is acknowledged.
• Only long conversations fail: log context item IDs and token allocation by category. Look for truncated policies, stale summaries, duplicated memories, and retrieval ordering changes. Test just below and above the limit.
• Intermittent duplicate actions: correlate model calls, queue delivery, browser submission, and tool requests. The edge may be a retried event, not the model. Require an idempotency key tied to the logical operation.
• Evaluation score drops after a “better” prompt: slice by case type and inspect judge disagreement. The prompt may help ordinary cases while breaking refusal or recovery behavior; one aggregate score hides the trade.
• Trace replay passes, live run fails: your replay omits timing or concurrency. Add virtual time, event ordering, cancellation, and parallel tool calls; preserve the raw event order before normalizing the trace.

The senior architect’s release gate

Before promoting an agent, require green deterministic contracts, no hard-safety regression, replay compatibility for stored traces, bounded tool calls, and a reviewed statistical comparison on the evaluation set. Canary the new configuration by tenant or workflow, monitor outcome invariants rather than token-level similarity, and keep a one-step rollback for model, prompt, tools, and memory schema.

Also test the test system. Seed one known-bad trace and confirm the gate rejects it. Expire a fixture intentionally. Rotate a tool schema. If the pipeline stays green, you have a dashboard, not a control. The most dangerous agent failure is not a red test; it is a green test that observed the wrong thing.

The plateau is where demos stop being the bottleneck and quality engineering becomes the product. Teams that cross it stop asking, “Did the agent say the right sentence?” They ask, “Did the system produce an authorized, observable, recoverable outcome under the conditions we know will occur?” That question is less glamorous than another prompt rewrite. It is also how an impressive prototype becomes dependable software.