Level Up Your AI Agents: Implementing Persistent Memory Systems for Stateful Test Infrastructure

The first version of an AI agent often feels magical. It remembers the current chat, calls a tool, and produces a useful result. Then the process restarts. The user returns tomorrow, a CI worker retries the task, or two tests run in parallel—and the agent behaves as if none of yesterday happened. A developer usually responds by appending more transcript to the prompt or saving one giant JSON file. That buys a demo, not dependable state.

The professional upgrade is not “install a vector database.” It is to define memory as infrastructure: a durable write contract, an isolation model, a retrieval policy, and tests for restarts, conflicts, expiry, and leakage. This guide builds that foundation with TypeScript, SQLite, FTS5, and Playwright. SQLite keeps the first deployment local and observable; the contracts transfer cleanly to Postgres or a managed retrieval service when scale requires it.

Why does an AI agent need persistent memory?

Persistent memory lets an agent recover verified, scoped facts after restarts without replaying every transcript or mixing data across users.

A model context window is working memory, not durable storage. It disappears after the request, and stuffing every past interaction into the next prompt makes relevance somebody else’s problem. Long context also does not guarantee recall. The LongMemEval paper evaluates 500 curated questions across extraction, multi-session reasoning, temporal reasoning, updates, and abstention; it reports a 30% accuracy drop for commercial assistants and long-context models over sustained interactions. That is a benchmark result, not a promise about your application, but it exposes the exact failure modes your tests should cover.

Purpose-built memory can reduce the amount of history sent back to the model. In the Mem0 paper, the authors report a 26% relative improvement over OpenAI Memory on their LLM-as-a-Judge metric, 91% lower p95 latency, and more than 90% token-cost savings compared with full-context processing on LOCOMO. Treat those as results for that system and benchmark—not universal production estimates. The useful lesson is architectural: select relevant evidence before inference.

Move from “save the chat” to a memory contract

A transcript records what was said. A production memory records what may be reused. Every item needs an owner, scope, observation time, validity window, provenance, confidence, and stable idempotency key. Without those fields, retrieval cannot distinguish “the customer prefers email” from “the customer preferred email before changing to SMS,” and a retry can write the same fact twice.

The append-only event is the source of truth. Search indexes and compact summaries are derived views that you can rebuild. This separation is why the system stays debuggable: if retrieval looks wrong, inspect the raw event, the current/superseded relationship, and the index independently. For a broader testing progression, pair this approach with our AI test infrastructure guide.

Example 1: Build an idempotent SQLite memory store

Install better-sqlite3, save this as memory-store.ts, and run it on Node 20+. WAL mode lets readers continue while a writer commits, while busy_timeout turns short lock contention into bounded waiting. The transaction makes the event and FTS index update atomic through SQLite’s trigger machinery.

// npm i better-sqlite3 && npm i -D @types/better-sqlite3
import Database from 'better-sqlite3';

export type MemoryInput = {
  id: string; // derive from task ID + tool-call ID for retry safety
  tenantId: string;
  agentId: string;
  scope: 'agent' | 'project' | 'global';
  kind: 'fact' | 'decision' | 'failure';
  content: string;
  observedAt: string;
  expiresAt?: string;
  supersedesId?: string;
};

export class MemoryStore {
  private db: Database.Database;

  constructor(filename: string) {
    this.db = new Database(filename);
    this.db.pragma('journal_mode = WAL');
    this.db.pragma('foreign_keys = ON');
    this.db.pragma('busy_timeout = 5000');
    this.db.exec(`
      CREATE TABLE IF NOT EXISTS memory_events (
        id TEXT PRIMARY KEY,
        tenant_id TEXT NOT NULL,
        agent_id TEXT NOT NULL,
        scope TEXT NOT NULL CHECK(scope IN ('agent','project','global')),
        kind TEXT NOT NULL CHECK(kind IN ('fact','decision','failure')),
        content TEXT NOT NULL CHECK(length(trim(content)) > 0),
        observed_at TEXT NOT NULL,
        expires_at TEXT,
        supersedes_id TEXT REFERENCES memory_events(id),
        created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
        CHECK(expires_at IS NULL OR expires_at > observed_at)
      );
      CREATE VIRTUAL TABLE IF NOT EXISTS memory_fts USING fts5(
        content, content='memory_events', content_rowid='rowid'
      );
      CREATE TRIGGER IF NOT EXISTS memory_ai AFTER INSERT ON memory_events BEGIN
        INSERT INTO memory_fts(rowid, content) VALUES (new.rowid, new.content);
      END;
    `);
  }

  append(input: MemoryInput): { id: string; inserted: boolean } {
    if (!input.id.trim() || !input.tenantId.trim() || !input.content.trim()) {
      throw new Error('id, tenantId, and non-empty content are required');
    }
    const write = this.db.transaction(() => {
      const existing = this.db
        .prepare('SELECT tenant_id, content FROM memory_events WHERE id = ?')
        .get(input.id) as { tenant_id: string; content: string } | undefined;

      if (existing) {
        if (existing.tenant_id !== input.tenantId || existing.content !== input.content) {
          throw new Error('Idempotency collision: event ID was reused with different data');
        }
        return { id: input.id, inserted: false };
      }

      this.db.prepare(`
        INSERT INTO memory_events
          (id, tenant_id, agent_id, scope, kind, content, observed_at, expires_at, supersedes_id)
        VALUES
          (@id, @tenantId, @agentId, @scope, @kind, @content, @observedAt, @expiresAt, @supersedesId)
      `).run({
        ...input,
        expiresAt: input.expiresAt ?? null,
        supersedesId: input.supersedesId ?? null,
      });
      return { id: input.id, inserted: true };
    });

    try {
      return write();
    } catch (error) {
      const message = error instanceof Error ? error.message : String(error);
      throw new Error('Memory write failed: ' + message, { cause: error });
    }
  }

  close(): void {
    if (this.db.open) this.db.close();
  }
}

The edge cases are deliberate. A replay with the same ID and payload is a safe no-op. Reusing the ID with different data is corruption and fails loudly. Empty content and impossible expiry ranges are rejected. A missing supersedesId fails its foreign key instead of creating an untraceable update. In a multi-process deployment, keep transactions short; WAL improves concurrency but SQLite still has one writer at a time.

What should persistent agent memory store?

Store reusable facts, decisions, failures, and provenance—not raw secrets, hidden reasoning, or every token the model happened to emit.

Store information that can change a later action: a verified user preference, a tool capability, a rejected migration, a known flaky selector, or a failure plus its confirmed fix. Keep the originating event or trace ID as provenance. Do not persist chain-of-thought, credentials, access tokens, or an unfiltered prompt. A memory pipeline is also a data-retention system, so consent, deletion, and tenant boundaries belong in its design.

• Facts: assertions that can be verified and later superseded.
• Decisions: chosen options plus constraints and the deciding actor.
• Failures: symptoms, diagnostic evidence, and a confirmed resolution—not guesses.
• Ephemeral state: short-lived values with an explicit expiry, never an implied “we will clean it later.”

Example 2: Retrieve bounded, current, tenant-safe evidence

Retrieval is an authorization boundary and a ranking step. The function below tokenizes untrusted search text instead of passing FTS operators directly, requires tenant and agent identity, excludes expired and superseded rows, and caps results. Save it as memory-search.ts.

import Database from 'better-sqlite3';

export type SearchContext = {
  tenantId: string;
  agentId: string;
  now?: string;
  limit?: number;
};

export type MemoryHit = {
  id: string;
  kind: string;
  content: string;
  observedAt: string;
  lexicalRank: number;
};

function safeFtsQuery(input: string): string | null {
  const tokens = input.toLowerCase().match(/[a-z0-9_]{2,}/g)?.slice(0, 8) ?? [];
  if (tokens.length === 0) return null; // edge case: emoji or punctuation only
  return tokens.map((token) => '"' + token.replaceAll('"', '""') + '"').join(' OR ');
}

export function searchMemory(
  filename: string,
  query: string,
  context: SearchContext,
): MemoryHit[] {
  if (!context.tenantId.trim() || !context.agentId.trim()) {
    throw new Error('tenantId and agentId are required for every memory read');
  }
  const ftsQuery = safeFtsQuery(query);
  if (!ftsQuery) return [];
  const limit = Math.min(Math.max(context.limit ?? 6, 1), 20);
  const now = context.now ?? new Date().toISOString();
  const db = new Database(filename, { readonly: true, fileMustExist: true });

  try {
    const rows = db.prepare(`
      SELECT m.id, m.kind, m.content, m.observed_at,
             bm25(memory_fts) AS lexical_rank
      FROM memory_fts
      JOIN memory_events m ON m.rowid = memory_fts.rowid
      WHERE memory_fts MATCH ?
        AND m.tenant_id = ?
        AND (m.agent_id = ? OR m.agent_id = '*')
        AND (m.expires_at IS NULL OR m.expires_at > ?)
        AND NOT EXISTS (
          SELECT 1 FROM memory_events newer WHERE newer.supersedes_id = m.id
        )
      ORDER BY lexical_rank ASC, m.observed_at DESC
      LIMIT ?
    `).all(ftsQuery, context.tenantId, context.agentId, now, limit) as Array<{
      id: string; kind: string; content: string;
      observed_at: string; lexical_rank: number;
    }>;

    return rows.map((row) => ({
      id: row.id,
      kind: row.kind,
      content: row.content,
      observedAt: row.observed_at,
      lexicalRank: row.lexical_rank,
    }));
  } catch (error) {
    const message = error instanceof Error ? error.message : String(error);
    throw new Error('Memory retrieval failed: ' + message, { cause: error });
  } finally {
    db.close();
  }
}

FTS5 is a strong first professional step because its matches are inspectable. It will miss paraphrases—“refund” may not match “reimbursement”—so evaluate real queries before adding embeddings. A mature hybrid pipeline runs lexical and vector retrieval in parallel, fuses ranks, then reranks a small candidate set. Keep the tenant filter before results leave storage; filtering inside a prompt is not security.

Also separate retrieval from reading. The retriever returns evidence IDs and timestamps. The prompt composer decides how much evidence fits the current token budget and tells the model to abstain when evidence is absent or contradictory. This makes “no relevant memory” a normal outcome instead of an invitation to hallucinate.

Example 3: Prove persistence, retries, expiry, and isolation with Playwright

Stateful infrastructure needs stateful tests, but not order-dependent tests. Each Playwright worker gets a unique temporary database. The first test closes and reopens the store to cross a real process-resource boundary; the others attack idempotency and tenant isolation. Save as memory.spec.ts beside the two modules.

// npm i -D @playwright/test tsx
import { test, expect } from '@playwright/test';
import { mkdtemp, rm } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { MemoryStore } from './memory-store';
import { searchMemory } from './memory-search';

let root = '';
let dbFile = '';
let store: MemoryStore | undefined;

test.beforeEach(async ({}, testInfo) => {
  root = await mkdtemp(join(tmpdir(), 'memory-' + testInfo.parallelIndex + '-'));
  dbFile = join(root, 'agent.sqlite');
  store = new MemoryStore(dbFile);
});

test.afterEach(async () => {
  const cleanupErrors: string[] = [];
  try {
    store?.close();
  } catch (error) {
    cleanupErrors.push('close: ' + String(error));
  }
  try {
    await rm(root, { recursive: true, force: true });
  } catch (error) {
    cleanupErrors.push('rm: ' + String(error));
  }
  if (cleanupErrors.length) throw new Error(cleanupErrors.join('; '));
});

test('retrieves a fact after the writer is closed and reopened', async () => {
  store!.append({
    id: 'task-42:tool-7',
    tenantId: 'shop-a',
    agentId: 'support-agent',
    scope: 'project',
    kind: 'decision',
    content: 'Refunds above 100 EUR require manager approval',
    observedAt: '2026-07-23T09:00:00.000Z',
  });
  store!.close();
  store = undefined; // catches accidental dependence on an in-memory cache

  expect(searchMemory(dbFile, 'refund manager', {
    tenantId: 'shop-a',
    agentId: 'support-agent',
    now: '2026-07-23T10:00:00.000Z',
  })).toEqual([
    expect.objectContaining({ id: 'task-42:tool-7', kind: 'decision' }),
  ]);
});

test('makes an exact retry harmless but rejects a changed payload', async () => {
  const event = {
    id: 'task-99:tool-2',
    tenantId: 'shop-a',
    agentId: 'support-agent',
    scope: 'agent' as const,
    kind: 'fact' as const,
    content: 'Customer prefers SMS',
    observedAt: '2026-07-23T09:00:00.000Z',
  };
  expect(store!.append(event).inserted).toBe(true);
  expect(store!.append(event).inserted).toBe(false);
  expect(() => store!.append({ ...event, content: 'Customer prefers email' }))
    .toThrow(/Idempotency collision/);
});

test('does not return another tenant or an expired memory', async () => {
  store!.append({
    id: 'private-a',
    tenantId: 'shop-a',
    agentId: 'support-agent',
    scope: 'agent',
    kind: 'fact',
    content: 'Refund passcode is ALPHA',
    observedAt: '2026-07-20T09:00:00.000Z',
    expiresAt: '2026-07-21T09:00:00.000Z',
  });

  expect(searchMemory(dbFile, 'refund passcode', {
    tenantId: 'shop-b',
    agentId: 'support-agent',
    now: '2026-07-23T10:00:00.000Z',
  })).toEqual([]);
  expect(searchMemory(dbFile, 'refund passcode', {
    tenantId: 'shop-a',
    agentId: 'support-agent',
    now: '2026-07-23T10:00:00.000Z',
  })).toEqual([]);
});

Run npx playwright test memory.spec.ts --workers=4 --repeat-each=10 locally and in CI. Repetition is useful here because lock and cleanup bugs are timing-sensitive, but do not publish an anecdotal pass count as a reliability statistic. Add a crash test at the service boundary: terminate the writer after beginning a transaction, reopen the database, and assert that either the complete event exists or none of it does.

The migration path: ship contracts before clever retrieval

Migrate in four controlled steps. First, instrument the current agent and capture candidate facts without injecting them back into prompts. Second, write events to the new store in shadow mode and compare them with existing behavior. Third, enable retrieval for one low-risk scope behind a feature flag. Finally, make memory available to actions only after isolation, update, abstention, and rollback tests pass.

• Version the memory schema and make migrations forward-only in deployments; rehearse restore on a copy.
• Log retrieved evidence IDs, not full sensitive contents, alongside the agent trace.
• Measure recall and precision on a hand-labeled query set from your own product before tuning top-k.
• Keep a kill switch that disables retrieval without blocking the underlying agent workflow.

The same discipline applies when moving from local SQLite to Postgres. Preserve the event ID, tenant key, validity semantics, and evidence response shape. Change the adapter, then run the same contract suite against both backends. If you are also upgrading browser checks, our Playwright sharding guide covers fixtures and CI isolation.

Troubleshooting persistent memory systems

“The event was written, but search returns nothing”

Query the base table by ID, then query the FTS table by rowid. If the event exists but the index row does not, a bulk import probably bypassed or preceded the trigger. Rebuild with FTS5’s rebuild command in a maintenance transaction. Also inspect expiry and supersession before blaming ranking.

“SQLite throws database is locked in CI”

Confirm every test has its own database path and every connection closes in finally. Enable WAL and a bounded busy timeout, but do not treat a larger timeout as a concurrency design. Long transactions, network calls inside transactions, and a shared test fixture are the usual culprits.

“The agent recalls the old preference”

Inspect the update chain. The new event must reference the exact old ID through supersedesId, and retrieval must exclude superseded rows. If two writers update concurrently, add a logical key plus expected-version check; otherwise both updates can be individually valid while disagreeing.

“Relevant paraphrases never match”

Build a failing evaluation set first. Then add synonym expansion or embeddings as a second retriever and retain FTS for exact identifiers, error codes, and names. Fuse ranks instead of comparing incomparable raw scores. A vector database cannot repair missing tenant filters or stale facts.

“Tests pass alone and fail in parallel”

Search for shared filenames, fixed event IDs, global clocks, and cleanup that runs before the last handle closes. Include the Playwright worker index in namespaces, generate test-owned IDs, and make time an explicit search parameter so expiry assertions do not depend on wall-clock timing.

Edge cases and gotchas to design now

• Concurrent updates: two new facts may supersede the same row. Use a logical key and optimistic version when only one successor is allowed.
• Clock skew: producer timestamps can arrive out of order. Store both observed and server-received time; define which drives expiry.
• Poisoned memory: tool output and user text are untrusted. Preserve provenance and never convert an instruction into a trusted system rule automatically.
• Legal deletion: append-only is an audit pattern, not an excuse to retain personal data forever. Purge rows, indexes, replicas, and backups according to policy.
• Embedding drift: record the embedding model and dimension. Re-index into a new version instead of mixing vectors silently.
• No evidence: retrieval must return an empty set and the agent must abstain. “Closest” does not mean “relevant.”

Your production-ready definition of done

You have leveled up when memory survives a restart, retries are harmless, outdated facts lose to current ones, expired facts disappear, one tenant cannot retrieve another’s data, and every injected statement points back to evidence. The database choice matters less than preserving those invariants.

Start with the boring version you can inspect. Append events. Retrieve a small, authorized set. Test the lifecycle rather than a single happy-path answer. Once you own those contracts, embeddings, knowledge graphs, and managed memory platforms become useful upgrades instead of expensive ways to hide an undefined state model.