Schema Design Patterns for Agent Memory

By Arc Labs Research10 min read

Why schema matters

A schema is a contract between memory writes and reads. Without one, you end up with a schemaless blob that's expensive to query and impossible to evolve.

Bad: Store everything as { content: string } and figure out structure on read. Good: Define upfront: fact vs. preference vs. event vs. entity, queryable fields, field cardinality.

Core types

Brain's memory model defines five semantic types. Use these as your schema foundation.

1. Fact

Persistent, low-cardinality assertions about the world.

type FactMemory = {
  type: "fact";
  content: string; // "User is based in San Francisco"
  confidence: number; // 0-1, learned from context
  source: "explicit" | "inferred" | "system";
  ttl?: DateTime; // Facts can expire
};

When to use: User location, expertise, preferences, constraints. Cardinality: 1–100 per user. Retrieval: Mostly keyword + metadata (location='SF').

Schema fields:

{
  type: "fact",
  category: "location" | "expertise" | "preference" | "constraint",
  field: string, // "home_city", "technical_stack", "dietary_pref"
  value: string | number | boolean,
  confidence: 0-1,
  source: string,
}

2. Preference

Stateful user choices that shape agent behavior.

type PreferenceMemory = {
  type: "preference";
  dimension: "communication" | "style" | "pace" | "format";
  value: string;
  strength: "soft" | "hard"; // hard = non-negotiable
  learned_from: string; // "user_stated" | "behavior"
};

When to use: Communication style (async/sync), response length, formality, timezone. Cardinality: 5–20 per user. Retrieval: Exact match on dimension (communication_style='async').

3. Event

Time-ordered occurrences with causality.

type EventMemory = {
  type: "event";
  action: string; // "user_asked", "agent_decided", "task_completed"
  timestamp: DateTime;
  outcome: "success" | "failure" | "partial";
  context: { [key: string]: any };
  next_action?: string; // Causal link
};

When to use: Conversation turns, task completions, errors, decisions. Cardinality: 100–10K per conversation. Retrieval: Temporal (last 10 events) + semantic (events about pricing).

4. Entity

Named things with attributes and relationships.

type EntityMemory = {
  type: "entity";
  entity_id: string; // "user_u1", "company_acme", "product_brain"
  entity_type: string; // "user", "company", "product"
  attributes: {
    name: string;
    category?: string;
    description?: string;
    [key: string]: any;
  };
  relationships?: {
    [relation_type]: string[]; // "works_at": ["company_acme"]
  };
};

When to use: People, companies, products, concepts that appear repeatedly. Cardinality: 10–500 per scope. Retrieval: Entity lookup (company='Acme') + graph traversal (employees of Acme).

5. Relation

Explicit edges between entities.

type RelationMemory = {
  type: "relation";
  subject_id: string; // "user_u1"
  predicate: string; // "works_at", "authored", "owns"
  object_id: string; // "company_acme"
  confidence: number;
  context?: string;
};

When to use: Explicit relationships (user works at company, message authored by user). Cardinality: 1–5 per entity pair. Retrieval: Graph queries (find all people who work at Acme).

Schema design checklist

1. Queryability

Every field you might filter by should be a first-class field, not buried in content.

// ❌ Wrong: buried in content
{
  type: "fact",
  content: "User is in San Francisco, timezone PST, works at Acme"
}

// ✓ Right: queryable fields
{
  type: "fact",
  category: "location",
  field: "home_city",
  value: "San Francisco",
  metadata: {
    timezone: "PST",
    company: "Acme",
  }
}

2. Cardinality awareness

High-cardinality fields (thousands of unique values per user) are expensive to index and filter.

Low cardinality (good):

  • communication_preference: "async" | "sync"
  • timezone: "PST" | "EST" | ... (< 50 values)
  • expertise_level: "junior" | "mid" | "senior"

Medium cardinality (OK):

  • home_city: San Francisco, NYC, London, ... (< 1000 values)
  • programming_language: Python, JS, Go, ... (< 100 values)

High cardinality (avoid indexing, use in metadata):

  • email_address: millions of unique values
  • user_comment: millions of unique values
  • product_sku: millions of unique values

3. Temporal fields

Always include timestamp and optional ttl. Memory without temporal context is noisy.

{
  type: "event",
  content: "User asked about pricing",
  timestamp: new Date("2026-05-12T14:30:00Z"),
  ttl: new Date("2026-05-19T14:30:00Z"), // Expires in 1 week
}

4. Versioning

Schema evolves. Plan for backward compatibility.

{
  type: "fact",
  schema_version: 2, // Increment when you add/remove fields
  content: "...",
  // v2 additions:
  category: "location",
  confidence: 0.95,
}

Anti-patterns

Anti-patternWhy it failsFix
Everything is contentCan't filter, all semantic searchUse typed fields + metadata
Over-normalized (100+ tables)Too many joins, slow queriesDenormalize; use JSON metadata
No TTLMemory grows unboundedSet TTL on transient types
High-cardinality indexesIndex explosion, slow writesKeep cardinality < 1000 per field
Mutable IDsRelationship links breakUse immutable UUIDs; never change
No timestampCan't answer "when did this happen?"Always include timestamp

Example: Multi-turn conversation schema

type ConversationSchema = {
  // Metadata (queryable)
  agent_id: string;
  user_id: string;
  conversation_id: string;
  session_start: DateTime;

  // Events (time-ordered)
  turns: Array<{
    type: "event";
    action: "user_message" | "agent_response" | "action_taken";
    timestamp: DateTime;
    content: string;
    embedding: Vector;
    metadata: {
      turn_number: number;
      intent?: string; // "question", "command", "clarification"
      sentiment?: "positive" | "neutral" | "negative";
      successful: boolean;
    };
  }>;

  // Facts (extracted, persistent)
  facts: Array<{
    type: "fact";
    field: "user_intent" | "user_preference" | "user_constraint";
    value: string;
    confidence: number;
    learned_turn: number;
  }>;

  // Entities (referenced throughout)
  entities: Array<{
    type: "entity";
    entity_id: string;
    entity_type: "user" | "product" | "company";
    attributes: { name: string; [key: string]: any };
  }>;
};

Queryability: Can filter by user_id, agent_id, intent, sentiment. Can retrieve by turn number (recency) or semantic search (what was the user's intent?).

Cardinality: Low (< 50 unique intents), medium (< 1000 users), high only in content (not indexed).

Schema evolution

Your schema will change. Plan for it.

Add a field

Old records have undefined for new fields. Backfill or use defaults.

// v1 → v2: add confidence field
type FactMemoryV1 = { type: "fact"; content: string };
type FactMemoryV2 = { type: "fact"; content: string; confidence: number };

// On read, migrate
function migrateV1toV2(fact: FactMemoryV1): FactMemoryV2 {
  return { ...fact, confidence: 0.5 }; // Default for backfilled
}

Remove a field

Mark deprecated, don't delete. Migrate readers to new field.

type FactMemoryV2 = {
  type: "fact";
  content: string; // Deprecated in v3, use `summary`
  summary?: string;
};

Rename a field

Create new field, migrate, deprecate old.

// v2: user_preference
// v3: communication_preference
type FactMemoryV3 = {
  communication_preference?: string; // v3+
  user_preference?: string; // deprecated, v2 compat
};

Testing your schema

Before shipping, validate:

  • Every queryable field is indexed
  • Cardinality of indexed fields < 10K per user
  • All records have timestamp
  • TTL is set for transient types
  • No field is both high-cardinality and indexed
  • Backward compatibility path exists for next version
  • You can answer: "How do I find X?" for every use case

Further reading

Related reading

Updates from the lab.

Engineering notes, research drops, occasional product updates. Roughly monthly.