Long-Term Memory for Agents

By Arc Labs Research9 min read

The persistence problem

Agent memory has two timescales:

Session memory (seconds to minutes): conversation context, intermediate states. Fine if lost.

Long-term memory (hours to years): user preferences, learned behavior, decisions. Must survive restarts, crashes, and agent updates.

Most teams conflate them and use the same storage. This is a mistake.

Storage tiers

Tier 1: Session (in-memory, non-durable)

Use for: Conversation context, intermediate reasoning, temporary state.

const sessionMemory = new Map<string, Memory>();
// or
const sessionMemory = new Brain({ storage: "memory" }); // auto-cleared on exit

Durability: None. Lost on restart. Latency: < 1ms. Cost: Free (RAM). Cardinality: 100–1K facts per session.

Tier 2: Mid-term (local persistent, single-node)

Use for: Recent decisions, conversation summaries, learned preferences (days).

const midTermMemory = new Brain({
  endpoint: "http://localhost:9090", // self-hosted node, durable to disk (arena + WAL + redb)
  ttl: 7 * 24 * 60 * 60, // 7 days
});

Durability: Survives restart (disk-backed). Lost if disk fails. Latency: 5–50ms (local disk I/O). Cost: Disk space (~1MB per 100K facts). Cardinality: 1K–100K facts.

Tier 3: Long-term (managed persistent, replicated)

Use for: User profiles, historical facts, policies (months/years).

const longTermMemory = new Brain({
  apiKey: process.env.BRAIN_API_KEY,
  region: "us-east-1",
  replication: "multi-region", // survives data center failure
});

Durability: Survives any single failure (replicated, backed up). SLA-guaranteed. Latency: 50–200ms P99 (network + DB). Cost: $0.001 per 1K vectors/month (managed). Cardinality: 100K–10M facts per user.

Multi-tier architecture

Most production agents use all three:

// Session: conversation in this request
const session = new Brain({ storage: "memory" });

// Mid-term: recent decisions + learned preferences (7 days)
const midTerm = new Brain({ endpoint: "http://localhost:9090", ttl: 7 * 24 * 60 * 60 });

// Long-term: user profile + historical facts (permanent)
const longTerm = new Brain({
  apiKey: process.env.BRAIN_API_KEY,
  region: "us-east-1",
});

// On each turn:
async function agentLoop(input) {
  // Read from all tiers
  const sessionContext = await session.retrieve(input);
  const recentDecisions = await midTerm.retrieve(input);
  const userProfile = await longTerm.retrieve(input);

  // Combine
  const fullContext = [sessionContext, recentDecisions, userProfile];

  // Decide and store
  const decision = await llm.complete(fullContext);

  // Write to appropriate tiers
  await session.store({ content: decision, type: "decision" });
  if (important) {
    await midTerm.store({ content: decision, type: "decision" });
    await longTerm.store({ content: decision, type: "insight" });
  }
}

Eviction policies

Memory grows. You need a policy for what to forget.

LRU (Least Recently Used)

Forget what you haven't accessed in N days.

Best for: User-scoped memory (chat history, preferences). Tradeoff: Might forget important but rarely-accessed facts.

{
  eviction_policy: "lru",
  window_days: 30,
  // Keep facts accessed in last 30 days
}

TTL (Time To Live)

Forget specific memories after a fixed time.

Best for: Transient facts (conversation state, temporary preferences). Tradeoff: No automatic cleanup for long-lived memories.

{
  type: "conversation_turn",
  ttl: new Date(Date.now() + 24 * 60 * 60 * 1000), // 1 day
}

Priority-based

Forget low-confidence or low-impact memories first.

Best for: Mixed-importance memory (facts + events + preferences). Tradeoff: Requires predicting what's important.

{
  eviction_order: "confidence_desc", // Forget low-confidence first
  // or
  eviction_order: "type", // Forget events before facts
}

Merge + compress

Instead of evicting, compress old memories.

Best for: Long conversations (summarize past turns). Tradeoff: Lossy; summary might miss nuance.

// After 7 days, summarize conversation
const oldTurns = await memory.query({
  type: "conversation_turn",
  timestamp: { before: 7 days ago },
});

const summary = await llm.summarize(oldTurns);
await memory.store({
  type: "conversation_summary",
  period: "week_1",
  content: summary,
});

// Delete old turns
await memory.delete({ type: "conversation_turn", timestamp: { before: 7 days ago } });

Durability guarantees

Write-ahead logging (WAL)

Every write is logged before it's applied. Survives crashes.

How: Write to log, then to memory. On restart, replay log.

Agent: "Remember: user is in SF"
  → Write to WAL: {memory_id: 123, content: "user in SF"}
  → Confirm write
  → Apply to memory store
  → Delete from WAL

What Brain does: the self-hosted server (local; arena + WAL + redb) and managed Cloud both use group-commit WAL fsync.

Replication

Every write is replicated to N replicas before confirming.

How: Write to primary, replicate to 2+ secondaries, confirm when N replicas ack.

Agent: "Remember: user is in SF"
  → Write to primary
  → Replicate to secondary-1 (ack)
  → Replicate to secondary-2 (ack)
  → Confirm to agent
  → If primary fails, secondaries elect new primary

Brain Cloud: 3-way replication within region, async replication across regions.

Backups

Regular snapshots of memory state, kept for disaster recovery.

Frequency: Hourly for active agents, daily for inactive. Retention: 30 days minimum (longer for compliance).

Brain Cloud: Automatic daily backups, configurable retention.

Retrieval patterns for long-term memory

Pattern 1: User profile (first-access warm-up)

On agent startup, load the user's profile facts (most important facts about them).

async function loadUserProfile(userId: string) {
  const profile = await longTermMemory.retrieve({
    query: "What is the user's profile?",
    filter: { user_id: userId, type: "fact", category: "profile" },
    topK: 20,
  });
  return profile; // Fast lookup, cached
}

Pattern 2: Decision history (context for reasoning)

Before deciding, retrieve past decisions on similar topics.

async function getDecisionHistory(topic: string, userId: string) {
  const history = await midTermMemory.retrieve({
    query: `What did we decide about ${topic}?`,
    filter: { user_id: userId, type: "decision" },
    topK: 5,
  });
  return history;
}

Pattern 3: Temporal context (recent events first)

Get recent events, then fall back to older facts.

async function getFullContext(userId: string, query: string) {
  // Recent (last 3 days)
  const recent = await midTermMemory.retrieve({
    query,
    filter: { user_id: userId, timestamp: { after: 3 days ago } },
    topK: 10,
  });

  // Historical (if recent is sparse)
  if (recent.length < 5) {
    const historical = await longTermMemory.retrieve({
      query,
      filter: { user_id: userId },
      topK: 10 - recent.length,
    });
    return [...recent, ...historical];
  }

  return recent;
}

Pattern 4: Graph traversal (relationships)

For entity-based memory, traverse relationships.

async function getCompanyContext(companyName: string) {
  // Find company entity
  const company = await longTermMemory.retrieve({
    query: companyName,
    filter: { entity_type: "company" },
    topK: 1,
  });

  if (!company) return null;

  // Find employees (traverse relationship)
  const employees = await longTermMemory.retrieve({
    query: "employees of " + companyName,
    filter: {
      relation_type: "works_at",
      object_id: company.entity_id,
    },
  });

  return { company, employees };
}

Cost optimization

Long-term memory can get expensive. Optimize:

1. Compression

Summarize old memories. Trade precision for cost.

// Daily: compress events from past 7 days into 1 summary
const oldEvents = await memory.query({
  type: "event",
  timestamp: { before: "7 days ago" },
});

const compressed = await llm.compress(oldEvents);
await memory.store(compressed); // 1 memory instead of 100

2. Sampling

Keep every Nth memory, not all.

// Keep 1 in 10 conversation turns
const sampling_rate = 0.1;
if (Math.random() < sampling_rate) {
  await longTermMemory.store(turn);
}

3. Selective indexing

Don't index low-value fields.

// Index only high-cardinality, frequently-queried fields
{
  index: ["user_id", "type", "timestamp"],
  store_only: ["content", "metadata"],
}

4. Tiering by value

Cold data goes to cheaper storage.

// Recent (7 days): fast, expensive
// Archive (> 7 days): slow, cheap
if (timestamp.getTime() < Date.now() - 7 * 24 * 60 * 60 * 1000) {
  await archiveStorage.store(memory); // S3, cheaper
} else {
  await hotStorage.store(memory); // SSD, faster
}

Checklist: Before shipping long-term memory

  • You have a multi-tier strategy (session/mid-term/long-term)
  • Durability is guaranteed (replication or backup)
  • Eviction policy is defined and tested
  • You can reconstruct recent context from logs if needed
  • Cost is predictable and monitored
  • Retrieval latency is acceptable (< 200ms P99 for long-term)
  • You have a backup restoration plan
  • Data is encrypted at rest and in transit

Further reading

Related reading

Updates from the lab.

Engineering notes, research drops, occasional product updates. Roughly monthly.