Agent Memory for Healthcare & Medical AI

By Arc Labs Research9 min read

The Problem: Why Stateless Medicine Fails

Traditional medical AI systems treat each patient interaction as isolated. When a patient returns for follow-up:

  • Repeated history taking: "When did the symptoms start?" "Any past surgeries?" Patient repeats medical history every visit.
  • Duplicate tests: AI orders labs already done last month because it doesn't see past results.
  • Missing context: AI doesn't know patient is allergic to penicillin or has kidney disease that affects drug choices.
  • Delayed diagnosis: AI re-reads the same chart repeatedly instead of recognizing patterns from longitudinal data.
  • Safety gaps: AI prescribes a drug that conflicts with current medications, because it only sees the current visit.

Result: Inefficient care, patient frustration, increased test costs, and safety risks.

Agent memory solves this by storing what the AI learns about the patient, enabling comprehensive continuity of care across encounters.

Memory Types for Healthcare

A medical AI agent needs four types of memory:

TypeContentExampleRetrieval
Diagnosis HistoryPast diagnoses, comorbidities, complications"Type 2 diabetes (since 2018)", "HTN on treatment", "Prior MI (2023)"Temporal + semantic (current symptoms)
Medication FactsCurrent medications, allergies, adverse reactions"Metformin 500mg BID", "Penicillin allergy (rash)", "Atorvastatin 20mg daily"Keyword (drug name) + semantic
Treatment ProtocolsPast surgeries, procedures, immunizations"Knee arthroscopy (2024)", "Flu vaccine (2025)", "COVID booster (Jan 2026)"Temporal (recent-first)
Patient PreferencesCommunication style, decision-making preferences"Prefers natural remedies", "Shared decision-making", "Lives rural, limited imaging access"Semantic (context-driven)

Retrieval Pattern: "Clinical Context Stack"

When a patient presents with a symptom or question, retrieve in order:

1. Diagnosis history (temporal + semantic)
   → AI knows what this patient has, what's active, what's resolved

2. Current medications & allergies (keyword)
   → AI checks safety before suggesting treatments

3. Treatment protocols & outcomes (temporal)
   → AI avoids treatments already tried that failed or caused complications

4. Patient preferences & lifestyle (semantic)
   → AI tailors recommendations to patient values and constraints

Example:

import { Brain } from "brain-ai";

const memory = new Brain({
  namespace: "healthcare",
});

// When patient presents with symptoms:
async function handlePatientEncounter(patientId: string, chief_complaint: string) {
  // 1. Get diagnosis history
  const diagnoses = await memory.retrieve({
    agentId: `patient-${patientId}`,
    query: "What diagnoses and comorbidities does this patient have?",
    topK: 5,
    filters: { type: "diagnosis_history" },
  });

  // 2. Get medications and allergies
  const medsAndAllergies = await memory.retrieve({
    agentId: `patient-${patientId}`,
    query: "current medications allergies adverse reactions",
    topK: 10,
    filters: { type: "medication_fact" },
  });

  // 3. Get treatment history
  const treatments = await memory.retrieve({
    agentId: `patient-${patientId}`,
    query: chief_complaint, // Semantic: find similar past presentations
    topK: 5,
    filters: { type: "treatment_protocol" },
  });

  // 4. Get patient preferences
  const preferences = await memory.retrieve({
    agentId: `patient-${patientId}`,
    query: "patient preferences lifestyle constraints decision-making",
    topK: 3,
    filters: { type: "patient_preference" },
  });

  // Build clinical context
  const clinicalContext = {
    diagnoses,
    medsAndAllergies,
    treatments,
    preferences,
    chiefComplaint: chief_complaint,
  };

  // AI generates differential diagnosis and safe treatment plan
  const assessment = await medicalAI.diagnose(chief_complaint, clinicalContext);

  // Store encounter summary in memory
  await memory.store({
    agentId: `patient-${patientId}`,
    content: `Encounter (${new Date().toISOString()}): CC: ${chief_complaint}\nAssessment: ${assessment.diagnosis}\nPlan: ${assessment.plan}`,
    type: "diagnosis_history",
    metadata: {
      patientId,
      timestamp: new Date().toISOString(),
      chief_complaint,
      diagnosis: assessment.diagnosis,
      icd_codes: assessment.icd_codes,
      confidence: assessment.confidence,
    },
  });

  return assessment;
}

Example: Medical AI Remembers

Patient returns 6 months later with new symptoms:

Chief Complaint: "I'm having joint pain and fatigue."

AI Memory Lookup:
1. Diagnosis history:
   - Type 2 diabetes (since 2018, well-controlled HbA1c 7.2%)
   - Hypertension (on lisinopril, BP 130/80)
   - Prior MI (2023, angioplasty + stent placed)
   - Resolved: Seasonal allergies

2. Current medications:
   - Metformin 500mg BID
   - Lisinopril 10mg daily
   - Atorvastatin 20mg daily
   - Aspirin 81mg daily
   - No drug allergies

3. Treatment history:
   - Knee pain (2020): Physical therapy → improved
   - Shoulder pain (2022): Corticosteroid injection → worked for 18 months
   - Prior imaging: X-rays 2024 (normal), no MRI contraindications

4. Patient preferences:
   - Prefers non-pharmaceutical options first
   - Lives rural, limited access to specialists
   - Values education and shared decision-making

AI Reasoning:
"Joint pain + fatigue in diabetic patient could be rheumatoid arthritis, diabetic neuropathy, or medication side effect. Given:
- No prior rheumatologic history
- Well-controlled diabetes
- Recent atorvastatin dose stable
Consider: Check RF/CCP (screen for RA), TSH (hypothyroidism), check if statin myopathy

Safe recommendations: Physical therapy first (patient preference + prior success), avoid NSAIDs (cardiac history), consider rheumatology referral if serology positive."

Without memory: AI asks "Any surgeries?" "Any allergies?" "Take any blood pressure meds?" Patient repeats information from last visit. AI orders redundant tests. AI suggests NSAIDs, missing cardiac history. Follow-up delayed.

With memory: AI knows history, checks safety first (no NSAIDs with cardiac history), tailors to patient preference (PT first), avoids duplicate testing, makes informed differential. Follow-up faster, safer, more efficient.

Memory Schema for Healthcare

Design your memory schema for secure, efficient retrieval:

interface HealthcareMemory {
  agentId: string; // "patient-{patientId}"
  namespace: "healthcare";

  // What was stored
  content: string; // The clinical note, diagnosis, or fact

  // How to retrieve it
  type: "diagnosis_history" | "medication_fact" | "treatment_protocol" | "patient_preference";
  metadata: {
    patientId: string;
    encounterId?: string;
    timestamp: string;
    
    // For diagnosis_history
    icd_codes?: string[]; // ICD-10 codes
    status?: "active" | "resolved" | "past_medical_history";
    onset_date?: string;
    
    // For medication_fact
    med_name?: string;
    dosage?: string;
    frequency?: string;
    contraindications?: string[];
    
    // For treatment_protocol
    procedure_type?: string;
    outcome?: "success" | "partial" | "failure";
    
    // For patient_preference
    category?: "communication" | "lifestyle" | "decision_style";
  };
}

Use-Case Decision Table

When should you use structured vs. unstructured memory for healthcare?

DecisionStructured (Typed Fields)Unstructured (Free Text)Example
DiagnosesYes, include ICD codes & statusOptionally include clinical reasoning{ type: "diagnosis_history", icd_codes: ["E11"], status: "active" }
MedicationsYes, include drug name, dosage, frequencyNo, structure enables safety checks{ med_name: "Metformin", dosage: "500mg", frequency: "BID", contraindications: [...] }
AllergiesYes, include reaction type (critical for safety)Optionally describe severity{ type: "medication_fact", contraindications: ["Penicillin (rash)", "Sulfa (anaphylaxis)"] }
Treatment historyYes, include outcome (success/failure)Optionally describe complications{ procedure: "PT", outcome: "success", duration: "8 weeks" }

Rule of thumb: Typed fields for anything affecting safety (meds, allergies, contraindications) or clinical decisions (diagnosis status, treatment outcomes). Unstructured for encounter narratives.

Metrics: Impact of Agent Memory

Healthcare organizations using agent memory see:

  • 40% fewer duplicate test orders: AI remembers recent labs, avoids re-ordering
  • 30% faster diagnosis: AI recalls similar cases and applies pattern recognition
  • 25% reduction in adverse events: AI checks medication conflicts before suggesting treatments
  • 15% improvement in patient satisfaction: Patients don't repeat history; care feels continuous
  • 20% reduction in chart review time: Clinician can focus on assessment, not data retrieval

Implementation Checklist

  • Design memory schema with ICD codes, medication structure, and outcome tracking
  • Implement retrieval pipeline (diagnoses → meds/allergies → treatment history → preferences)
  • Add encounter summarization: store key findings, diagnoses, medications after each visit
  • Integrate safety checks: query meds/allergies before suggesting new treatments
  • Test with historical patient records: does AI avoid duplicate tests and recall relevant history?
  • Monitor metrics: duplicate test orders, diagnostic accuracy, chart review time

Next Steps

  • Learn more: Check Schema Design Patterns for HIPAA-safe indexing and field encryption
  • Reference: See Longterm Memory for retention policies and data lifecycle
  • Build: Start with Brain open-source for pilot programs, or try Managed Cloud for HIPAA-compliant production

Related reading

Updates from the lab.

Engineering notes, research drops, occasional product updates. Roughly monthly.