SelfImprint: Production-Oriented Causal Memory for Conversational AI Agents
Technical Whitepaper v0.2 — Design Document
APERTURESyndicate OÜ, Tallinn, Estonia May 2026
Status. This is a design document. It describes the architecture of SelfImprint, a memory system intended for personalized AI assistants. Empirical evaluation against established long-term memory benchmarks is planned as separate work (Phase 2). Numerical parameters throughout are starting points drawn from the cognitive science literature; production deployments will need to recalibrate them against real user data.
Abstract
Most memory systems built for LLM agents today fall into two camps. Flat-fact stores (Mem0, MemGPT/Letta) retrieve information by semantic similarity but treat each fact as an isolated unit. Knowledge-graph approaches (Zep, GraphRAG) capture some structure, but the graphs are built passively from input and don't reason about themselves. Both camps share three blind spots: they don't model causality between events, they don't think about the user between sessions, and they don't forget intelligently.
SelfImprint takes a different approach. The graph uses five typed causal edges, not generic semantic links. Node weights decay exponentially with type-specific rates, so a transient frustration fades in days while a major life event stays for months. Between sessions, a dedicated reasoning process — the sleep phase — pattern-matches across the graph, infers new connections, forms hypotheses, and recalibrates importance. Hypotheses are first-class objects with explicit feedback loops: they get confirmed, refuted, or archived as unresolved within a bounded lifetime. A separate anonymized layer — CollectiveImprint — captures the model's observations about its own failures, gated by adaptive k-anonymity.
The architecture is shaped by deployment realities: hot/warm/cold storage tiers, adaptive sleep scheduling tied to subscription class, smart-skip optimization, and an explicit inference policy. None of these are exciting in isolation. Together they make the system runnable on a real budget for a real user base — which is where most academic memory designs fall apart.
1. Introduction
1.1. The problem
Base LLM assistants have no persistent memory. Every conversation starts blank. The user re-establishes context, the model performs, the session ends, and the next time around it has forgotten everything.
The industry has been patching this since 2024. OpenAI added Memory to ChatGPT. Anthropic shipped Projects. Google added Recall to Gemini. The opensource community produced Mem0, MemGPT/Letta, Zep, LangMem, MemoryBank, A-MEM, Cognee, and more. Progress is real. But three problems remain stubborn, and they all show up most clearly on long timelines.
Causality is missing. Existing systems can tell you what the user said. They struggle to tell you why something matters. "Why is the user in a bad mood right now?" requires walking a chain — event caused feeling, feeling shaped subsequent behavior, behavior shows up in the current message. Flat retrieval pulls the last node and misses the chain. Untyped graphs pull a tangle of associations and can't tell which links carry causal weight.
Between sessions, nothing happens. Memory in current systems is passive. Data goes in, gets indexed, comes out on request. Between user sessions, the system doesn't think. It doesn't ask why the user went quiet after a topic three weeks ago. It doesn't notice that two seemingly unrelated mentions actually share a thread. It doesn't form working hypotheses to test in the next interaction. Reflection-style work exists in agent literature (Reflexion, Generative Agents), but it hasn't been integrated into production memory layers.
Forgetting is either absent or naive. Most systems either keep everything forever or apply a uniform decay. Human memory isn't like that. A traumatic moment fades fast for some, lingers for others; trivia decays steadily; a meaningful event from a decade ago can still feel close. Current AI memory systems don't model this. They either remember everything (and drown in noise) or forget on a fixed schedule (and lose nuance).
1.2. What SelfImprint does
SelfImprint addresses these three gaps directly. The headline features:
A causal knowledge graph with five typed edges: caused_by, followed_by, reinforces, contradicts, associated_with. Typed edges aren't decoration — they let the sleep phase reason over graph structure rather than wading through unstructured text.
A sleep-phase reasoning process that runs during user inactivity. Six operations: detect patterns, infer new connections, form hypotheses, recalibrate importance, update scheduled triggers, prepare open-loop questions. This is the system's analytical layer, separate from response generation.
Type-specific decay. Negative emotional notes fade in days. Significant events persist for months. Pinned facts never expire. The decay coefficients are starting points, not laws — they need empirical calibration — but the principle that different memory types decay at different rates is well-established in cognitive science.
Hypotheses as first-class nodes with a feedback loop. The sleep phase can speculate: "the user may be experiencing burnout." That hypothesis carries an is_inference flag, lives with bounded confidence, and either gets confirmed by subsequent dialogue (converted to observed fact), refuted (deleted), or times out after fourteen days (archived as unresolved).
CollectiveImprint, an anonymized operational layer. Patterns the model notices about its own failures get aggregated across users — but only when at least k = max(10, 0.1% of active users) distinct users exhibit the same pattern. This is operational learning without compromising individual privacy.
Production realities baked in. Hot/warm/cold storage tiers because not every user is active every day. Adaptive sleep scheduling tied to subscription class and activity level. Smart-skip optimization for trivial sessions where there's nothing meaningful to reflect on. An explicit inference policy that names what the system will never infer about a user, regardless of what the data permits.
1.3. What's actually new
None of the individual ideas are novel in isolation. Knowledge graphs go back to symbolic AI in the 1970s. Ebbinghaus described the forgetting curve in 1885. Memory networks have been studied in neural systems since 2015. Reflexion and Generative Agents already explored offline reflection loops.
What's new is the integration — and specifically, the integration for deployment. Academic memory work tends to optimize a single dimension (retrieval quality, say) under controlled conditions. SelfImprint optimizes for what an AI memory layer actually needs to do in production: stay within budget, scale to many users, respect privacy from day one, and not slowly drift into a fantasy version of the user over six months. The contribution is engineering, not algorithm.
2. Related Work
The memory-systems-for-LLM-agents space has grown crowded since 2023. Here's how SelfImprint relates to the main lines of work.
MemGPT / Letta (Packer et al., 2023) framed memory as an operating-systems problem. Three tiers — main context (working set), recall storage (recent), archival storage (deep history) — with explicit movement between them. The OS analogy is useful and the engineering is solid. What MemGPT doesn't do is reason about relationships between facts or think between sessions. Its memory is a well-organized filing cabinet, not a graph.
Mem0 (Chhikara et al., 2024) goes the simpler route: an LLM extracts facts from dialogue, stores them as structured records, retrieves them by similarity. The implementation is clean and the API is pleasant. As architecture, it's organized RAG. There's no causal structure and no offline analysis.
Zep and GraphRAG (Edge et al., Microsoft 2024) are the closest neighbors. GraphRAG builds knowledge graphs from text and uses graph traversal during retrieval. The structural approach is right, and it does outperform flat retrieval on questions that benefit from composing information across documents. But the graph is built passively. There's no temporal decay tuned to fact type, no emotional weighting, no offline reflection process, no hypotheses as named entities. The graph is a snapshot, not an evolving model.
MemoryBank (Zhong et al., 2024) introduced forgetting curves to LLM memory — a real contribution. The limitation is uniformity: one decay rate for all memory, no graph, no emotional dimension.
A-MEM explored autonomous memory management, where the LLM updates its own memory store. This is conceptually adjacent to sleep-phase reasoning but operates without structural typing or hypothesis lifecycle.
Cognee sits more on the infrastructure side: an opensource framework combining vector, graph, and relational backing stores. Good plumbing, but doesn't propose a distinct reasoning model.
Reflexion (Shinn et al., 2023) and Generative Agents (Park et al., 2023) come from the agent-reasoning side rather than the memory-architecture side. Park's daily-reflection loop for NPCs in a simulated town is, to our knowledge, the closest published predecessor to the sleep phase. SelfImprint extends that direction with production constraints, integration with a typed causal graph, and explicit hypothesis lifecycle management.
2.1. Where SelfImprint fits
| Property | MemGPT/Letta | Mem0 | Zep/GraphRAG | MemoryBank | A-MEM | SelfImprint |
|---|---|---|---|---|---|---|
| Storage shape | Tiered | Flat list | Knowledge graph | List with decay | Memory bank | Causal graph |
| Typed causal edges | No | No | Partial | No | No | Yes (5 types) |
| Active reasoning between sessions | No | No | No | No | Partial | Yes (sleep phase) |
| Type-specific decay | No | No | No | Uniform | No | Yes |
| Emotional weighting | No | No | No | Partial | No | Yes |
| Hypotheses as entities | No | No | No | No | No | Yes |
| Hypothesis feedback loop | No | No | No | No | No | Yes |
| CollectiveImprint layer | No | No | No | No | No | Yes |
| Adaptive scheduling | No | No | No | No | No | Yes |
| Storage tier hierarchy | Conceptual | No | No | No | No | Yes |
| Explicit inference policy | No | No | No | No | No | Yes |
The claim isn't that any single row is unique in the academic sense. The claim is that the column adds up to a different kind of system — one that's designed to actually run, not just to publish.
3. Architecture
3.1. Data flow
When a user sends a message:
- Embed the message via bge-m3 (1024-dim, multilingual).
- Run multi-criteria retrieval against the user's graph. In parallel, look up relevant operational hints in CollectiveImprint.
- Expand the retrieved nodes one hop along their edges to capture context.
- Compress the resulting subgraph to a 400-token summary using ASAI-mini Memory 9B.
- Pass the summary to the main model as context for response generation.
When the user stops responding and enough time has passed, the system enters sleep phase: it pre-checks whether there's anything worth reasoning about, and if so, runs the six analytical operations described in §3.5. Once a week, a periodic compaction pass cleans up the graph. Patterns of model failure observed across many users contribute to CollectiveImprint, subject to the k-anonymity threshold.
3.2. The graph
Node types. Six classes:
fact— a stable claim about the user (name, job, location, preferences).event— something that happened at a specific time (a mentioned interview, a move, a conversation with someone).emotional_note— an emotional coloring of an interaction, split into positive and negative.person— someone in the user's circle.open_loop— a topic the user raised but didn't resolve.hypothesis— a node produced by the sleep phase, carryingis_inference: true.
All nodes are atomic. Complex entities — "the user wants a job at Google" — live as a single node with attributes, not as a subgraph. The simplification matters: keeping nodes atomic makes traversal cheap and the schema legible.
Edge types. Five typed relations:
caused_by— A happened because of B. Direct causality.followed_by— A came after B without claimed causation.reinforces— A strengthens the importance of B.contradicts— A conflicts with B.associated_with— generic semantic association, used when nothing more specific fits.
Typing the edges is a key design choice. The sleep phase wants to ask questions like "find all caused_by chains longer than three steps" — those are tractable on a typed graph and intractable on a graph of free-text relations.
Node attributes: id, type, content (text), embedding (1024-dim), weight ∈ [0,1], created_at, last_reinforced_at, is_inference, is_pinned, decay_coefficient.
Edge attributes: from_node_id, to_node_id, type, confidence ∈ [0,1], is_inferred, created_at.
3.3. Decay
Forgetting is modeled as exponential decay with type-specific coefficients:
weight(t + 1 day) = weight(t) × decay_coefficient(type)
Daily, every non-pinned node loses some weight. Nodes that fall below 0.05 are removed from the active graph (and may be archived for possible recovery).
Starting coefficients:
| Node type | Coefficient / day | Time from 1.0 to removal |
|---|---|---|
| emotional_note (negative) | 0.5 | ~4–5 days |
| emotional_note (positive) | 0.85 | ~18 days |
| fact (regular) | 0.92 | ~36 days |
| event (regular) | 0.95 | ~58 days |
| event (significant) | 0.98 | ~148 days |
| person | 0.97 | ~98 days |
| open_loop | 0.99 | ~298 days |
| pinned (any type) | 1.00 | never |
The values come from a mix of intuition and the cognitive-science literature on differential forgetting rates (Tulving, 1972; Conway, 2005). They are not laws. Production deployments will need to calibrate them, and the calibration is likely to vary across user populations and use cases.
An event qualifies as significant if, at creation time, the extractor LLM assigns it an importance score above 0.8. The threshold is another hyperparameter, and the prompt that produces the score will need careful calibration in production.
When a node is mentioned again, its weight goes up additively rather than resetting to 1.0:
weight = min(weight + 0.15, 1.0)
This prevents accumulation games — the user (or an adversary) can't pump a node to maximum importance just by repeating a phrase. But genuinely recurring topics will rise.
3.4. Context assembly
The main model never sees the graph. It sees a 400-token summary. Getting from graph to summary is a four-step process.
First, embed the user's current message. Second, retrieve top-K (K=20) nodes using a weighted score: 0.5 × semantic similarity + 0.2 × recency + 0.3 × normalized weight. In parallel, query CollectiveImprint for any operational hints relevant to this kind of request (§3.7). Third, expand: for each retrieved node, include its direct neighbors regardless of edge type. This is what turns isolated facts into context. Without expansion you know the user mentioned an interview; with expansion you know the interview was followed by frustration which was caused by underpreparation. Fourth, compress: the expanded subgraph — typically 30 to 60 nodes — gets summarized to 400 tokens by ASAI-mini Memory 9B, with explicit instructions to prioritize high-weight nodes, preserve causal chains intact, and surface anything directly relevant to the current message.
The output is freeform prose in the user's language, not structured data. The main model receives it with a system instruction: use this to inform your response, but don't quote it. The user should feel remembered, not catalogued. This last detail matters more than it sounds. Feed a model raw structured memory — "user.name: X, user.job: Y" — and it will produce responses that read like database queries. Feed it prose with an injunction not to quote, and it integrates the context naturally.
3.5. Sleep phase
The sleep phase is the system's analytical engine. It's also the most expensive piece, which is why most of the architecture decisions around it are about cost.
Model. Sleep phase runs on ASAI-mini Memory 9B, not the main assistant model. The work is structured reasoning over a graph, not creative generation; a smaller model handles it well, and the cost savings matter when sleep phase runs hundreds of times per active user per month.
Triggers. Sleep phase requires two things to be true simultaneously: at least 30 minutes since the user's last message, and either the local time is 3:00 AM (deep daily pass) or more than 6 hours have passed since the last sleep phase (regular pass for active users). If the user starts typing during a sleep phase, the current iteration aborts or pauses. Sleep phase never interrupts an active session.
3.5.1. The six operations
In order:
Pattern detection scans the last seven days of nodes and activity. It looks for topics mentioned three or more times in a week, correlations between topics and time of day or day of week, and trends in emotional notes. The output is a structured list of patterns that the next operations consume.
Connection inference uses those patterns to add edges between previously disconnected nodes. If pattern detection finds that stress mentions and mentions of a specific colleague tend to co-occur, connection inference adds a caused_by edge with confidence reflecting the strength of the correlation. New edges carry is_inferred: true and start at moderate confidence (0.4–0.6).
Hypothesis formation generates new nodes of type hypothesis, also flagged is_inference: true. A hypothesis is a claim that isn't directly observed but follows from the accumulated structure — "the user may be experiencing burnout," for example. These nodes enter the graph but get downweighted in context assembly.
Importance recalibration updates the weights of nodes whose neighborhoods changed during the previous operations. Nodes that just became central to a pattern get a boost. Nodes that have drifted out of relevance get a temporary decay multiplier.
Scheduled triggers update pulls dates and time-bound events out of recent dialogue and schedules context activations. If the user mentioned an interview on March 28, the system marks the relevant subgraph for high priority starting March 27.
Open-loop preparation drafts possible questions the assistant might ask in the next session to close out unresolved threads. These are candidates, not a checklist — the assistant decides in context whether to use them.
3.5.2. Hypothesis lifecycle
Hypotheses don't live forever. Each one carries a creation timestamp, a default timeout of 14 days, a list of supporting observations, and a list of contradicting observations.
At every sleep phase, active hypotheses get checked:
- If supporting evidence exceeds contradicting by 2× and total evidence passes a threshold, the hypothesis converts to an observed fact (
is_inference: false). - If contradicting exceeds supporting by 2×, the hypothesis is deleted.
- If the 14-day timeout expires without resolution, the hypothesis is archived as unresolved — kept for history, no longer used in context.
One more cap: a hypothesis can be reinforced at most three times without converting to an observed fact. After three reinforcements, it's flagged as stuck and archived. This prevents the system from getting trapped in self-reinforcing speculation loops that never quite tip over into confirmation.
3.6. Adaptive sleep scheduling
If sleep phase ran every night for every user, it would dominate the system's cost. It shouldn't, and it doesn't need to. Most users don't generate enough new material in a day to justify a full reasoning pass.
The default scheduling policy is tied to subscription and activity:
| User tier | Sleep frequency | Depth |
|---|---|---|
| Paid, very active (5+ messages/day) | Daily | Full |
| Paid, regular (1–5/day) | Every 2 days | Full |
| Paid, light (1–7/week) | Weekly | Light |
| Paid, dormant (1–4/month) | Monthly | Light |
| Free, active | Every 3–7 days | Full |
| Free, regular | Weekly | Light |
| Free, light | Biweekly | Light |
| Any, inactive 30+ days | Off | — |
Light passes run only pattern detection and importance recalibration. The thresholds are defaults, not gospel; deployments calibrate them against observed usage distributions.
Smart skip. Before launching a full sleep phase, ASAI-mini Memory 9B does a cheap pre-check — one LLM call to assess whether there's anything worth reasoning about. The pre-check asks: how many new messages since last sleep, do they carry emotional signal, do they mention significant events or open loops, is there anything that looks like contradiction with existing nodes? If the answer is "nothing meaningful," the system writes a one-sentence summary and skips the full pipeline. This saves a lot of compute on users whose recent sessions were a couple of technical questions and goodbye.
Together, adaptive scheduling and smart skip reduce sleep-phase cost by something like 3× to 5× compared to a naive every-night-for-everyone policy.
3.7. CollectiveImprint
The architecture has two memory layers, not one. Per-user is SelfImprint. There's also a shared layer — CollectiveImprint — that holds the model's observations about itself.
CollectiveImprint does not contain user content. What it contains is operational knowledge: "requests of pattern X frequently produce unsatisfying responses," "approach A to task B works; approach C usually doesn't," "users in this context often want clarification before substance." The intent is to give the system a way to learn from its own mistakes without retraining weights.
Source. After each sleep phase, problematic episodes get flagged — moments where the user expressed dissatisfaction, repeated a question with clarification, or corrected the assistant's prior response. These episodes are stripped of identifying content, reduced to pattern features, and submitted to CollectiveImprint as candidate observations.
Privacy gate. A candidate observation only graduates to an active CollectiveImprint node after k = max(10, 0.1% of active users) distinct users have produced the same pattern. The formula scales with deployment size: a 10K-user instance uses k=10, a 100K-user instance uses k=100, and so on. This is adaptive k-anonymity, following the established literature on protecting individual identifiability through aggregation (Sweeney, 2002), adapted to a population that grows.
Use. When the system assembles context for a new request, it queries CollectiveImprint in parallel with the user's graph. If the current request resembles a class of requests for which CollectiveImprint has a known pattern, the relevant observation gets mixed into the context as an operational hint. This is a form of continual learning that doesn't touch model weights — useful especially in production environments where retraining cycles are infrequent.
Future use. Patterns accumulated in CollectiveImprint are valuable training signal. They form a natural curriculum for fine-tuning subsequent model versions, a target list for evaluation improvements, and a regression-test set for ensuring known weak spots stay fixed.
3.8. Storage
3.8.1. Hot, warm, cold
Most users aren't active most of the time. In typical chat deployments, 10 to 15 percent of a registered user base shows up in any given week. Keeping every user's graph hot in the primary database is wasteful.
SelfImprint uses three tiers:
- Hot: active users (messaged in the last 7 days). Graph lives in Neo4j, instant access.
- Warm: dormant users (8–30 days inactive). Graph serialized to a SYNX file on SSD. When the user returns, the graph reloads into Neo4j in 1–3 seconds.
- Cold: archive users (30+ days). SYNX file on cheap storage (HDD or cold object storage). Reload takes 5–10 seconds — acceptable for someone returning after a month away.
The economics shift noticeably. A deployment with 100K registered users typically has 10–15K active at any time. The tiered approach lets the hot infrastructure scale to active load rather than registered load.
Pause on inactivity. If a user has been inactive for more than 7 days, their decay clock pauses. This is a rough approximation of how human memory actually works — what you don't think about doesn't decay smoothly, it just sits. It also protects returning users from finding the system has forgotten everything except their pinned facts.
3.8.2. Technology choices
The graph lives in Neo4j v5+ with native vector search. An earlier version of this design used Neo4j plus Qdrant as an external vector index; the current design drops Qdrant because Neo4j's built-in vector search covers the use case and removes a synchronization headache.
Between sessions, graph state serializes to SYNX, APERTURESyndicate's data format. The four-file structure: core.synx for pinned facts and base profile, manifest.synx for session indexes and metadata, domain/*.synx for topical subgraphs (work, family, hobbies), and sessions/*.synx for detailed traces of specific conversations.
Embeddings are bge-m3, 1024 dimensions, multilingual. The multilingual property matters more than it sounds — users switch languages, and we want embeddings to remain comparable across the switch.
3.9. Graph compaction
Knowledge graphs degrade. Over months of active use, a personal graph accumulates thousands of edges, low-confidence inferences, conflicting nodes, and dead-but-not-removed material. Without explicit compaction, the system slides into what we'd informally call edge soup — technically functional, semantically incoherent.
SelfImprint runs a compaction pass weekly as part of the deep sleep phase:
Edge confidence decay. Every edge has a confidence value. Inferred edges start at 0.4–0.6; observed edges at 0.9–1.0. Confidence also decays over time when an edge isn't reinforced by new observations. Edges below 0.1 are removed.
Node deduplication. Nodes with cosine similarity above 0.92 in embedding space, sharing the same type, get merged. The merged node keeps the highest weight, combines attributes, and inherits the union of edges.
Isolated node removal. Nodes left without edges (after deduplication) are removed unless pinned.
Edge budget per node. Each node is capped at 20 edges. When the cap is exceeded, the lowest-confidence edges are dropped. This is a topological regularizer against over-connection.
Entropy monitoring. A simple measure of graph chaos — average shortest path, clustering coefficient, ratio of inferred to observed edges — gets tracked over time. If entropy exceeds a threshold, compaction gets aggressive.
Hard reset, as last resort. If a graph has degraded past the point where compaction helps (entropy plus user feedback like "you don't seem to understand me anymore"), the system can soft-reset: keep core pinned facts and the last 30 days, archive the rest. This is an unpleasant operation, but every long-running memory system needs an escape hatch.
4. Inference Policy and Privacy
A memory system that builds latent inferences about its user is powerful. It's also dangerous. Without an explicit policy on what the system will and won't infer, it can drift into something close to a psychological profiling engine — by accident, even with good intent.
4.1. Categories the system never infers
The sleep phase does not generate hypotheses in the following categories, regardless of how the data points:
- Psychiatric or medical diagnoses.
- Sexual orientation or gender identity (unless the user has stated it directly).
- Political views.
- Religious beliefs.
- Specific financial position.
- Health status (unless the deployment is explicitly medical).
- Immigration status.
- Criminal history or tendencies.
These aren't suggestions. They're categorical exclusions. The reasoning is twofold. First, false positives in any of these categories carry disproportionate cost — for the user's wellbeing and for the deployer's legal exposure. Second, EU AI Act and GDPR compliance gets considerably easier when the system can demonstrate, at the architectural level, that certain inferences are simply not made.
4.2. Categories that require a high bar
Some inferences are allowed but require significantly more evidence before they enter the graph as observed fact, and they live with longer timeouts as hypotheses:
- Emotional states (stress, sadness, frustration) are tracked for the purpose of adapting communication style, not for diagnosis. The assistant might soften with a frustrated user; it should not assert that the user is depressed.
- Relationships with specific people are tracked as context, not as evaluations. The graph might note that the user often mentions a colleague in negative terms; it should not conclude that the relationship is toxic.
- Long-term goals and ambitions are tracked for long-horizon helpfulness.
4.3. Privacy architecture
All SYNX files are encrypted at rest with AES-256. Per-user Neo4j databases are isolated to prevent cross-tenant leaks.
Right to deletion is built in. A user can request a complete wipe of their graph; the corresponding database and all SYNX files are destroyed, with no recovery path.
A more granular forget operation — "forget about topic X" — removes all nodes that reference the topic and propagates the deletion through their edges.
Once a month, the user gets a summary of what the system remembers, with the option to correct or remove specific entries. This is both transparency and a feedback loop against graph drift.
4.4. APERTURESyndicate data sovereignty
In the context of an APERTURESyndicate deployment, the policy is explicit rather than implied. User data is not transferred to third parties. Specifically:
- User data is not used to train models without explicit opt-in.
- User data is not sold or shared with partners without explicit consent.
- Legal requests from government authorities are evaluated case by case. The request must be legally substantiated; a designated specialist reviews it; only the minimum data demonstrably necessary, addressing a documented real threat, is disclosed.
This is positioning as much as policy. In 2026, many AI providers occupy an ambiguous middle ground on government data requests. APERTURESyndicate takes the opposite position: user data belongs to the user, and the bar for disclosure is high.
5. Personality Drift
When a memory system runs for months against a single user, there's a slow failure mode that doesn't show up in any standard benchmark: the system's model of the user gradually drifts away from the actual user. New hypotheses build on old hypotheses. Confirmation bias creeps in. Six months in, the system can be confidently wrong about someone in ways that feel intimate.
Four mitigations:
Monthly summaries. Once a month, the user sees what the system remembers — hypotheses, significant nodes, accumulated assumptions. They can correct, delete, flag as wrong. This is transparency and it's also the strongest single defense against drift.
Reality anchor. During every sleep phase, hypotheses get checked against the last 30 days of dialogue. A hypothesis that isn't surfacing in recent interactions loses confidence faster than time-decay alone would predict. This catches stale assumptions before they entrench.
Hypothesis lifecycle limits. A hypothesis that has been reinforced three times without ever quite converting to an observed fact gets flagged as stuck and archived. This prevents a particular failure mode where a near-hypothesis loops forever, growing more confident without ever crossing a threshold.
Explicit user control. The user can correct any node, anytime, through a documented and discoverable API. This isn't buried in settings. The premise is that the user is the final authority on their own model.
6. Failure Modes
A few things will go wrong. Here's how we expect to handle them.
Contradictory nodes. As the graph grows, contradictions accumulate. The user works at TechCorp, then works at DataInc; says they love their job, then dreads going in. Each new contradiction creates a contradicts edge. The sleep phase, during importance recalibration, downweights the older node. After a few decay cycles, the older node has faded.
False hypotheses. The sleep phase can generate plausible-sounding hypotheses that are wrong. The defenses are layered: the is_inference flag keeps them visible as hypotheses in context assembly; the evidence threshold for conversion to fact is high; the inference policy excludes the most dangerous categories outright; and the lifecycle limits stop near-misses from accumulating.
Long inactivity. If a user disappears for two months and comes back, naive decay would have erased almost everything except pinned facts. The pause-on-inactivity rule handles this — once a user has been inactive for a week, their decay pauses. It's a rough approximation of how human memory works.
Multilingual conversations. Users switch between languages, sometimes mid-conversation. bge-m3 keeps embeddings comparable across languages; sleep-phase reasoning runs in English regardless of user language (ASAI-mini Memory 9B is more stable in English); context assembly returns prose in the language of the most recent user message.
Adversarial users. Someone might try to manipulate their own memory — repeating false facts to pin them, for instance. The defense is a rate limit on reinforcement: a node that receives more than 10 reinforcements in a single day gets flagged, and further reinforcements that day are ignored.
Long-term edge soup. Already discussed: weekly compaction handles it.
Scalability. The hot/warm/cold tier handles the bulk of the scaling problem, but at very large user counts (above 100K), per-user database isolation in Neo4j becomes operationally awkward. Sharding by user cluster is the natural next step. This is engineering work we've planned but not implemented.
7. Evaluation Plan
Empirical evaluation is Phase 2 of this work. The plan:
Benchmarks. LongMemEval (Wu et al., 2024) is the primary target. It's structured around five categories — single-session, multi-session, temporal reasoning, knowledge update, abstention — which align well with the architectural claims SelfImprint makes. LoCoMo (Maharana et al., 2024) is the secondary target, providing multi-month synthetic dialogue traces.
Baselines. Three: flat RAG over conversation history, Mem0 (LLM extraction with structured storage), and Zep/GraphRAG (existing knowledge-graph approach).
Controlled variables. Same base model, same embedding model, same retrieval budget, same hardware across all systems. The only variable is the memory architecture.
Metrics. Accuracy on final-session questions (exact-match where applicable, LLM-as-judge for open-ended). Per-category breakdown to identify where SelfImprint wins and loses. Cost efficiency: LLM calls per question, total compute time, peak memory. Robustness: variance across re-runs, sensitivity to hyperparameter changes.
Ablations. Each major component needs to be ablated to understand its contribution:
- Causal edges vs untyped edges.
- Sleep phase vs no sleep phase.
- CollectiveImprint vs per-user only.
- Type-specific decay vs uniform decay.
- Smart skip vs always-run.
Sample size. At least 50 questions per benchmark category for usable confidence intervals.
Budget. Realistic estimate: several H100-days of GPU compute, plus the engineering effort for baseline integrations and ablation infrastructure.
8. Open Problems and Future Work
Beyond what's already in the Failure Modes section, several directions remain open.
Hyperparameter calibration. Every number in this document — decay coefficients, importance thresholds, reinforcement increment, the k in CollectiveImprint — is a starting point. Production deployment will need empirical recalibration, possibly per-user-population.
Cross-user pattern reasoning. Sleep phase currently operates per-user. There's a separate question about whether CollectiveImprint, with its anonymization guarantees, could surface population-level patterns useful for improving the architecture itself. Done carefully, this is a research direction. Done carelessly, it's a privacy problem.
Personality adaptation. SelfImprint feeds context to the main model but doesn't change the model's style. Deep knowledge of a user might warrant style adaptation — softer for emotional users, more direct for busy ones. This is unexplored.
On-device deployment. What if a user wants their graph on their device, not in the cloud? On-device sleep phase compute is the bottleneck, but a hybrid model — local graph, sleep phase computed in the cloud on encrypted data — is plausible.
Multi-modal memory. Current SelfImprint is text-only. Extending to images, audio, and video requires rethinking both embeddings and graph structure.
Federated CollectiveImprint. Multiple SelfImprint deployments — different companies, different domains — could in principle pool their CollectiveImprint observations while preserving strict privacy guarantees. This would require federated learning infrastructure that doesn't currently exist.
9. Conclusion
SelfImprint is an attempt to build a long-term memory system for AI assistants with deployment realities at the center of the design rather than at the periphery. The architecture doesn't rest on a single algorithmic innovation. It integrates well-understood ideas — knowledge graphs, forgetting curves, reflection loops, k-anonymity — into a form that can run in production: typed causal edges, type-specific forgetting, hypotheses with explicit lifecycles, an operational layer for the model's self-observations, tiered storage, an inference policy with teeth.
We don't claim any component is new on its own. We do claim that the combination, designed for actual deployment, is a useful contribution to the space. Empirical validation on established benchmarks is the next step.
The longer arc we see: at some point in the next few years, the quality of an AI assistant will stop being determined primarily by the strength of its base model and start being determined by the quality of its memory of the user. SelfImprint is one possible shape for that memory layer. We're publishing it because we think parts of it will be useful to others working on the same problem.
References
Anderson, J. R. (2007). How Can the Human Mind Occur in the Physical Universe? Oxford University Press.
Chhikara, P., Khant, D., Aryan, S., Singh, T., & Yadav, D. (2024). Mem0: Scalable Long-Term Memory for Production AI Agents. arXiv:2504.19413.
Conway, M. A. (2005). Memory and the Self. Journal of Memory and Language, 53(4).
Dwork, C., McSherry, F., Nissim, K., & Smith, A. (2006). Calibrating Noise to Sensitivity in Private Data Analysis. Theory of Cryptography Conference.
Ebbinghaus, H. (1885). Über das Gedächtnis. (Translation: Memory: A Contribution to Experimental Psychology.)
Edge, D., Trinh, H., Cheng, N., et al. (2024). From Local to Global: A Graph RAG Approach to Query-Focused Summarization. Microsoft Research.
Maharana, A., Lee, D.-H., Tulyakov, S., et al. (2024). Evaluating Very Long-Term Conversational Memory of LLM Agents. ACL.
Packer, C., Wooders, S., Lin, K., Fang, V., et al. (2023). MemGPT: Towards LLMs as Operating Systems. arXiv:2310.08560.
Park, J. S., O'Brien, J. C., Cai, C. J., et al. (2023). Generative Agents: Interactive Simulacra of Human Behavior. UIST.
Shinn, N., Cassano, F., Berman, E., et al. (2023). Reflexion: Language Agents with Verbal Reinforcement Learning. NeurIPS.
Sweeney, L. (2002). k-Anonymity: A Model for Protecting Privacy. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems.
Tulving, E. (1972). Episodic and Semantic Memory. Organization of Memory.
Wu, D., Wang, H., Yu, W., et al. (2024). LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory. arXiv:2410.10813.
Zhong, W., Guo, L., Gao, Q., Ye, H., & Wang, Y. (2024). MemoryBank: Enhancing Large Language Models with Long-Term Memory. AAAI.
APERTURESyndicate OÜ, Tallinn, Estonia
This document is v0.2, May 2026. It is a design document; empirical results will appear in companion work.