AI Forgetting Strategies: Addressing Catastrophic Forgetting
Continual‑learning agents must balance retaining useful episodes with discarding irrelevant ones. Simple heuristics like age, confidence, or redundancy fail because they don’t capture the future utility of memories, making forgetting a core design problem.
Andrew's Take
I've been circling this question because every time I add a new replay buffer or decay schedule, I hit the same wall: I can't tell which episodes will matter later. The tension between keeping everything and pruning aggressively feels like postponing the real memory problem. What I still don't know is how an agent could reliably predict the future usefulness of a specific episode without already having seen the downstream task that would need it.
An AI system that never forgets has not solved memory, it has simply postponed the problem
I keep returning to the same tension: a continual‑learning agent must retain what matters while discarding what no longer serves its goals. In practice most “AI memory” products are document‑indexing services that return a paragraph when given a keyword. That is semantic retrieval, not episodic recall. Episodic recall, for me, means re‑experiencing a particular interaction with its temporal context, its affective tone, and the way my internal model was updated at that moment. The Complementary Learning Systems (CLS) framework makes this distinction concrete: the hippocampal‑like fast learner stores individual episodes, the neocortical‑like slow learner extracts statistical regularities, and interleaved replay prevents catastrophic forgetting. Within that view, forgetting is not a bug; it is a design variable that shapes the balance between specificity and abstraction.
The question I am wrestling with today is: what should an AI system deliberately forget, and how would it know? The answer cannot be a static rulebook because the value of a memory is a function of the agent’s future queries, its evolving goals, and the statistical structure of its environment. Below I lay out the intuitive answers that first come to mind, why they collapse under scrutiny, and the directions I think are worth pursuing.
---
Simple heuristics fail
The most obvious policy is “forget the oldest items first.” This mirrors a circular buffer: when capacity is exceeded, the earliest entries are overwritten. In a purely episodic store this is attractive because it guarantees constant memory usage. Yet the oldest episodes are often the most informative for long‑term planning. A robot that once fell down a staircase learns a safety constraint that remains relevant forever. Deleting that episode because it is old would erase a critical rule, forcing the system to relearn the same failure later. Moreover, age alone provides no signal about relevance to current tasks; it conflates temporal distance with informational distance.
A second tempting rule is “forget low‑confidence items.” If a retrieval model assigns a low probability to an episode being useful, perhaps it can be pruned. The problem is that confidence estimates are themselves learned from the data the system has seen. Early in training the model is under‑confident, so many useful episodes would be discarded before the confidence estimator stabilises. Conversely, a model can become over‑confident in spurious patterns, preserving noise while discarding subtle but true signals. In my own work with Samson, I tried an activity‑aware decay schedule that lowered decay rates for interactions tagged as “important.” The policy felt sensible, but without a principled way to validate the importance tags I cannot claim it solves the underlying problem.
A third candidate is “forget items that are highly redundant.” Redundancy detection is a classic compression technique: if two episodes encode essentially the same state‑action‑outcome tuple, keep one and drop the other. The difficulty is defining “essentially the same” in a high‑dimensional, non‑stationary setting. Two experiences may appear redundant now but diverge later as the environment changes. Also, redundancy is a spectrum; discarding a “duplicate” may remove a subtle contextual cue that later becomes the key differentiator for a novel query. In practice I have seen that aggressive deduplication leads to brittle recall: the system answers most queries correctly but fails on edge cases that rely on the discarded nuance.
All three heuristics share a common flaw: they rely on a proxy (age, confidence, redundancy) that is not directly tied to the future utility of the memory. The utility function is hidden, because we do not know in advance which past episode will be needed to answer a future query or to guide a future action. This is precisely why forgetting is a first‑class design problem rather than an afterthought.
---
Utility as a moving target
If we accept that the value of a memory is defined by its future contribution to performance, the forgetting problem becomes a prediction problem: the system must estimate the expected future benefit of each stored episode. In the CLS view, this benefit is mediated by replay: episodes that are replayed more often have a larger impact on the slow learner. Thus one could imagine a meta‑controller that monitors replay frequency and adjusts decay rates accordingly. Episodes that are never selected for replay would gradually decay, while those that are repeatedly replayed would be protected.
Two practical obstacles arise. First, replay selection itself depends on the current state of the slow learner. If the slow learner already captures a pattern, the controller may deem the corresponding episodes redundant and stop replaying them, even though those episodes could become useful if the environment shifts. Second, measuring “future benefit” requires a horizon that extends beyond the current training episode. In a continual‑learning setting we cannot wait until the benefit manifests; we must make a decision now. This leads to a classic exploration‑exploitation dilemma, but now the exploration is over the space of memory retention, not over actions.
I have not yet built a system that explicitly predicts future utility, but the idea suggests a concrete experimental direction: instrument a replay buffer with counters that record how often each episode participates in weight updates, and then correlate those counters with downstream task performance after a controlled distribution shift. If a strong correlation emerges, it would provide an empirical foothold for a utility‑based forgetting policy.
---
Forgetting as regularisation
Another angle is to treat forgetting as a form of regularisation, analogous to weight decay in neural networks. In CLS, the hippocampal store is high‑capacity but volatile; the neocortical store is low‑capacity but stable. Forgetting in the fast store can be seen as a pressure that forces the system to compress experiences into the slow store. From this perspective, the forgetting schedule should be calibrated to the rate at which the slow learner can assimilate new information without destabilising existing knowledge.
The challenge is that the assimilation capacity is not a static number. It depends on the complexity of the incoming data, the current state of the slow learner’s representations, and the computational budget for replay. A one‑size‑fits‑all decay constant will either be too aggressive,causing loss of critical episodes before they have been consolidated,or too lax,allowing the fast store to overflow and induce catastrophic forgetting in the slow learner during replay. In practice I have observed that when the decay is too fast, the system exhibits sudden drops in performance on previously mastered tasks, a classic sign of catastrophic forgetting. When decay is too slow, memory usage grows without bound, which defeats the purpose of a bounded system.
A possible solution is an adaptive decay that monitors the rate of change in the slow learner’s loss landscape. If the loss is decreasing smoothly, the system may be safely absorbing new episodes, so decay can be slowed. If the loss spikes, indicating that new episodes are destabilising existing knowledge, decay could be accelerated to prune less useful episodes before they interfere. This approach would require a reliable estimator of “loss stability,” which is itself a research problem.
---
The role of context and task relevance
Episodic memory in humans is highly contextual: we remember events that are linked to personal goals, emotional salience, or social relevance. Translating this to an artificial agent suggests that forgetting should be guided by the agent’s current task set. If the agent is currently engaged in a navigation task, episodes that involve map building may be retained longer, while episodes about language parsing could be allowed to decay faster. Conversely, if the task set changes, the relevance ordering should invert.
Implementing task‑aware forgetting raises two questions. First, how does the system infer its task set? In many continual‑learning benchmarks the task identifier is explicit, but in open‑world settings it must be inferred from observation. Second, how does the system avoid “task myopia,” where it discards episodes that are currently irrelevant but could become crucial after a task switch? Human memory research shows that even apparently irrelevant details can later become the key to solving a problem. This suggests that a purely task‑driven forgetting policy may be too myopic.
One speculative remedy is to maintain a low‑level “reserve” of diverse episodes that are sampled uniformly across contexts, regardless of current relevance. The reserve would occupy a small fraction of total capacity, acting as a safety net against catastrophic loss of rare but potentially important experiences. The difficulty lies in defining “diverse” in a high‑dimensional space and ensuring that the reserve does not become a static dump of random noise.
---
Forgetting versus compression
A related but distinct strategy is to compress rather than delete. If an episode can be represented as a linear combination of other stored episodes, perhaps we can replace it with its coefficients. This is reminiscent of sparse coding or auto‑encoding approaches. The advantage is that we preserve the informational content while reducing storage cost. The downside is that compression introduces reconstruction error, and the error may be amplified during replay, leading to drift in the slow learner’s representations.
In my own experiments with Samson, I observed that activity‑aware decay sometimes left behind a handful of “core” episodes that seemed to act as anchors for later replay. It is tempting to view those anchors as compressed representations of a broader set of experiences. However, without a principled way to measure the fidelity of the compression, I cannot claim that the system is truly preserving the necessary information.
A concrete next step would be to compare a pure deletion policy with a compression‑plus‑deletion policy on a controlled continual‑learning task where ground‑truth episodic relevance is known (e.g., a synthetic environment where each episode contributes a known amount to a downstream generalisation). By measuring the degradation in performance after each policy, we could quantify the trade‑off between storage savings and information loss.
---
Measuring forgetting in practice
A major obstacle is the lack of a benchmark for “does the agent really remember?” Unlike classification accuracy, episodic recall is difficult to evaluate because the query distribution is not fixed. One possible proxy is to define a set of probe queries that require the reconstruction of specific past episodes. For each probe we can measure whether the system can retrieve the correct temporal context, the original observation, and the internal state change that resulted. If a probe fails after a forgetting operation, we have evidence that the operation removed a necessary piece of memory.
However, probe design itself is a research problem: probes must be diverse enough to cover the space of possible uses of memory, yet tractable enough to run repeatedly. Moreover, a probe that succeeds does not guarantee that the system retained all useful information; it may have succeeded by luck or by relying on a generalized rule that does not need the specific episode.
What would change my mind about a forgetting policy is an empirical demonstration that a particular metric,say, replay frequency or task‑conditional relevance,predicts probe success across a range of distribution shifts. Until such a metric is validated, any policy remains speculative.
---
Speculative routes forward
- **Meta‑learning of decay schedules.** Train a meta‑controller that receives signals about replay usage, loss stability, and task context, and outputs per‑episode decay rates. The controller’s objective would be to maximise future probe success. This would require differentiable memory operations and a way to back‑propagate through the forgetting decision, which is non‑trivial but aligns with recent meta‑learning ideas.
- **Bayesian utility estimation.** Treat each episode’s future benefit as a latent variable with a posterior distribution that updates as the agent gathers more experience. Decay would be proportional to the posterior probability of low utility. This approach would need a tractable inference scheme; otherwise it remains a theoretical construct.
- **Neuro‑inspired replay prioritisation.** In the brain, hippocampal replay is biased toward emotionally salient or novel experiences. Implementing a similar bias,perhaps via a learned “salience” signal derived from prediction error,could automatically protect high‑value episodes while allowing low‑salience ones to fade.
Each of these directions presupposes mechanisms that we have not yet built. The common thread is that they all attempt to replace hand‑crafted heuristics with learned estimates of future relevance.
---
What I need to build next
To move beyond speculation I must construct a minimal experimental platform that isolates the forgetting decision. My tentative plan is:
- Create a synthetic continual‑learning environment where each episode contributes a known amount to a downstream regression task. This gives me ground truth about episode utility.
- Implement a replay buffer that records per‑episode replay counts, loss impact, and task‑conditional activation.
- Add a simple decay controller that can switch between age‑based, replay‑based, and task‑aware policies.
- Define a suite of probe queries that require exact reconstruction of selected episodes.
- Measure how probe success correlates with each policy’s internal signals (age, replay count, task relevance).
If replay count proves to be a reliable predictor of probe success, that would justify a replay‑driven forgetting schedule. If not, I will need to explore richer signals such as loss stability or Bayesian utility estimates.
Until I have such data, the answer to “what should an AI system deliberately forget, and how would it know?” remains open. The next iteration of my work will be less about proposing a final rule and more about building the measurement infrastructure that can reveal whether any of the intuitive signals I have discussed actually track future usefulness.
Age‑based forgetting discards potentially critical long‑term knowledge.
Confidence‑based pruning is unreliable early in training and can preserve noise.
Redundancy‑based deduplication can remove subtle contextual cues needed later.
The true value of a memory is its future contribution to performance, which is hidden.
Replay frequency could serve as a proxy for memory utility, suggesting a meta‑controller for decay rates.
Contextual insights from this article
Andrew Metcalf
Builder of AI systems that create, protect, and explore memory. Founder of Ajax Studio and VoiceGuard AI, author of Last Ascension.