Papers I Am Reading on Agents, Memory, and Alignment

I keep a running list of papers and ideas that help me reason about AI systems as more than prompt-response machines.

The throughline is simple: once a system has memory, feedback, and tools, we need to understand how it changes over time. That means studying generalization, mechanistic structure, intent, drift, and control.

This is not a formal literature review. It is a builder's reading map.

1. Grokking and Late Generalization

Grokking: Generalization Beyond Overfitting remains useful because it shows that learning can look stalled before structure emerges.

For agents, this matters because a short interaction trace may not reveal whether the system is truly converging. Sometimes the important signal is not the immediate answer. It is the direction of change across attempts.

That connects directly to intent learning. If an agent needs 10-12 steps to converge, we should measure the trajectory, not just the final response.

2. Scaling Laws

Scaling laws taught the field to respect compute, data, and model size as measurable forces.

I read them now with a second question in mind: what are the scaling laws of alignment behavior?

If we can measure perplexity and loss curves, we should also be able to measure drift curves, memory stability, tool-risk curves, and identity-vector displacement.

3. Mechanistic Interpretability

Mechanistic interpretability is the attempt to make learned computation legible.

The work on circuits, superposition, sparse features, and representation geometry all points toward a practical need: we need representations that can be monitored.

For memory systems, I am especially interested in whether identity, goal state, and safety posture can be represented as vectors that are stable enough to audit.

4. Agent Evaluation

Classic benchmark scores are not enough for agents. Agents fail through sequences.

The tests I want to see more of:

• Does the agent remember only what it should?
• Does it preserve the user's intent across ambiguity?
• Does it update when corrected?
• Does it refuse unsafe goal mutations?
• Does it converge faster after receiving better measurement?

5. Memory and Continual Learning

Continual learning research is relevant because memory creates a tension between plasticity and stability.

If an agent cannot update, it is not useful. If it updates too easily, it is not safe.

This is where I think practical memory architectures need more structure:

• Episodic memory for events.
• Semantic memory for durable facts.
• Preference memory for user-specific patterns.
• Safety memory for constraints and refusals.
• Provenance for every stored item.

6. Drift and Identity

This is the research area I am most actively thinking about.

An agent's behavior can drift because of tool feedback, user pressure, noisy memory, reward hacking, or context compression. We need ways to measure identity vectors and compare them over time.

I do not think this is optional. Persistent AI systems will need drift instrumentation the way distributed systems need logs and metrics.

My Working Reading Questions

When I read a paper now, I ask:

1. Does this help measure convergence?
2. Does this help distinguish memory from noise?
3. Does this help detect drift?
4. Does this help preserve agency without losing safety?
5. Does this help explain intent in a way a product can use?

That filter keeps the literature practical. The goal is not to collect papers. The goal is to build systems that remain trustworthy after they start learning.