The Geometric Memory: Teaching AI Without Breaking It

The Problem With Brain Surgery

If you have ever tried to fine-tune a large language model, you know the ritual: take a massive frozen network, inject your data, and run backpropagation through a stack of transformer layers.

It can feel like performing open-heart surgery while the patient is running a marathon.

The problem is that backpropagation is blunt. When you update the weights of a large model, you risk catastrophic forgetting. The model learns the new task, but it may lose some of the structure that made it useful in the first place: language fluency, code ability, reasoning habits, or instruction-following behavior.

What if we did not have to rewrite the brain?

What if, instead of changing the neurons, we changed the geometry of the space they inhabit?

That is the idea behind a WarpKernel: an experimental adapter pattern that teaches an AI system by warping a representation map, not by touching the main model weights.

This is a prototype idea, not a claim that geometric memory can replace every form of fine-tuning. The useful promise is narrower: stable, inspectable adaptation around a frozen model.

Tilting the Table

Imagine a marble sitting on a flat table.

Standard fine-tuning pushes the marble directly. If you push too hard, it flies off the table. If you push in the wrong direction, it gets stuck somewhere strange. You are manipulating the state itself.

Geometric memory tilts the table. You do not touch the marble. You reshape the landscape so the marble rolls toward the target on its own.

In a neural network, the marble is the hidden state vector: the model's current representation of what it is thinking. The table is the high-dimensional vector space that representation lives inside.

By learning a Riemannian metric, we can reshape that space so useful memories become valleys and rejected behaviors become hills. The frozen model does not need to be retrained. It needs a better map.

What "No-Backprop" Should Mean Here

The useful part is not magic. It is scope control.

"No-backprop" can be a misleading shorthand, so I want to define it carefully. The backbone is not trained, and gradients do not flow backward through the transformer's deep layers. But the small WarpKernel still uses ordinary gradients locally. We are not removing optimization. We are moving it into a tiny adapter that can be inspected, reset, or swapped.

1. Freeze the Backbone

Take a pretrained model and freeze it completely. Not one transformer weight is updated. That is the safety property: the adapter is not directly rewriting the base system's learned weights.

2. Add the WarpKernel

Attach a tiny low-rank module to the representation you want to shape. The module learns a metric tensor:

$$g = I + A^T A$$

Here, $I$ is the ordinary Euclidean distance, and $A^T A$ is the learned correction that bends the space.

Because the correction is positive semi-definite, $g$ stays well-behaved. The metric is not allowed to invent negative distances or break the space.

3. Score the Depth

For a hidden state $h$, the kernel defines a scalar depth:

$$\text{Depth}(h) = h^T g h$$

Low depth is a valley. High depth is a hill.

So the training signal is simple: pull rewarded examples downward and push rejected examples upward. The backbone stays frozen; the small matrix $A$ learns the shape of the landscape.

4. Keep the Update Local

Instead of propagating error through every transformer layer, we update only the WarpKernel. The gradient is shallow and local. It depends on the current representation and the reward signal, not on rewriting the whole network.

That is the core trade: less expressivity than full fine-tuning, but much better isolation.

For Oxygen AI, this is the part worth paying attention to. A memory layer should be inspectable, reversible, and small enough to reason about. If adaptation requires rewriting the whole model, supervision becomes harder. If adaptation lives in a narrow geometric layer, it becomes something you can test, swap, and audit.

A Compact PyTorch Version

Here is a minimal WarpKernel. In practice, you would feed it hidden states from a frozen model or embeddings from a local model server.

import torch
import torch.nn as nn


class WarpKernel(nn.Module):
    """
    A low-rank Riemannian metric that warps representation space.
    The backbone model stays frozen; only A is learned.
    """

    def __init__(self, dim: int, rank: int = 8):
        super().__init__()
        self.A = nn.Parameter(torch.randn(rank, dim) * 0.01)

    def metric(self):
        identity = torch.eye(self.A.size(1), device=self.A.device, dtype=self.A.dtype)
        return identity + self.A.T @ self.A

    def depth(self, h):
        g = self.metric()
        return torch.einsum("bd,dd,bd->b", h, g, h)


kernel = WarpKernel(dim=3072, rank=8)
optimizer = torch.optim.Adam(kernel.parameters(), lr=1e-3)

for hidden_states, reward in dataloader:
    optimizer.zero_grad()

    depth = kernel.depth(hidden_states)

    # Positive reward should lower depth. Negative reward should raise it.
    loss = (reward * depth).mean()

    loss.backward()
    optimizer.step()

The important line is the loss. If a good behavior is a valley, then a positive reward should reduce depth. If a bad behavior is a hill, then a negative reward should increase depth. That still uses backpropagation, but only through the kernel parameters.

Why This Matters

Lower Catastrophic Forgetting Risk

If the backbone weights never change, this adapter does not directly overwrite the model's original knowledge. It can still steer behavior at the system level, and that steering still needs evaluation, but the base model remains intact.

Efficiency

A rank-8 kernel over a 3,072-dimensional representation has a tiny number of trainable parameters compared with a full model. That makes the memory portable and cheap to update, though not nearly as expressive as a full fine-tune.

Privacy and Portability

The learned memory can live in a small file. You can imagine swapping kernels like profiles: one for coding style, one for writing voice, one for a project vocabulary, one for safety preferences.

The base model does not have to move. The memory layer does.

Interpretability

Because the kernel is a metric, it gives you something to inspect. You can ask which examples became valleys, which examples became hills, and how strongly the space was warped.

That is very different from hoping a hidden full-model fine-tune did the right thing.

A Practical Prototype Path

There is one practical caveat: LM Studio's OpenAI-compatible local server usually returns text completions and embeddings, not internal transformer hidden states.

That does not make the experiment useless. It just changes the first prototype.

Instead of reading hidden states directly, you can use local embeddings as a proxy representation:

1. Run an embedding model in LM Studio.
2. Store examples with rewards: preferred tone, rejected phrasing, project facts, or coding style.
3. Train a tiny WarpKernel over those embeddings.
4. Use the learned depth score to rank memories before they enter the prompt.

That is not the same as modifying the model's inference loop, and it should not be described as true hidden-state training. But it tests the most important idea in a safe local loop: memory as a learned geometric landscape.

Limits

This is not a replacement for full fine-tuning.

A low-rank metric cannot learn every behavior. It will struggle with complex transformations that require new internal circuits. It also depends heavily on the quality of the representation you feed it.

The right use case is not "teach the model everything." The right use case is smaller and more interesting: shape retrieval, preference memory, reward-guided context, and stable adaptation around a frozen model.

The Future Is Geometric

We have spent years making models bigger and training them harder. That work matters, but it is not the only path.

The next frontier may be about smarter ways to adapt systems without breaking them.

By treating representation space as a malleable landscape, we can teach AI to remember without rewriting. The table tilts. The marble rolls. The memory becomes geometry.