Pythia-70M fast-memory experiment

On top of a frozen Pythia-70M, forward-derived fast memory was added to experiment with one-shot symbolic recall without weight updates.

• In the write step, with the correct answer given, the memory value was computed as E[target] - sum_over_tokens p(token | h) * E[token].
• In the read step, the hidden geometry of the invented word was used as a key to find the memory via cosine top-1, and a correction vector was added at the answer position for greedy generation.
• As an example, blicket was bound to red in game A and blue in game B to test different contextual meanings.

Exact match in the same context, based on both_top1, was write 1.000 and read 0.805, while plain was 0.008 and unrelated was 0.000.

This showed that even in a frozen open-weight transformer, limited one-shot binding is possible using external memory and output embedding geometry.