Text as a Serious Optimization Layer

Many ML researchers believe that 'real learning happens in the weights,' and tend to underestimate Text Optimization such as prompt or context modification. However, text optimization is an important state update mechanism that includes prompts, memory, RAG databases, and more.

Text optimization has three key advantages:

• A legitimate update mechanism: It performs the same functional role as gradient-based weight updates, in the sense that it changes future behavior in response to new information.
• High sample efficiency: In low-data regimes, text conveys information in a compressed, short, high-probability form, providing a far more efficient Inductive bias than weight updates.
• A new scaling axis (Update-time compute): Like increasing compute at inference time, text optimization allows a system to invest more compute into learning from a single experience.

In particular, through Reflective learning, a system can hypothesize about and test its own behavior in text form. This is highly useful in environments where the cost of failure is high, where the desired behavior is difficult to specify, or where offline trace data is abundant.