Proposal for 'Infinite-Parameter LLM' Architecture That Generates and Adapts Weights with Real-Time Data

Existing LLMs have fixed weights after training, meaning they cannot reflect real-time data or modifications provided after deployment within the model. This creates a structural limitation where reliance on prompts results in information being discarded once the request ends. To address this, the proposed 'Infinite-Parameter LLM' draws inspiration from MoE (Expert Mixture). A compact hypernetwork converts runtime data into low-rank modulation of a shared base network, generating feed-forward weights in real time. Unlike existing weight generators, it does not read the context once and freeze it. Instead, it maintains a Bayesian belief over the generator's latent code and updates it online, re-deriving effective weights as the session progresses. While storage space remains fixed, the number of weights the model can compile is effectively infinite. Compared to in-context learning and retrieval, this architecture is expected to offer amortized computing costs, reduced dependency on the context window, persistent knowledge across turns, and better generalization performance.