Research on PoLar Technology to Improve LLM Inference Efficiency

LLMs fundamentally execute all layers in a fixed order, but this research presents the Program-of-Layers (PoLar) concept, which composes layers like modules without training, allowing them to be skipped or repeated according to specific inputs.

The main contents of the research are as follows:

• Dynamic execution structure: Generates an optimized execution program by skipping or looping layers depending on the input value.
• Efficiency and accuracy: Results on mathematical reasoning benchmarks demonstrate that, compared to the existing fixed layer execution method, accuracy can be increased or maintained while using fewer layers.
• PoLar prediction network: Proposes a method that dynamically generates an execution program optimized for each input through a lightweight prediction network.
• Potential applications: This technology enables the construction of a flexible inference stack that allows users to directly choose the trade-off between inference speed (Speed) and inference capability (Competence) at inference time.

Experiments were conducted on various open-weight models including Llama-3.2, Qwen1.5, Qwen2.5, Qwen3, and it was confirmed that the performance improvement persists even on out-of-distribution data.