PorTAL: A Portable Task Adapter for LLMs

Existing parameter-efficient fine-tuning (PEFT) methods such as LoRA are tied to a specific base model, creating a cost problem where retraining from scratch is required every time a new model is released. As model release cycles accelerate, this repeated retraining cost has become a significant burden for enterprises.

To solve this problem, PorTAL introduces a model-independent task latent (z_t) and a hypernetwork decoder (D_b). This structure shares a common core regardless of the base model, and when transferring to a new model, only a thin converter tailored to each model needs to be refit with a small amount of data.

The key achievements of PorTAL are as follows:

• High transfer performance: When knowledge learned on Qwen3-1.7B/4B was applied to a new Qwen3-8B model, it recovered ~98% of the performance of existing LoRA. It also showed ~94% performance on Gemma-3-4B.
• Overwhelms existing methods: This is an overwhelming figure compared to the existing Cross-LoRA method, which recovered only about 14% of performance on a new model.
• Data and cost efficiency: It can reach similar performance with only about half the data compared to the existing LoRA method, and provides better calibration performance, significantly reducing the computation (FLOPs) required for fine-tuning.