AutoAdapt: Automated Domain Adaptation for Large Language Models

AutoAdapt is a framework that automates LLM domain adaptation for high-reliability areas such as law, healthcare, and cloud-incident response. Based on an input goal, domain data, and latency·hardware·privacy·cost constraints, it designs an executable adaptation pipeline.

At its core is the Adaptation Configuration Graph (ACG). This structured configuration graph organizes the adaptation space—including approaches such as RAG, supervised fine-tuning, and LoRA—so that only valid pipelines can be explored.

On top of this, a planning agent selects a strategy and explains its reasoning, excluding choices that don't meet the required conditions. Rather than simply running black-box optimization, it produces an executable workflow that reflects best practices and explicit constraints.

Finally, AutoRefine adjusts hyperparameters through a budget-aware refinement loop. Even with limited feedback, it strategically chooses the next experiment, turning tuning that could take weeks manually into a more reproducible and auditable process.

In experiments, it showed more stable performance than competitive baselines across a variety of tasks including reasoning, question answering, coding, classification, and cloud-incident diagnosis. The added cost was small, averaging about 30 minutes of time and around $4, with low overhead relative to the performance gains.

The implication is clear: domain adaptation can now be treated not as ad hoc work but as an engineering procedure, allowing results to be produced faster, reproduced more easily, and audited more rigorously. The framework has been released as open source.