Separate the Conflicts, Merge the Performance: MERIT, an Efficient LLM Post-training Strategy

Recently, Instruction Mixture, a method of mixing diverse data, has become the standard approach for training multimodal LLMs (MLLMs). However, when data of different natures are trained simultaneously, problems arise such as Negative Transfer, where one capability degrades, and Gradient Conflict, which incurs massive inter-GPU communication costs.

MERIT (Merge-Ready Instruction Tuning), proposed by Naver Cloud, first groups data according to similarity, then trains each group independently, and finally merges the weights. This approach requires no communication between groups during training at all, making it highly efficient even in environments with fragmented resources.

The core process of MERIT is as follows.

• Dataset-level Gradient Estimation: Computes the representative gradient of each dataset to identify the training direction.
• Conflict Matrix Construction: Measures the degree of conflict between datasets via Cosine Similarity.
• Balanced Partitioning: Separates datasets with high conflict and groups similar datasets together.
• Independent Training and Merging: After independently fine-tuning each group, produces the final model through Token-weighted Averaging.

This strategy leverages the fact that the model is already positioned in a well-aligned Flat Basin region, allowing an integrated model to be obtained without performance degradation through simple weight averaging alone.