Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

This study establishes a mathematical correspondence between discrete, hierarchical Decision Trees and continuous, dynamic Diffusion Models, unifying the two models.

In this unification process, the researchers discovered a shared optimization principle called Global Trajectory Score Matching (GTSM), and proved that an idealized form of Gradient Boosting is asymptotically optimal with respect to this principle.

For practical use, the study proposes two implementations:

• treeflow: When generating tabular data, it provides 2x faster computation while maintaining higher fidelity than existing methods.
• dsmtree: A new distillation method that transfers hierarchical decision-making logic into neural networks, tracking teacher model performance to within 2% across multiple benchmarks.