Marketing Cost Optimization via Uplift Modeling with Multiple Treatments

The core of marketing cost optimization is deciding who to give how much to. Excluding users who would convert even without marketing, the problem of spending budget only on users likely to respond to promotions was defined as uplift modeling. Here, this is explained from a CATE perspective, and extended into a Multiple Treatments problem where the amount is divided into multiple levels such as 1000 won and 2000 won.

User groups are broadly divided into four types.

• Sure things: customers who convert regardless of marketing
• Persuadables: customers whose conversion probability increases due to marketing
• Lost causes: customers who don't convert regardless of marketing
• Sleeping dogs: customers for whom marketing has a negative effect

For actual model training, randomizationed interventional data was needed. By randomly assigning treatment amounts and collecting outcomes, confounding that simultaneously affects treatment and outcome can be reduced, and bias arising from observational data can be removed in advance. In the same context, testing both small and large amounts first also allows checking which treatment is more efficient at an early stage.

Among several CATE estimation methods, Double Machine Learning (DML) was chosen for the model. DML predicts outcome and treatment separately from confounders and then uses the residuals to estimate the causal effect, reducing bias and mitigating overfitting through orthogonalization and cross-fitting. The author cited as reasons for the choice its versatility in handling treatment and outcome whether continuous or categorical, and its ability to handle Multiple Treatments.

The biggest difference in causal estimation is that the ground truth cannot be directly observed. Since a single user only experiences either receiving or not receiving treatment, the counterfactual cannot be directly observed. Therefore, the model must estimate not only the actual outcome but also the outcome of the opposite situation, and for binary treatment, it is defined as CATE = E[Y(T=1) - Y(T=0) | X]. For multiple treatments, the score for each treatment was calculated, and CATE was derived as the difference relative to treatment=0.

AUUC was used for evaluation. Area Under Uplift Curve (AUUC), similar to ROC-AUC, calculates uplift while varying the score threshold, and checks the model's relative performance through the difference with the control group. In a situation where the ground truth CATE cannot be directly known, it was used as an offline metric to see how much better the model performs compared to a random model or other models.

At the policy generation stage, the process did not simply end with choosing the treatment with the largest CATE. Since a larger amount is not always better, a heuristic called CATE gap was applied to calculate the difference in CATE between larger and smaller treatments. Even when showing the same high effect, budget efficiency was improved by assigning larger treatments to users who need a larger amount, rather than to users who could respond sufficiently even with a smaller amount.

In the online A/B test, Cost Per Acquisition (CPA) was used as the key KPI. The control group was given promotion amounts randomly, and by comparing the CPA of the treatment group and the control group, it was confirmed how much cost was reduced compared to when the model was not applied. In the example calculation, the treatment group CPA was 1,429 won and the control group CPA was 1,643 won, confirming a savings effect of 214 won per user.

Ultimately, what determined success or failure was not model complexity but how well confounders were selected. If meaningful confounders are missing, a causal inference model is not much different from a general ML model. Marketing cost optimization should not be treated as a simple prediction problem, but as an end-to-end problem spanning experimental design, causal estimation, policy-making, and online validation.