Kurly Data Science Team Shares Operational Practices for Demand Forecasting Models and Leakage Prevention

Kurly's Data Science Team shared their experience developing and operating a demand forecasting model to improve logistics efficiency for dawn delivery. The team supports timely product ordering, inventory maintenance, and securing packaging materials, and since even a 1% difference in accuracy can significantly impact logistics costs, precise forecasting is essential.

Model Operations and Collaboration Principles

After model deployment, in the course of field operations, post-deployment work, performance measurement, and communication take up a larger share than development itself. An Anomaly Detection system is used to manage missing data or outliers, and the team responds sensitively to business changes or external factors (holidays, promotions, etc.). When modeling reaches its limits, changes to field processes or rule-based responses are used in parallel.

Leakage Prevention and Time Series Modeling

Rather than traditional time series models, the team primarily uses tree-based Regression models such as LGBM and Random Forest. To prevent the Leakage problem that frequently occurs in time series forecasting, they strictly separate the point in time when a forecast is made from the point when data becomes available for collection. For example, in a one-day-ahead forecasting model, lagging features are generated starting from n=2 to block the inflow of future information.

Project Management and Expectation Setting

Consistently forecasting in one direction can be more useful for a model than a mix of over- and under-prediction. In some cases, a simple formula suited to the purpose is more useful than complex modeling. At the early stage of a project, a WBS is used to agree with the requester on the shape of the deliverable, and it's important to align expectations through actual samples rather than verbal discussion alone.