LOTUS: An LLM-Based Framework for Large-Scale Data Processing

LOTUS is a framework for applying LLM and agent-based processing to large-scale datasets quickly and economically. Developed by researchers at Stanford University and UC Berkeley, it applies query optimization principles from relational databases to LLM calls.

The key feature is the separation of 'what to do (declarative programming)' from 'how to execute it (optimization)'. When a user defines a task (map, filter, reduce, etc.) in natural language, the LOTUS optimizer determines the execution plan using the following techniques.

• Batching: Grouping multiple calls together to increase efficiency
• Model Cascade: Appropriately allocating cheaper models and high-performance models based on difficulty
• Lazy Execution: Performing operations only when needed to prevent resource waste

The main operators include sem_map for data transformation, sem_filter for filtering data that meets conditions, sem_agg for aggregating data, and sem_join for combining two sets. It also supports both agentic operators for tasks requiring complex tool use and LLM-type operators optimized for clear-cut tasks.