Stable Cinemetrics: A Structural Taxonomy and Evaluation for Professional Video Generation

Stable Cinemetrics (SCINE) organizes filmmaking practices into a Setup, Event, Lighting, Camera disentangled hierarchical taxonomy to evaluate professional video generation. This structure consists of a total of 76 fine-grained control nodes, enabling systematic handling of prompts and evaluation questions tailored to real industry practice.

Based on this, the team built benchmarks tailored to professional use cases along with a pipeline for automatic classification and question generation, and conducted a large-scale evaluation of 20K videos generated by 10+ models. 80+ film professionals participated in the evaluation, and the results showed that even top models exhibited large gaps, particularly in Event and Camera control.

For reproducibility and scalability, the team also trained an automatic evaluator. This vision-language model, aligned with expert annotations, outperforms existing zero-shot baselines, pointing toward a direction for automating the evaluation of professional video generation.