Hugging Face's Design Philosophy for Transformers

Instead of applying DRY (Don't Repeat Yourself), a general principle of software development, to the Transformers library, Hugging Face adopted a 'single model file' policy. This approach concentrates all the code needed for a model's forward pass into a single file (e.g., modeling_bert.py).

The main reasons behind this design decision are as follows:

• Optimizing for open-source contribution: By keeping model code independent, side effects on other models are minimized when fixing bugs for a specific model or adding a new model, making review easier.
• User-centric code design: The primary users of Transformers are developers who read and modify code. By gathering all logic into one file, it becomes easy to understand and customize how a model works.
• Responding to rapid technological change: AI research evolves extremely fast. If attention mechanisms and the like were standardized into common modules, there would be a high risk of breaking existing models each time a new architecture emerges, so each model's independence is guaranteed.
• The static nature of models: Machine learning models have a static characteristic in that their structure does not change once deployed.