Scaling Git at Every Scale: Cursor's New Git Storage System Continuity

Hosting large-scale Git repositories presents significant challenges in terms of scalability and reliability. While Git was originally designed for distributed workflows, it acts as a constraint in modern centralized hosting environments.

Limitations of packfiles

Data in Git repositories is stored in compressed packfiles. While these binary files are convenient locally, they are not suitable for large-scale management on servers. Since packfiles must exist on the file system, it is difficult to distribute them across multiple machines for parallel processing or to prevent failures.

Continuity's Approach

Cursor solves this problem by changing the internal server structure while maintaining compatibility with Git clients. It retains the constraint of exchanging packfiles over the network but avoids using packfiles internally, instead utilizing a content-addressed data store.

Git objects align well with distributed key-value stores that use SHA-1 hashes as keys. However, since Git's actual structure is a directed acyclic graph (DAG), simple key-value lookups are insufficient for performing repository operations. Because the DAG must be traversed step by step, optimizations to minimize network round trips are necessary.