AI BriefingBuilding an LLM for Code Repair
Key point
Replit is building a Replit-native LLM that interacts directly with the IDE environment and automatically fixes code errors.
Details
Current programming-focused LLMs are trained on source code and natural language data, but their ability to directly interact with the development environment or use tools is limited. To overcome this limitation, Replit aims to build more powerful AI tools by creating Replit-native models embedded in the development environment.
The first goal is a Code Repair model that leverages LSP(Language Server Protocol) diagnostic data. LSP identifies errors in code, but it doesn't provide solutions in every case. Replit leverages its rich LSP diagnostic data as training data.
To build the data, Replit uses Operational Transformations(OTs) to track every modification history of the code, combining this with session events to construct a timeline. This timeline includes not only LSP diagnostics but also various events such as code edits, package installations, and shell commands.
The data pipeline goes through the following process:
- Reconstruct the file system at the point of diagnosis.
- Generate and validate synthetic diffs using an LLM.
- As a result, produce a dataset of (code, diagnostic) pairs.
To improve the model's training efficiency, the training data excludes CodeActions that provide deterministic solutions, style-related rules, and non-Python projects.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.