Gemini 4 Carbon takes up to an hour on bug-fixing tasks, driven by unnecessary work and slow inference
·2026.11.20 08:38
Key point
Analysis of bug-fixing traces reveals that 50% of Gemini 4 Carbon's steps are wasted on auditing code and closing loopholes not requested by the task.
Details
- Gemini 4 Carbon takes close to an hour to complete bug-fixing tasks that Opus and Astra finish in minutes, with Carbon's high-effort runs averaging 215 steps compared to Astra's 19.
- Approximately 50% of Carbon's steps are wasted on work the task ticket did not request, specifically auditing code beyond the bug (70 steps) and closing loopholes the ticket had already ruled out (47 steps).
- Slower inference accounts for 16–30% of the time gap compared to Astra and 45–49% compared to Opus, while mechanical inefficiencies like re-reading 39% of lines and making fewer tool calls per step account for roughly 25% of the excess runtime.
- Injecting a check-in prompt every five turns reduced Carbon's runtime from 37 minutes to 28 minutes, suggesting the slowness stems from behavioral habits rather than a lack of knowledge about when to stop.
Performance Metrics
- Carbon processes 24.7M input tokens and writes 152k output tokens per bug at high effort, compared to Astra's 1.5M input and 20k output at max effort.
- Carbon takes 23–24 seconds per step at medium/high effort, whereas Astra takes 12–18 seconds and Opus takes 6–15 seconds.
- Carbon identifies the bug cause within 6 minutes but waits 25 minutes before making the first code edit, a delay that shrinks to 10 minutes with periodic check-ins.
Methodology and Scope
- The analysis uses a private eval where the agent diagnoses and fixes a bug in a codebase based solely on symptom reports, requiring engineering judgment to trace symptoms to the correct code path.
- Tests were conducted across different harnesses (opencode, mini-SWE-agent) with no meaningful difference in performance, ruling out harness-specific issues.
- Carbon's low-effort setting often failed to produce a fix, resulting in circular behavior where the model re-read code without making changes.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.