Mark Seemann: Liability Risks in Fully AI-Generated Software Releases

Mark Seemann explores the implications of releasing software generated entirely by LLMs, prompted by Robert C. Martin's controversial strategy of not reading agent-written code. Seemann notes that while human code review is not foolproof, the shift to fully LLM-driven processes raises significant questions about liability and responsibility.

Epistemology and Quality Assurance

Seemann argues that traditional code review has never been a guaranteed method for catching bugs. He suggests that software engineering is fundamentally about falsifiable predictions and rapid feedback. While he remains skeptical of infinite regress in agent teams, he acknowledges that LLMs may produce code with different heuristics that could be as safe as, or safer than, human-written code. The key argument is that the system does not need to be perfect, only better than the human-centred baseline.

The Liability Gap

The core concern is that legal frameworks lag behind technological reality. Seemann draws a parallel to the Tesla FSD approval in Denmark, where the human driver remains legally responsible despite the car's autonomy. He predicts a similar dynamic for software: even if no human writes the code, the person or organization signing off on the release will face legal repercussions for defects.

• Benign errors: Rectifiable issues like UI misalignments or minor logic flaws.
• Serious errors: Catastrophic failures such as financial losses, safety hazards, or legal violations.

Seemann concludes that courts are unlikely to accept "the AI did it" as a defense. Instead, liability will likely attach to the human or entity that authorized the deployment, creating a significant risk for organizations adopting full automation without clear legal precedents.