Sullivan & Cromwell and OpenAI develop Agreement Analyzer for legal review

Sullivan & Cromwell, in collaboration with OpenAI, has developed Agreement Analyzer, an AI tool designed to identify issues in draft agreements by integrating the firm’s legal expertise into a single review workflow. The tool allows lawyers to compare provisions against precedents and track interactions, producing analysis and proposed changes that attorneys can inspect and refine.

Workflow and Integration

Partner John Savva initially built Agreement Analyzer as a single-user proof of concept using Codex. He then worked with OpenAI to improve output quality, enhance efficiency through parallel processing, and scale the application for multiple simultaneous users. The tool integrates Sullivan & Cromwell’s workflows, curated firm precedents, negotiating playbooks, and ChatGPT Enterprise.

Lawyers input a draft agreement along with selected precedents, a playbook containing firm and client positions, and market practice information. The tool performs multiple review cycles from different perspectives, synthesizes outputs, and returns a memorandum with proposed tracked changes for the lawyer to review, adapt, or reject.

Transparency and Adversarial Review

Savva emphasizes that trust in AI output relies on full transparency, allowing users to see the source of any claim about market practice. The application employs an adversarial process where agents develop conclusions and other agents challenge them. Specialized agents review agreements in parallel for market deviations, risk allocation, clause interactions, and ambiguities.

Results are categorized into findings supported after challenge, dismissed findings, and unverified issues retained with qualifications. Lawyers can inspect these issues, their explanations, and relevant precedents to decide which findings warrant changes in the final markup.

Future Developments

Sullivan & Cromwell continues to explore tools shaped around specific legal tasks, aiming to support more of the research and writing process. Savva notes that advances in reasoning capabilities with GPT-6 Astra have moved things forward in terms of reasoning quality, addressing a previous bottleneck.