Analyzing Blueprints Like an Expert Engineer with Agentic Workflow

To reduce the building design blueprint analysis that used to take a skilled engineer 2-3 days down to 10 minutes, the AWS Generative AI Innovation Center built a multi-agent system based on Agentic Workflow. The goal was not simple OCR, but to transplant into AI the thought process an engineer follows when reading and reasoning about drawings.

There were three core challenges.

• Since symbols and layouts differ by design firm, rule-based automation doesn't work well
• Not all information is explicitly stated in the drawings, so tacit knowledge must be interpreted
• There is a resolution constraint where feeding a large CAD drawing in all at once causes loss of detailed text and symbols

The solution was an orchestration structure based on AWS AgentCore, which analyzes the original .dxf file, separates common information from individual information, and distributes it to agents at each stage. As a result, schematic diagrams and control point information were processed in parallel, cutting the time from 2-3 days → about 10 minutes compared to manual work, and achieving 91% accuracy, with 90 out of 99 correct.

The workflow consists of 4 stages.

• Stage 1: Understanding CAD file structure to determine the subsequent work strategy
• Stage 2: Patching, where the drawing is cut into logical and functional units and re-rendered so the VLM doesn't miss anything
• Stage 3: Information extraction, where legend and equipment information is organized with text into .json and symbols into .png
• Stage 4: Deep reasoning, where the overall schematic context and tacit knowledge are combined to determine control point functions and connected equipment

In particular, patching was implemented not as fixed rules but as Context-Aware Intelligent Patching. Claude Opus 4.6 looked at good splitting examples and learned the criteria on its own, then repeated an Analyze → Modify → Evaluate loop, analyzing the cause of failures and modifying the code. For cases where the title was outside the box, it added top margin, and for complex drawings mixing Blueprint and Table, it improved by separating out the table information to leave only the pure Blueprint.

In deep reasoning, the role of the target control point was first identified from the overall schematic, then used together with surrounding patches, legends, equipment lists, and tacit knowledge prompts about each piece of equipment compiled by engineers. For example, when a DI is connected to an EC FAN, it is interpreted as a status signal, and even for the same DM, depending on its position in the schematic and the airflow, it was determined to have different names and functions such as outdoor air bypass damper, outdoor air damper, or mixing damper.

Ultimately, this project goes beyond simply building an AI that reads drawings — it is a case of structuring and reproducing, within a multi-agent system, the intuition and judgment criteria that engineers have accumulated over decades. It demonstrated that AI can handle at scale the highly difficult task of reading unstated information and judging the role of individual elements within the overall context.