Monte Carlo: Building a Data + AI Observability Agent with LangGraph and LangSmith

Data engineers spend enormous amounts of time on manual troubleshooting every time a data alert fires, investigating failed jobs and tracing code changes. Monte Carlo introduced an AI agent capable of reviewing hundreds of hypotheses simultaneously to reduce this 'data downtime.'

Monte Carlo chose LangGraph to implement its complex investigation process as a graph-based decision flow. When an alert occurs, the system follows a structured methodology as below, with each node spawning child nodes as needed to perform parallel investigations.

• Checking code changes from the past 7 days
• Identifying changes affecting specific data pipelines
• Investigating events in the hours before the issue occurred
• Simultaneously investigating multiple potential root causes

During development, the team used LangSmith to visualize and debug the graph-based workflow. This allowed product managers to rapidly iterate on prompt engineering without going through engineering cycles.

On the architecture side, the team leveraged AWS services to build a scalable and secure system. In particular, they used Amazon Bedrock to integrate the latest foundation models into the agent without the burden of infrastructure management.