iAWorks Builds Agentic AI PoC Validation and Evaluation Framework with Amazon Bedrock AgentCore

• Validated a LangGraph supervisor-based Agentic structure as a PoC to overcome the limitations of existing fixed workflows
• Separately applied Claude Sonnet 5, Haiku 4.5, and Opus 5 based on the nature of the question to optimize cost and quality
• Through an evaluation-driven improvement loop based on AgentCore Evaluations, identified and corrected flaws in the evaluation framework before agent defects
• Reduced response time for complex analytical questions by approximately 60% based on prototype standards through prompt engineering and structural optimization

Architecture and Model Configuration

• The triage node classifies questions into analyzable, insufficient information, and out-of-scope, and encodes routing via structured output
• The SQL ReAct subgraph performs schema search, SQL generation and execution, and interpretation, rewriting context as resolved_question during multi-turn conversations
• Designed to validate session continuity with AgentCore Memory and use a LangGraph checkpointer in production

Tool Integration and Evaluation Methods

• Standardized over 20 tools, including schema search and SQL execution, via MCP (Model Context Protocol) and connected them to AgentCore Gateway
• Utilized JSONL-based evaluation datasets and assertions to score quality using the LLM-as-a-judge method
• Applied a relative judgment method based on consistency with actual agent execution query results, rather than absolute figures