Karrot Releases QAHub for Running Real-Device Tests from a Single Sentence Input

Karrot's Mobile Division has released QAHub, a test automation platform. This platform automates the process where users input a single sentence, AI generates a test Flow, and the app is manipulated and verified on real devices.

Centralized Test Data and AI Utilization

Initially, scattered test records were consolidated to establish a foundation managing 22 projects and over 2,000 test cases. Subsequently, Test Copilot was introduced to analyze planning documents and code to generate test scenarios, unifying writing formats and preventing omissions.

Self-healing for Execution Stability

To resolve issues where tests halt due to pop-ups or changed Selectors, Self-healing functionality was applied. AI analyzes the screen to identify blocking elements or close pop-ups to proceed, and upon successful recovery, recommends Selectors via Slack to assist with code reflection. As of August, over 90% of tests requiring recovery passed without human intervention.

Combination of Code and AI and Path Replay

Fixed behaviors are handled by code, while parts requiring screen judgment are handled by AI to enhance efficiency. In particular, the Path Replay feature allows AI to save and reuse successful execution paths. Screen screenshots are compared using 64-bit fingerprints to recognize similar screens, and when a remembered path is used, execution completes in seconds without AI calls. As of August, 44% of regression tests utilized this feature to reduce execution time and costs.

Persona Verification and Failure Analysis

Virtual user personas are set to check text comprehension and usability issues, and upon failure, step-by-step screenshots and AI judgment records are provided to support debugging. When explaining failure causes, it relies on actual logs and records to minimize false positives.