Is Toss's Speed and Quality Sufficient with Commercial Tools? — Tossion

The Toss QA Platform team built their own QA platform, Tossion, to solve the problem of scattered automated tests, manual tests, test cases, and judgment criteria. The goal is to move beyond the limited scalability and slow improvement cycles of commercial TCM (Test Case Management System) tools by directly developing necessary features and reflecting them immediately.

Tossion is based on a Project → Suite → Section → Test Case structure. Even if a test case changes, past verification records are preserved by storing test runs separately. When creating a run, the Assignee, Test Step, and Description are copied with their values at that time and preserved as a single row.

The key behaviors of test runs are as follows:

• Manage status in the order of Active → Completed → Closed
• Record status change history in each row of the test run
• Fix test cases, comments, and automation results as snapshots when Closed
• Select test cases based on Section, Type, and Platform
• Test cases manually specified by humans are prioritized over filters
• Provide charts for progress by Assignee and remaining test count

Considering situations where multiple people test simultaneously, it displays current users and items being edited, and supports item-level locking and automatic unlocking. Status changes made by other users are reflected without refreshing. Fields such as Status, Type, Assignee, Version, Platform, RNR, and History can be added or removed according to actual usability to keep only necessary items.

Toss determined that waiting for a tool's roadmap is insufficient to secure both speed and quality. They explained that to streamline overall QA tasks with AI, such as test case generation, PR analysis, and writing and executing regression test automation, new requirements must be reflected in the platform immediately.

Tossion's PR Analyzer collects PRs included in release milestones or RC tags immediately after RC build deployment. It analyzes PRs with QA labels separately from those without, specifically re-identifying changes classified by developers as not requiring QA but actually needing verification, and presents them along with a full release summary.

The analysis is handled by the QA Agent registered in Tossion. The agent periodically checks tasks and then executes the AI logged into the QA server. When there are many PRs, it splits the total into multiple batches for simultaneous processing to reduce omissions and increase analysis speed.