Performing QA Work with Generative AI: The Start of Smart Software Testing

Generative AI is establishing itself as an essential tool in the QA field, improving work efficiency and accuracy across every stage of software testing, from requirements analysis to defect analysis. AI automates the writing of repetitive basic test cases, allowing QA engineers to focus on exception cases or exploratory testing.

ChatGPT-4o was adopted as the final chosen model for its strength in analyzing planning documents in image or PDF format and automatically generating output in Excel format. This shortens the time needed for document analysis and data organization, and allows potential ambiguities in complex requirements to be identified in advance.

Prompt Engineering Techniques

Various prompting techniques are used to obtain accurate results.

• Chain-of-Thought Prompting: Breaks down complex problems step by step to induce in-depth analysis, and is effective for analyzing planning documents.
• Chain-of-Question Prompting: Identifies logical flaws or security risks early through a series of successive questions.
• Least-to-Most Prompting: Starts from basic test cases and gradually expands to more complex scenarios, handling structural exception situations.
• Role-based & Contextual Prompting: Assigns the AI the role of a QA engineer and provides specific work context to generate practical test cases.

Real-world Application and Limitations

When pilot-applied to the verification of a point coupon purchase service, cases requiring correction were found, but overall it was useful for functional verification. AI helps detect unexpected exception cases, but complex cases still need supplementation, and the output is not 100% perfect, making manual review essential. Going forward, the plan is to maintain a balance between the advantages of AI and manual testing, and to continue developing optimized prompts as new models emerge.