LG AI Research 337

LG AI Research proposed the QASA (Question Answering on Scientific Articles) approach and benchmark for sophisticated reasoning on scientific questions. Reflecting the nature of scientific literature, which requires more complex associative and logical reasoning than conventional machine reading comprehension settings, it includes 1,798 expert-level question-answer pairs.

The proposed QASA methodology goes through the following 3-step multi-step reasoning process to answer a question.

• Step 1. Associative Selection: Extracts the key paragraphs of the paper associated with the question.
• Step 2. Rationale Generation: Generates rationales—including the main answer, explanation, and supplementary information—based on the extracted paragraphs.
• Step 3. Systematic Composition: Integrates all generated rationales without redundancy to compose the final, highly readable answer.

Experimental results showed that among pretrained models, InstructGPT(175B) achieved the best performance, but the fine-tuned Flan-T5 model outperformed InstructGPT on the Full-stack QA task, demonstrating that QASA is an effective testbed. In addition, omitting the Rationale Generation step led to a drop in performance, confirming the importance of that step.