LG AI Research Wins 1st Place at the EHRSQL Task in the NAACL 2024 ClinicalNLP Workshop

LG AI Research and KAIST achieved 1st place in the EHRSQL task at the ClinicalNLP 2024 workshop held at NAACL 2024. This research focused on building a system that converts natural language questions from medical professionals into SQL queries to extract accurate answers from Electronic Health Record (EHR) databases.

In the medical field, incorrect information can be fatal, so securing the model's Reliability is essential. To this end, the team introduced a mechanism that allows the model to identify and avoid questions it cannot answer on its own. Using the evaluation metric Reliability Score (RS), they simultaneously measured the model's ability to generate accurate queries and its ability to refrain from answering uncertain questions.

The research team used the following approaches to improve model performance:

• Self-training: After training an initial seed model, a two-stage strategy was applied that used the model to identify unanswerable questions and then retrained the model on an expanded dataset.
• Filtering mechanism: Comprehensive filtering techniques were used to ensure the quality of the generated SQL queries.

This achievement is expected to help medical staff without technical expertise easily access complex EHR data, contributing to closing the gap between technology and clinical practice.