Patient Dies Due to Judgment Error in AI Bed Management System

Core-MG, a state-operated AI bed management system in Brazil, rated a patient's severity as low at 6.8 points (out of 10), causing a 5-day delay in transfer to the intensive care unit (ICU).

At the time of the incident, the patient was hospitalized for gallstones, and despite the condition rapidly worsening, the patient was not transferred because other patients who received higher scores were ranked ahead due to the AI's scoring method. It was pointed out as a major problem that the medical staff's autonomy to reject the system's decision and intervene directly was limited.

This incident serves as a case showing the AI safety and ethical risks that can arise when medical automation systems replace or override clinical decision-making, and the bereaved family is currently pursuing legal action, raising issues with the system's role.