science

The device had already been on the market for about three years. Until then, the U.S. Food and Drug Administration had received unconfirmed reports of seven instances in which the device malfunctioned and another report of a patient injury. Since AI was added to the device, the FDA has received unconfirmed reports of at least 100 malfunctions and adverse events.

Medical devices, including those used in surgeries and critical monitoring should never have AI because we know AI hallucinates and will always make avoidable mistakes. It may be useful for research and diagnosis (or secondary monitoring) where a human has time to review and confirm, but when someone’s life is on the line, we don’t need to add additional variables that might ignore critical information or take circuitous routes for no good reason, compounding the potential risk of human error that already exists—which AI is not reducing.