"In the past decade, AI's success has led to uncurbed enthusiasm and bold claims-even though users frequently experience errors that AI makes. An AI-powered digital assistant can misunderstand someone's speech in embarrassing ways, a chatbot could hallucinate facts, or, as I experienced, an AI-based navigation tool might even guide drivers through a corn field-all without registering the errors. People tolerate these mistakes because the technology makes certain tasks more efficient."
"Increasingly, however, proponents are advocating the use of AI-sometimes with limited human supervision-in fields where mistakes have high cost, such as health care. For example, a bill introduced in the U.S. House of Representatives in early 2025 would allow AI systems to prescribe medications autonomously. Health researchers as well as lawmakers since then have debated whether such prescribing would be feasible or advisable."
AI has achieved major successes and generated widespread enthusiasm, even as models regularly make errors. Digital assistants can misunderstand speech, chatbots can hallucinate facts, and navigation tools can produce dangerous guidance without recognizing mistakes. Users tolerate these flaws because AI increases efficiency. Advocates increasingly seek AI deployment with limited human oversight in high-stakes areas such as health care. Proposed legislation would permit autonomous AI prescribing, raising the risk of harmful outcomes. Complex systems research reveals interacting components produce unpredictable results. Training-data properties contribute to AI errors, making some mistakes likely to persist despite further research.
Read at Fast Company
Unable to calculate read time
Collection
[
|
...
]