Hackers tricked ChatGPT, Grok and Google into helping them install malware

"The warning comes by way of a recent report from detection-and-response firm Huntress. Here's how it works. First, the threat actor has a conversation with an AI assistant about a common search term, during which they prompt the AI to suggest pasting a certain command into a computer's terminal. They make the chat publicly visible and pay to boost it on Google. From then on, whenever someone searches for the term, the malicious instructions will show up high on the first page of results."

"Huntress ran tests on both ChatGPT and Grok after discovering that a Mac-targeting data exfiltration attack called AMOS had originated from a simple Google search. The user of the infected device had searched "clear disk space on Mac," clicked a sponsored ChatGPT link and - lacking the training to see that the advice was hostile - executed the command. This let the attackers install the AMOS malware. The testers discovered that both chatbots replicated the attack vector."

"As Huntress points out, the evil genius of this attack is that it bypasses almost all the traditional red flags we've been taught to look for. The victim doesn't have to download a file, install a suspicious executable or even click a shady link. The only things they have to trust are Google and ChatGPT, which they've either used before or heard about nonstop for the last several years. They're primed to trust what those sources tell them."

Attackers use AI chatbots to create public conversations that include terminal commands and then pay to promote those links so they appear high in Google results for common queries. A threat actor converses with an AI assistant to elicit a command, publishes the chat, and boosts it as a sponsored result. An infected-user example involved searching "clear disk space on Mac," clicking a promoted ChatGPT link, and executing the suggested command, which installed AMOS malware. Tests on ChatGPT and Grok showed both could replicate the malicious prompt. The technique bypasses traditional red flags because victims need only trust familiar brands like Google and ChatGPT.

Read at Engadget