"A lot of these AI businesses are looking for readily available, structured databases of content," Robert Hahn, head of business affairs and licensing for The Guardian, told . "The Internet Archive's API would have been an obvious place to plug their own machines into and suck out the IP."
Although it's hard to recall the pre-chatbot world, ChatGPT and Perplexity only launched a little over three years ago, and Microsoft's Copilot followed a year later. In that short time, AI and AI search have taken root, and ads were bound to follow. The heat of the moment Chatbots could unlock major opportunities for advertisers - specifically, the ability to reach consumers at the exact moment they're ready to purchase something, with a highly specific understanding of what they're looking for.
Meta is informing some users that they will soon be restricted in how many link posts they can share each month, unless they pay for its Meta Verified subscription service. As you can see in this explainer email, which was re-shared by social media expert Matt Navarra, some Page and professional mode profiles are being informed of the new organic link limit.
It's increasingly important for publishers to understand when, how and why their content is being used as a source inside AI tools and platforms (often without attribution or traffic). The search landscape has changed exponentially this year, as more people turn to AI tools and platforms like ChatGPT to get information. And though referral traffic from AI platforms is still minuscule, many publishers are starting to track where they are cited in AI-generated responses to users' queries.
Publishers produce the content that makes streaming valuable, yet they carry the heaviest burden and earn the smallest share. Meanwhile, OEMs and streamers have tightened control of distribution and data. What began as a basic imbalance between supply and demand has evolved into a structure where the largest intermediaries set the rules and everyone else plays by them. Industry discussion often blames fraud, invalid traffic or unavoidable complexity.
It's been accepted wisdom that traditional publishers play a pivotal role in the digital media landscape, with standout editorial content serving as a rich source of premium inventory for advertisers. But that view is increasingly being challenged by the growing impact of AI search, whether it's Perplexity, ChatGPT, or Google AI Overviews. Only last month, Google expanded AI mode to more than 40 new countries and 35 new languages.
A good place to start finding answers is the most recent State of the Bots report from AI startup TollBit. For publishers that are feeling the heat of AI, it attaches real numbers to the presence of AI in the media ecosystem and how quickly it's growing. And while the rise of AI bots is a worrisome trend to those in the content business, it may also be an opportunity.
Google has one crawler, which means they use the same crawler for their search, where they still send us traffic, as they do for their AI products, where they steal our content,
The idea behind RSL is brutally simple. Instead of the old file -- which only said, "yes, you can crawl me," or "no, you can't," and which AI companies often ignore -- publishers can now add something new: machine-readable licensing terms. Want an attribution? You can demand it. Want payment every time an AI crawler ingests your work, or even every time it spits out an answer powered by your article?
Google's CEO, Sundar Pichai, said in May that web publishing is not dying. Nick Fox, VP of Search at Google, said in May that the web is thriving. But in a court document filed by Google on late Friday, Google's lawyers wrote, "The fact is that today, the open web is already in rapid decline."
