Anthony Katsur, CEO of IAB Tech Lab, advocates for structured agreements between publishers and generative AI models due to issues of content theft. He unveiled the LLM Content Ingest API Initiative during the IAB NewFronts, which aims to help publishers license content effectively and monetize it through structured interaction. By imposing rules on LLMs regarding content usage, Katsur argues this can lead to better attribution, pricing control, and enhanced user experiences, moving away from the unregulated practices of content scraping seen today.
If an LLM is crawling a publisher and repurposing that content for its chatbots and there's no business agreement in place... then that's theft.
Publishers need structure and control. I would argue LLMs would benefit from structure and control as well.
Lock arms, block the crawling, stop the crawling. Engage with the LLMs, but do it in a way that's structured and within your control.
None of that is possible in the current crawling architecture.
Collection
[
|
...
]