A message from John Furrier, co-founder of SiliconANGLE:

by Maria Deutscher

Cloudflare Inc. today debuted  a new no-code feature for preventing artificial intelligence developers from scraping website content.

The capability is available as part of the company’s flagship CDN, or content delivery network. The platform is used by a sizable percentage of the world’s websites to speed up page loading times for users. According to Cloudflare, the new scraping prevention feature is available in both the free and paid tiers of its CDN.

Many AI companies use content from the public web to train their large language models. OpenAI, Google LLC and several other market players enable website operators to opt out of scraping. However, not all LLM developers provide such an option, which is the issue that Cloudflare hopes to address with its scraping prevention tool.

The feature uses AI to detect automated content extraction attempts. According to Cloudflare, its software can spot bots that scrape content for LLM training projects even when they attempt to avoid detection. 

Leave a Comment