Cloudflare announced that websites using its services will block AI crawlers by default unless the site owner chooses to allow it.
Additionally, Cloudflare is testing a new feature called Pay Per Crawl, which allows websites to charge AI companies whenever their bots access a page.
Will Allen, who oversees AI-related tools at Cloudflare, stated that more than a million websites had already switched on the company’s older AI-blocking tool.
Did you know?
Subscribe – We publish new crypto explainer videos every week!
Crypto Fees Explained: How Not to Overpay? (Animated)
Currently, with the new setting turned on by default, even more sites will be protected without requiring additional steps. Allen also noted that Cloudflare can detect bots, even if they attempt to hide their identity, using tools that analyze patterns in their behavior.
Previously, websites used a system called robots.txt to guide bots on what they could and could not access. However, this method depends on companies choosing to follow the rules, and many AI firms do not.
With Cloudflare’s new approach, website owners can have more control over who sees or uses their content. Nicholas Thompson, CEO of The Atlantic, said AI companies have often used content freely, and that may change if they are required to pay or negotiate access.
Meanwhile, Microsoft recently introduced a new AI system, which it claims can outperform doctors when diagnosing complex medical cases. How does the AI system work? Read the full story.
Having completed a Master’s degree in Economics, Politics, and Cultures of the East Asia region, Aaron has written scientific papers analyzing the differences between Western and Collective forms of capitalism in the post-World War II era.With close to a decade of experience in the FinTech industry, Aaron understands all of the biggest issues and struggles that crypto enthusiasts face. He’s a passionate analyst who is concerned with data-driven and fact-based content, as well as that which speaks to both Web3 natives and industry newcomers.Aaron is the go-to person for everything and anything related to digital currencies. With a huge passion for blockchain & Web3 education, Aaron strives to transform the space as we know it, and make it more approachable to complete beginners.Aaron has been quoted by multiple established outlets, and is a published author himself. Even during his free time, he enjoys researching the market trends, and looking for the next supernova.
