In what is becoming an all too common refrain, iFixit CEO Kyle Wiens says Anthropic’s AI crawler has been hammering the company’s servers, ignoring its TOS.

Wiens took to X to ask Anthropic to stop crawling the iFixit website:

Hey @AnthropicAI: I get you’re hungry for data. Claude is really smart! But do you really need to hit our servers a million times in 24 hours?

You’re not only taking our content without paying, you’re tying up our devops resources. Not cool.

Kyle Wiens (@kwiens) | July 24, 2024

Wiens goes on to say that iFixit’s TOS expressly forbid AI crawlers from scraping its content, and the TOS was in effect before Anthropic began scraping the site. iFixit has since added Anthropic to its robots.txt file, but it remains to be seen if Anthropic will respect it.

Wiens post raises a number of concerning issues:

• It is inexcusable that an AI company trying to set itself apart as the responsible company in an industry beset with ethics issues is blatantly ignoring other companies’ TOS in order to grab content.
• It is even more inexcusable that Anthropic’s crawler is hitting any one company’s “servers a million times in 24 hours.” Such behavior has more in common with malicious DDoS attacks than it does legitimate activity.

Unfortunately, iFixit’s experience is far from unique. Read the Docs cofounder Eric Holscher responded to Wiens to say his organization had experienced the same thing:

Yea, they were hammering us over at @readthedocs as well. We were planning to write a blog post on it, since this behavior is definitely gonna get all AI crawlers blocked because of abuse, not even because of the copyright issues.

Eric Holscher (@ericholscher) | July 24, 2024

Several other individuals from well-known organizations—such as Wikipedia and PythonAnywhere—reported similar behavior from AI firms.

iFixit’s experience illustrates why so many people don’t trust AI companies. Such companies are playing fast and loose with the ethics of content ownership, not to mention creating a very real burden on the organizations whose content they are accessing. Rather than acting responsibly, AI companies are in a mad dash to vacuum up as much data as possible before legislation properly regulates the industry.

Hopefully Anthropic will do the right thing and adjust its policies in the wake of iFixit’s revelation.