Quantcast

Could Botnets Inflate Search Engines’ Query Counts?

DuckDuckGo notices suspicious traffic

Get the WebProNews Newsletter:
Could Botnets Inflate Search Engines’ Query Counts?
[ Search]

Gabriel Weinberg, the founder of the DuckDuckGo search engine had an interesting blog post over the weekend, talking about some weird traffic the site was getting. He points to a couple SERPs generated by what he deems to be queries from botnets.

In the comments, Eric Klein says one of them looks like its trying to boost ranking by “making it seem that there are a lot of requests.”

“Could they be trying to game their placement on search pages by hitting the search sites often enough to pretend to be meaningful?” he asks.

Weinberg responds, “Ahh, that’s a very plausible theory.”

“It’s known as ‘referral spam,” adds @TristanPerryIX in the comments. “It spams servers with HTTP referrers for two purposes:

1) Sometimes webmasters will (like you done :)) review the logs, and then click on their website. I assume these spammers are (wrongly!) assuming that this is ‘good’ traffic which might result in a sale/advert click.

2) Some websites have an automatic ‘these websites link to us..’ section or widget. Naturally referral spam could be used to game this section, thus getting a backlink (thus gaming the search engines).”

DuckDuckGo doesn’t save IPs, which is one of its selling points. It blocks them at the firewall level, Weinberg says, noting that this is dissociated from query data. “If we didn’t block the most egregious botnet machines and abusers, our machines would almost instantly be under water,” he adds.

“This discussion now makes me wonder if other search engines include this errant traffic in their query counts,” says Weinberg. “We work hard to keep them completely out because they would overwhelm our real direct queries #s and therefore distort our perception of progress. We also separate out API requests for the same reason, which now also makes me wonder whether everyone else is doing that too.”

It’s an interesting factor to consider, when you think about search market share.

Below, you can view a recent interview we conducted with Weinberg:

Could Botnets Inflate Search Engines’ Query Counts?
Top Rated White Papers and Resources
  • http://www.gfwebsoft.com Glenn

    There are what I think of as “benign” bots, although I’m not sure they are always viewed that way. For example a “rank checker” has to query a Search Engine to determine how a keyphrase ranks. You may get a series of queries for multiple phrases at once. (For example, when an SEO is trying to determine how a customer – or their competitor – is ranking for a series of critical key phrases.)

    I had thought that these bots were viewed by Search Engines as harmless — since they serve a positive purpose. I’ve actually written one of these but have not yet added it to my website.

    What is your view ?

    • http://www.webpronews.com/ Chris Crum

      I don’t think I can comment on that with any certainty.