Lies, Damn Lies & Click Fraud Stats

    May 24, 2007

Fair Isaac Corporation (FIC), a company with years of experience detecting fraud, recently turned their sights on click fraud.  FIC, who in the past has dealt with credit and insurance fraud, issued a press release with some early findings.  They report click fraud rates in the range of 10 to 15 percent.  But wait, it gets better…

These numbers are for fraudulent clicks after filtering.  In other words, customers are charged for clicks that are 10 to 15 percent fraudulent.  Compare this with Google’s filtered numbers of "less than 0.02 percent".  Quite a discrepancy. 

Recent studies by Google, Yahoo, and Click Forensics put unfiltered click fraud at 10 percent, between 12 and 15 percent, and 14.2 percent, respectively.  Both Google and Yahoo claim that these fraudulent clicks are filtered and customers are not billed for them.  The FIC data seems to contradict this.

Kevin Newcomb of Search Engine Watch, recently issued a plea for calm.  His argument is that the sample FIC used may not reflect the whole, something which FIC appears to agree with.  They state in their press release that "these are early results based upon a limited view of the market."

Fair enough.  Without a valid sample we cannot confidently extrapolate from the current data and deem it indicative of the whole.  I couldn’t agree more.  But then Kevin goes on to say:

The huge difference between FIC’s findings and the previous findings should have given them enough pause to wait until they had better data to report.

He’s referring to the earlier numbers by Google, Yahoo, and Click Tracks.  So let me get this straight, Kevin, you (rightfully) question the statistical validity of the FIC numbers but take numbers from Google and Yahoo on faith and imply their numbers should be used as a sanity check for any future research?  Google revealed nothing about their methodology (it’s proprietary), and they obviously have an interest in the results.  Google’s numbers (and Yahoo’s, for that matter) should be taken with a grain of salt the size of a Volkswagen if not outright suspicion.

Google’s response to the study is ironic:

Without knowing more about Fair Isaac’s data gathering and methodology it is difficult to comment on their study except to say that a handful of data is not a representative sample for our hundreds of thousands of advertisers.

This from a company that publishes a study and reveals absolutely nothing about their data gathering or methodology.  But despite their obvious hypocrisy, the statement is valid.  Maybe all involved should apply this level of scrutiny to research coming from Google.