Butler Business Accelerator has released findings from a new study of mobile search engines and question and answer platforms. The study says ChaCha answered questions better than ten other mobile search engines and Q&A platforms including Google, Bing, Yahoo, Siri, Ask.com, Answers.com, and Quora.
“ChaCha delivered the highest quality responses consistently across the largest group of categories and question types,” said Trent Ritzenthaler, operating director of the Butler Business Accelerator.
Here’s a look at the rankings of the Butler University Q&A Intelligence Index, which measured the likelihood that “a user could expect to receive a correct answer in a timely manner to any random query using natural language.”
Here’s what the study had to say about Google:
Google’s response rate was 100%, but the first non-sponsored result on the search results page (which often times was not fully visible as an organic search result on the presented page on a mobile device) only presented an accurate answer about 50% of the time, according to the Butler University Q&A Intelligence Index. On a mobile phone, when accounting for the clutter of ads and the likelihood of extra clicks to achieve the answer, allowing for the answer to be within the first non-sponsored search result might be considered generous. Again, this study differs from the results found in the Piper Jaffray study, but differences are likely due to variations in methodology. For example Piper Jaffray found that Google scores highest in terms of navigation and information (Elmer-DeWitt, 2012).
…
In terms of handling structured data more effectively, Google is promoting direct answers using its new Knowledge Graph and Google Now technology, “which tap into the collective intelligence of the web and understand the world a bit more like people do,” (Google, 2012). The limits of Google’s algorithmic technologies are evident in the empirical results of this study and users’ actual experiences. Other Q&A platforms in this study are also incorporating similar algorithmic solutions.
Okay, the study was sponsored by ChaCha, but Search Engine Land says CEO Scott Jones told them the company had no involvement in the methodology, “nor did it seek to influence the outcome in any way.” So, take that how you will.
We did talk to Jones recently, and he was without a doubt, incredibly confident in ChaCha’s abilities. The company is on the verge of launching a new and vastly improved mobile experience (at least, according to Jones). More on its upcoming app here. Jones did admit that the current ChaCha desktop web experience is “crappy.”
Either way, ChaCha is really geared towards answering different kinds of questions than most of the other services on this list. Even Google thinks ChaCha has the best answers to some questions, despite hitting it with the Panda update:
I should note that this was totally an organic find. ChaCha did not point it out to us (and frankly, the answer ChaCha provides doesn’t really even answer the question).
You can find the study here.
