Has Google Begun Changing How it Indexes the Web?

Last summer Google announced a new project called "Caffeine", which was described as a re-write of Google’s web search architecture. Around that time, Matt Cutts discussed Caffeine with WebProNews, comparing it to the "Big Daddy Update" of 2005, which consisted of changes to the way Google crawls and indexes websites. It appears that more people are now seeing the effects from Caffeine out in the wild.

Have you seen possible Caffeine effects in use? Tell us about it.

Back before the holidays, Google made it a point to assure everybody that Caffeine would not be rolled out (except for at one data center) until after the holidays were over – January at the earliest. The reason for this was that Google didn’t want to shake everything up during a key time for businesses (they didn’t want a repeat of the Florida update).

The company let everyone know about its intentions at PubCon in November. In fact, a few days ago, Google’s Matt Cutts posted a video running through his presentation from that event on his blog. He also provided the slideshow. It covers much more than just Caffeine, but if you missed it, you may want to consider watching it anyway (Caffeine discussion starts at about 22:10 in the video and at slide 29 in the presentation).

"It’s a re-write of our indexing infrastructure. It’s taking the old way that we used to index things that we’d crawled around the web, and we’re replacing that with new architecture that’s fresh and that had been written to be more scalable, more flexible, [with] the ability to attach different types of data, and in the process of indexing, the ability to do more documents for a more comprehensive version of the web, and the ability to do it faster," Cutts says of Caffeine.

But enough background. Barry Schwartz at Search Engine Roundtable points to a WebmasterWorld forum thread where administrator Tedster claimed to have seen Caffeine in action at a number of IP addresses. He wrote:

I’m seeing the Caffeine data-set being served via this set of IP addresses: 64.233.169.147, 64.233.169.105, 64.233.169.103, 64.233.169.104, 64.233.169.99,64.233.169.106

It seems to take 5 IP addresses to build the complete SERP, where in the past it often took only 3.

Schwartz also pointed to another member’s post (Whitenight), who said:

Well, just tripled checked with offices/employees in Texas, Colorado, and Indiana. All 5 "control" keywords/sites showed live Caffeine.

That member’s latest post says that the Caffeine Dataset is also on http://66.102.7.99 and http://66.102.7.104.

We don’t know for sure if this is all really Caffeine in action though. Google hasn’t commented on it, and has not made any announcements regarding Caffeine since what Matt said above. Some people don’t believe this is Caffeine at all. As Schwartz notes, we’ll have to wait for Google to say something.

Still, January is almost over, and Google said it would wait until after the holidays, specifically mentioning the month of January. It’s about time for this to be rolling out to some extent. Speed has been emphasized a significant amount in Caffeine discussion, and Cutts told us that page speed would likely become a ranking factor. Regardless of whether or not you are witnessing Caffeine in action yet, rest assured that it will be here sooner or later, and any edge you can give yourself in the meantime is for the good of your own site’s performance. Speed will not only supposedly help you in search going forward, but it just makes for a better user experience.

Share your thoughts about Google’s Caffeine update.

Related Articles:

> Matt Cutts Talks Google Caffeine Update

> New Details on Google Caffeine Update

> A Markup That Could Have Big Implications for SEO