"It’s important to realize that it’s not really actual robots or actual spiders out there…instead, it’s banks of machines …at Google’s data centers who open up an HTTP connection and request a page and then get it back," he says. "So any bank of machines (even 50 machines) could easily be requesting a bunch of different content."

"We try to refresh a large fraction of the web every few days," he adds. "So it turns out you really don’t need a ton of machines. Even a relatively small amount of machines operating in parallel and fetching pages in parallel can really be able to crawl of find new pages on the web in a very quick way."

Matt says that Google doesn’t give out the exact number, but that it’s somewhere between 25 and 1,000. I’m not sure what you can really do with that information, but it’s worth hearing a quick rundown of how it works for those who aren’t real familiar with how Google indexes content.

Related Articles:

> Google Rolls Out Breadcrumb Display in SERPs

> Google Makes it Easier to Tell Where Results Originate From

> Get More Links in Your Actual Google Results

Some web addresses help you understand the structure of the site and how the specific page fits into the site hierarchy. For example, consider a search for the biography of Vint Cerf (Google’s Internet Evangelist). The URL for one result, "www.google.com/corporate/execs.html," shows that the page is located in a page about "execs," under "corporate," which is on the "google.com" site. This can provide valuable context when deciding whether to click on the result.

Often, however, URLs are too long, too short, or too obscure to add useful information.

That’s where the breadcrumbs come in. Google says the feature should be available globally within the next few days.

Original Article: Google appears to be testing breadcrumbs in some search results, at least in some areas. If you are unfamiliar with the term breadcrumbs, it refers to the hierarchical display commonly used in site navigation. For example: Home Page>Product Page>Product A Page.

Do you utilize breadcrumbs on your site? Comment here.

Several bloggers have noticed Google displaying these types of breadcrumbs in various places in seemingly random results to some queries. For example, Rob Hammond provides the following screen shot:

Leo Fogarty provides another, which shows the breadcrumbs displayed in a different position within the search result:

Google’s use of breadcrumbs appears to only be a test, and a limited one at that. Google has talked repeatedly about sites having good site architecture in the past. This allows Google to more easily and quickly crawl sites.

Bing acknowledges this too. Rick DeJarnette of Bing Webmaster Center recently said, "You can have great content and a plethora of high quality inbound links from authority sites, but if your site’s structure is flawed or broken, then it will still not achieve the optimal page rank you desire from search engines."

Here are some tips from both Google and Bing regarding site architecture issues. In addition, Google recently provided this related information on getting your site crawled faster.

If Google begins incorporating the breadcrumbs display as in the above tests, on a mainstream level, that will be all the more reason to clean your site architecture up, at least in the navigation area. Site architecture certainly goes beyond this, but it is a key part of usability anyway.

Have you seen breadcrumbs show up in Google results? What do you think about the idea? Share your thoughts.

"This is a very exciting change as your published docs linked to from public websites will reach a much wider audience of people," says Google Employee "Marie F," on the Google Docs Help Forum.

The change does not apply to documents that are set to "allow anyone with the link to view (no sign-in required)." Any concerned users, who do not wish for their published documents to be indexed can un-publish them by:

- going to the "share tab"

- For documents and spreadsheets, choosing "publish as web page". For presentations choosing "publish/embed"

- Clicking the button that says "stop publishing".

Google notes that Google Apps users may find that they’re unable publish documents to the world if the admin of the domain has disallowed publishing outside the domain.

The company said they will be launching the change in a couple weeks. Keep an eye on the help forum for an update on when the change goes live.

For example, a site that loads something like this in Flash:

..might appear in a Google SERP like this:

This is new to Google’s Flash-crawling abilities. On Google’s Webmaster Central Blog, Software Engineer Janis Stipins breaks down just what Google can do when it encounters SWF files:

- Index textual content displayed as a user interacts with the file. We click buttons and enter input, just like a user would.

-  Discover links within Flash files.

-  Load external resources and associate the content with the parent file.

-  Support common JavaScript techniques for embedding Flash, such as SWFObject and SWFObject2.

- Index sites scripted with AS1, AS2, and AS3 even if the ActionScript is obfuscated.

With regards to AS3, Stipins says, "The ActionScript version isn’t particularly relevant in our Indexing process, so we support older versions of AS in addition to the latest."

Webmasters who have SWF files on the web that don’t want them or any of their external resources crawled, can take care of this with their robots.txt file.

"We hope that webmasters around the world can use the guide to improve their sites’ crawlability and indexing in search engines," says Brandon Falls of Google’s Search Quality Team.

If you are unfamiliar with Google’s SEO Starter Guide, it is a "compact guide" that lists best practices according to Google that both teams within the company and external webmasters can use to improve the indexing of their sites.

The guide was announced back in November, focusing on areas like improving title and description meta tags, URL structure, site navigation, content creation, anchor text, etc.

Here’s the list of the languages it is now available in:

- Arabic
- Bulgarian
- Catalan
- Chinese (Simplified)
- Chinese (Traditional)
- Croatian
- Czech
- Danish
- Dutch
- English
- English (GB)
- Filipino
- Finnish
- French
- German
- Greek
- Hebrew
- Hindi
- Hungarian
- Indonesian
- Italian
- Japanese
- Korean
- Latvian
- Lithuanian
- Norwegian
- Polish
- Portuguese (BR)
- Portuguese (PT)
- Romanian
- Russian
- Serbian
- Slovak
- Slovenian
- Spanish
- Swedish
- Thai
- Turkish
- Ukrainian
- Vietnamese

You can get the download link for any of those languages here. If you want to make sure your site is ranking as well as it should be in Google, it’s probably not a bad idea to go over it and apply the advice to your site – particularly if you are new to the game.

The resident decided that simply submitting a removal request was too easy–and lacked any kind of chance for a settlement from Google–so the case is now before a judge. While legal-eagles will enjoy the debate over whether this truly is an invasion of privacy, or trespassing, I got a kick out of Ars Technica’s comparison to how Google should have treated the sign:

In the real world, things like private roads and trespassing signs serve the same purpose as the tools Google provides for turning away its indexing robots; they are opt-out mechanisms from an earlier age. Forcing people to build a private road, erect a sign, and then still use some online tools to have the pictures pulled (after already being available to the world) seems unduly burdensome on a common-sense level, and it has little to do with whether a stranger pulls into your driveway simply in order to turn around.

The lesson here? If you don’t won’t Google driving on private property, use a "NoIndex, NoFollow" sign instead of "No Trespassing."

Comments

Here are the 7 Proven Ways to get Your Website on Page 1 Organically & then Convert as stated by the presenter Shawn Moore.

1. Content is King
   What your content says, what your images and video show not only give users a first impression, but those items are also important to how a search engine is going to interpret your website.
2. Navigation and Architecture
   Looking into the way that your menus and internal linking structure are important. Can the web crawlers see and index your navigation? Or is it in JavaScript or Flash that may not be as search friendly?
3. Blogs
   Using a blog is just another way to generate more content for the search engines to index and use to rank your website.
4. Quality and Keyword Rich Inbound Links
   The number of links you have is important, but the quality of those links is more important. Get links from relevant sites and ensure that those links are using keyword phrases and not just your website address or business name.
5. Database
   Shawn said that choosing the right database for your site is important. Some are not search engine friendly and that choosing the wrong one will cause lots of issues down the line.
6. Electron Press Releases or Optimized Releases
   These releases give your site more exposure in search engines and the keyword rich links in the content can help increase your rankings.
7. Domain Name Strategy
   It was recommended that, if you can, choose a domain name with keyword phrases in it as it’ll give an added boost in search engines. You can also purchase additional domain names and forward them onto your website. Try thinking about what a user may type into the address bar of their browser and then register that domain for easy access.

Some of the tips are well known, others were a bit questionable to me. I’m still trying to figure out how a certain type of database can hurt a site as it’s all about how the content is displayed on that page and the URLs used that is the part to be concerned with.

Overall, the session was a good one for those just starting out in the industry.

Comments

Swap its anchor text, nofollow it, Google doesn’t care, it seems. That first link on a page with multiple links to a second page receives Google’s notice. The rest, not so much.

Branko Rihtman at SEO Scientist noted the first link phenomenon isn’t a new concept. Rand Fishkin made that point at SEOMoz in March 2008.

Rihtman wanted to delve into the idea a little bit farther, resulting in his test of one page, two links, one destination page, to see if Google really did appear to index the first, and only the first, link:

So, in order to test the theory, I picked two sites that sometimes double as my furry lab animals and set them up so that site A links to site B with two links using different anchor texts. The phrases appeared only on a site A, they were not to be found anywhere on site B and if that site was to rank for any of those two phrases, it would be only due to the anchor text of the links pointing from the site A.

Based on his testing, Rihtman found Google only picked up on the first link. Then he dropped a nofollow attribute on the first link, to see if Google would skip it and rank the site for the second link. It didn’t happen.

“Besides the fact that nofollowing the top link will not get you around the Google ignoring the second link, it is interesting to see that Google will still count the top link even though it is nofollowed,” he said.

Webmasters who want to control their content and its entry into search indexes sometimes turn to the NOINDEX meta tag to help. Here’s the problem: it isn’t being handled consistently at the major search engines, according to Cutts. Even Google isn’t sure what to do with pages tagged as NOINDEX.

“The question is whether Google should completely drop a NOINDEX’ed page from our search results vs. show a reference to the page, or something in between?” Cutts said in his post. Options fall into one of three possibilities: drop a NOINDEX page from the index, show links/references to those pages, or find some other middle ground.

Naturally the topic has generated some webmaster debate. One commenter at Cutts’ post said it’s “disrespectful” to index something marked as NOINDEX. Most of the follow-up comments expressed similar sentiments.

You can express yours in the comment section below, too.

It read: Sympathize with Gaza.

When images of the statement did not appear on Google’s image index, the conspiracy theories began. When the image didn’t appear a few days after the match, Google was flooded with enough accusations that the company censored the image at the request of the Israeli government that the Google Images team was compelled to respond.

Matt Cutts lent the blogp-space on his blog to clear things up.

The Google EMEA (Europe, Middle East, and Africa) Team said all images could take a few days to appear in the index:

"[W]e definitely didn’t do this," they wrote. "The reason for the delay in the image showing up on Google Images was that it can take a few days between when an image appears and when its crawled by the Googlebot…. No-one [sic] from any government has contacted us about this image, and we have no reason to remove it."

They further point out that the image is available now when searching for [Aboutrika] or [Aboutrika Gaza].

Cutts reminds readers in that blog post that indexing images takes time, and that Google has made significant progress over the years since it once took months instead of days to index images.