Robots Articles

How Many Spiders Does Google Have?
· 4

Google has posted a short but interesting video to its Webmaster Central YouTube channel. A user asked the question, "How many bots/spiders does Google currently have crawling the web?" and Google’s Matt Cutts gave his answer.

Google Indexing Sites in 1 Day Again
· 1

I created a new site on Friday, and by Saturday exactly 24 hours later it was in Google’s Index. I posted about this just over a month ago in my post, 7 Steps to Get Your New Site Indexed in 24 Hours.

I had a lot of comments about whether or not Adwords was necessary, so I thought I’d try it again without running Adwords this time. Here’s how it all played out:

How Meta Commands Can Help You Love Spiders
· 2

Nearly all search engines utilize spiders (which are also known by their original name, robots) to go out and scour the web looking for web pages. These search engine spiders then bring the data back to be indexed by the engine.

Nofollow Outbound Links on Home & Category Pages

As you might have seen, I have now nofollowed all outbound links on my homepage and category pages.

Google Teaches Robots Tool About Sitemaps

The robots.txt analysis tool at Google Webmaster Central received some much-needed updating, and should be more effective for webmasters today.

Become a Business Referral Representative for Google

Keth of Gspy points us to the Google Business Referral Representative job application page. Your mission? To assemble information on local businesses, and to convince them to add their listing to Google Maps (and to pay for Google AdWords ads). From the program FAQ:

Google Flexes Robots Exclusion Protocol

Two new features added to the protocol will help webmasters govern when an item should stop showing up in Google’s web search, as well as providing some control over the indexing of other data types.

SES: Built By People, for People & Robots

Finally catching my breath from Search Engine Strategies Toronto 2007. As close to an unqualified success as I could have hoped. Many attendees including Pauline from HighRankings (who interviewed me for Jill’s newsletter) noticed that the freshness of the program spurred speakers into coming up with new, fresh, cutting-edge material.

Google May Ask You To Prove You’re Human
· 2

British digi-rag the Inquirer’s Fernando Cassia had a run in with the "GoogleMind," as he calls it, after entering a query that sent up red flags. To continue his search, Cassia had to prove he wasn’t a robot.

Technology Advancing To Replace People
· 1

The Internet is growing rapidly in many different formats. In an interview with WebProNews at the Web 2.0 Expo, Renat Khasanshyn of Apatar spoke with us about this growth and how it affects businesses.

AP Considers Selling News Ala Carte

From a story about AP by AP:

Rock Em Sock Em: MSN Versus Google
· 11

We know you’ve been waiting minutes, hours, and even days for the release of the Rock Em Sock Em Robot’s Fight from the SES conference. You can all rest better now because the much-anticipated rumble is up and ready for your enjoyment. Only WebProNews was there to bring you exclusive coverage of this event to find out which search engine is truly best.

Ensuring a Page is NOT Indexed by Search Engines

Dear Kalena…

I want to publish a private page on the web, that only I and a few other people will use. How can I ensure that this page is NOT picked up by search engines?

It will be a wiki style page, so there may be lots of content which could be indexed by the Search Engines. This is what I want to avoid.

Google On Making Use Of Robots Meta Tags

Owing to Danny Sullivan’s post on how search engines handle meta tags, Google’s Vanessa Fox has written a post on how Google handles these tags.

Robots for the Rest of Us

The robots.txt file.

Nothing can be more confusing to a website owner as the robots.txt file. Born out of technology in the programming world, the robot.txt file is nothing more than a server command for search engines. Unfortunately, while search engines understand the file, humans have a difficult time understanding machine language.

Google On Robots Exclusion Protocol

A post on official Google blog informs about Robots Exclusion Protocol. Sometime back we informed you about a previous post on Robots.txt file.

Korea Vows Online Robots In Every Home

A heaping dose of Battle Bots and movies like Terminator and I, Robot have instilled a (healthy?) paranoia of centrally controlled robot futures – at least in the States. In South Korea, not so much, as the country announced its plan to have a robot in every home – very, very soon.

Mobile Searchs Future Is Now

More and more, “sometime in the future” is yesterday, and the ubiquitous “New Year’s Tech Predictions” lists find themselves rendered obsolete by mid-January.

Anybody Care That The Robots Are Winning?

This shift to contextual advertising caused by intelligent software has privacy advocates going squirrelly, particularly because the larger public seems relatively unconcerned. After all, says the public, it’s all about the end-user experience, not whether a robot saw my iTunes list or my Gmail inbox and made a suggestion.

Japan Surrendering Moon To Robots

The Japan Aerospace Exploration Agency imagines a future where humanoid robots populate a lunar base in 2025.

UK Robots Will Hunt Underwater Mines

Britain’s Royal Navy should gain ten advanced unmanned mine-hunting craft next year.