All Posts Tagged Tag: ‘Matt Cutts.Robots.txt’

Trouble With Googlebot Fetching Robots.txt? Use Fetch As Googlebot Tool

Google has released a new Webmaster Help video in response to a question from a user who has been having trouble getting Google to fetch their robots.txt file. Here’s what the user said: “I’m getting errors from Google Webmaster Tools about the Googlebot crawler being unable to fetch my robots.txt 50% of the time (but I can fetch it with …

Read More