The plugin can generate both comprehensive sitemaps of all URLs seen in server traffic and sitemaps dedicated to store URLs that have changed recently.

“Having both comprehensive and delta Sitemaps provides you with significant benefits, as you will always have a full, up-to-date list of all URLs on your website that search engines can use for deep crawl, as well as a concise Sitemap of URLs that were modified recently, which search engine crawlers can prioritize,” says Bing’s Duane Forrester. “This can help in keeping bot traffic bandwidth down. In addition, the Sitemap Plugin automatically adds <lastmod> values to your Sitemap, and generates <priority> values to the Sitemap based on how popular your URLs are.”

“SEO is a complex and ever changing area but there are some key elements that are still as relevant now as they were a few years ago,” he says. “One of them is building high-quality XML Sitemaps, a comprehensive and accurate representation of your website. That way, you keep control of what you want search engines to index in order of priority and you inform them of new or updated content on your website.”

More on the plugin here. It can be downloaded here.

This includes details based on content-type, like stats from Web, Videos, Images and News featured more prominently.

“This lets you see how many items of each type were submitted (if any), and for some content types, we also show how many items have been indexed,” explains Webmaster Tools engineer Kamila Primke. “With these enhancements, the new Sitemaps page replaces the Video Sitemaps Labs feature, which will be retired.”

There is also now the ability to test a sitemap. “Unlike an actual submission, testing does not submit your Sitemap to Google as it only checks it for errors,” says Primke. “Testing requires a live fetch by Googlebot and usually takes a few seconds to complete. Note that the initial testing is not exhaustive and may not detect all issues; for example, errors that can only be identified once the URLs are downloaded are not be caught by the test.”

Google also has a new way of displaying errors, which the company says better exposes what types of issues a sitemap contains. Rather than repeating the same kind of error numerous times for one sitemap, Google will group errors and warnings, giving a few examples.

For sitemap index files, Google aggregates errors and warnings from the child sitemaps that the sitemap index encloses, so users won’t have to click through each child one at a time.

The functionality of the delete button has changed as well. It will now remove the sitemap from Webmaster Tools from both your account and the accounts of the other owners of a site.

"Site owners have been leveraging Sitemaps to let Google know about their sites’ content since Sitemaps were first introduced in 2005," explains Google Webmaster Trends Analyst Jonathan Simon. "Since that time additional specialized Sitemap formats have been introduced to better accommodate video, images, mobile, code or geographic content. With the increasing number of specialized formats, we’d like to make it easier for you by supporting Sitemaps that can include multiple content types in the same file."

"The structure of a Sitemap with multiple content types is similar to a standard Sitemap, with the additional ability to contain URLs referencing different content types," he adds. You can see an example here.

Google reminds webmasters that previous sitemap rules still apply. There is a 50,000 URL per file limit, and a 10MB uncompressed file size limit.

Just recently, however, David Harkness posted to that same thread, pointing to official Google documentation for sitemap errors, which says under the "Too many Sitemaps" error:

The list of Sitemaps in your Sitemap index exceeds the maximum allowed. A Sitemap index can contain no more than 50,000 Sitemaps. Split your Sitemap index into multiple Sitemap index files and ensure that each contains no more than 50,000 Sitemaps. Then, resubmit your Sitemap index files individually.

The larger number was confirmed by Simon, who came back to the conversation, saying, "Thanks for resurfacing this thread as we’ve improved our capacity a bit since then. The limit used to be 1,000. The Help Center article you point to is correct. The current maximum number of Sitemaps that can be referenced in a Sitemap Index file is 50,000."

As Barry Schwartz at Search Engine Roundtable, who stumbled across this post points out, "This is a huge increase in capacity…Still, each Sitemap file can contain up to 50,000 URLs, so technically 50,000 multiplied by 50,000 is 2,500,000,000 or 2.5 billion URLs can be submitted to Google via Sitemaps."

In other words, you can have a lot of sitemaps in one sitemap index file. That’s some good information to know, and it is a little surprising that there wasn’t a bigger announcement made about this.
 

Related Articles:

> Google Highlights Answers in Search Results

> Google Addresses Sitemaps Issues for News Publishers

> Make it Easier for Google to Crawl Your Videos

"While we’ve become smarter at discovering this information on our own, we’d certainly appreciate some hints directly from webmasters," says Google’s Michael Cohen, Product Manager for the Video Search Team.

The formats cater to simple things like titles and descriptions within the HTML of a video page. Google by the way also suggests that webmasters make their markup on video pages appear in the HTML without the execution of JavaScript or Flash.

On top of supporting the aforementioned formats, Google has also kicked off a series of Webmaster Central Blog posts, which are aimed at giving tips to get your videos indexed. One subject they have already discussed is the submission of video sitemaps.

Webmasters can submit their video sitemaps to Google via Webmaster Tools. The video sitemap uses the Sitemap protocol, but it also has additional video-specific tags. The details on how to create a video sitemap are explained here.

Keep an eye the Webmaster Central Blog for further tips in the near future. You can see what Facebook Share and Yahoo SearchMonkey RDFa look like here.

"While there may be some sawdust and loose nails lying around at the moment, it won’t be long before a sturdy new edifice has been completed, and we think publishers will agree that the hard work will have been worth it," says Abe Epton of the Publsiher Support Team. Epton points out the following two issues:

- Webmaster Tools may report an incorrect "Last downloaded" date for News sitemaps, display a strange number of articles indexed, or display a News sitemap as "Pending" () even though Google News is already crawling the sitemap. The best way to determine if we’re crawling a sitemap or not is to check your server logs for Googlebot.

- From time to time, it may become necessary to resubmit your sitemap (for example, if the Type switches from News to Web). In order to do so, don’t click the "Resubmit" button at the bottom of your sitemap list; instead, click the "Submit a Sitemap" link, select Google News from the Type dropdown box, and give us your sitemap’s URL.

A couple months ago, Google posted a Frequently Asked Questions Page for publishers who either have their content currently picked up by Google News or are looking to do so. This is a good resource for publishers to keep bookmarked in case they have issues. Keeping an eye on the Google News blog is also a good idea.

Earlier this year, it was discovered that Gooogle had added over 20,000 publishers to Google News in a year’s time. There’s no telling how much that will increase in another year’s time.

For one, Google and the other search engines that make up sitemaps.org are now supporting as many as 50,000 "child sitemaps" of sitemaps index files. In the past, they only supported 1,000. With this increased limit, webmasters can submit up to 2.5 billion URLs with one sitemap index.

Google’s Webmaster Tools design update shows users all sitemap files that were submitted for a verified site. By the way, here’s a brief look at the update:

"This is particularly useful if you have multiple owners verified in Webmaster Tools or if you are submitting some Sitemap files via HTTP ping or through your robots.txt file," notes John Müller, Webmaster Trends Analyst at Google Switzerland. In addition, the indexed URL count for sitemap files in Webmaster Tools is "more precise" according to Müller.

XSD schemas have been updated to allow sitemap extensions. This allows for the creation of better sitemaps through the verification of more features. Müller says that sitemap file processing is also much faster than before, meaning the time it takes to submit a sitemap file, process it and see initial data, is much shorter.

The question is, are you taking advantage of all of this? If you don’t use sitemaps or aren’t sure if you’re doing it the right way, I would suggest perusing the following resources:

- The Study Mentioned Earlier – Sitemaps: Above and Beyond the Crawl of Duty

- Sitemaps.org

- Google’s Sitemaps FAQs

- Google’s Sitemaps Help Center

- Googles Forum Search

Just because they’re putting a stop to uploading videos to Google Video, doesn’t mean that you can’t still submit your videos to them. By the way, Google Video Product Manager Amit Paunikar notes that Google Video search algorithms power YouTube as well as universal search.

Google has simplified the process for submitting sitemaps, and extended Video sitemaps support to include Media RSS feed. That said, Google offers the advice of including the following things in your sitemaps to make your videos easier to find:

1. Landing page URL: This is the page where the video is hosted. It’s better to have a unique landing page for each video on your site.

2. Video thumbnail URL: Thumbnails provide a strong visual cue to the user. Your video thumbnail should be representative of a snapshot from the video, and should not be misleading in any way.

3. Title & Description: If these are accurate and descriptive, they not only help Google understand your video, but also help users choose the best video search result. Providing information about category, keyword tags and duration is always helpful.

As I recently discussed, video is becoming increasingly important to search. Making it easier for Google to crawl your videos has to help.

"While most Sitemap generators either crawl websites or list the files on a server, we have created a different kind of Sitemap generator that uses several ways to find URLs on your website and then allows you to automatically create and maintain different kinds of Sitemap files," says Google Webmaster Trends Analyst John Mueller.

The new Sitemap Generator is open source, and finds new and modified URLs from server traffic, log files, or files on the server. Then it can create the following sitemap files:

– XML Sitemaps
– Mobile Sitemaps
– Code Search Sitemaps (for source code you make available to users)

The Generator will also ping Google Blog Search and other search engines that support the sitemaps.org standard for all new and modified URLs (you can optionally include the URLs of the Sitemap files in your robots.txt file).

If you’re unsure how to get started, Mueller says, "Google Sitemap Generator is a server plug-in that can be installed on both Linux/Apache and Microsoft IIS Windows-based servers. As with other server-side plug-ins, you will need to have administrative access to the server to install it. You can find detailed information for the installation in the Google Sitemap Generator documentation."

Help and support for Google’s new Sitemaps Generator are available. There is a help group as well as a Webmaster Help Forum.

Google still prefers webmasters use Google Webmaster Tools for submission, though, but they also accept files listed in the robots.txt file, or submitted via the HTTP ping method.

The XML sitemap file format can be used for all web pages, which is also supported by all search engine. If a webmaster doesn’t trust Google to sift through their file types, control freaks can bask in their own specificity via other Google-supported file formats:

RSS and Atom 1.0 feeds

Text files with URLs

XML sitemap files for video search, indexed in Google Video

Media-RSS feeds for video search

XML sitemap files for Google Code Search, indexed in Google Code Search

XML sitemap files for mobile web pages

XML sitemap files for geo-data, for geographic data in KML or GeoRSS format

XML sitemap for News, a special format for those registered with Google News
 

More info