SEO Tip – Prevent Search Engines from Indexing your XML Sitemaps

You may prevent search engines from indexing your XML sitemaps with the help of x-robots-tag set to noindex.

July 20, 2012

XML Sitemaps are plain text files that help you inform Google about every single web page, image and video file that is available on your site. For instance, if there’s a page on your site that may be hard to discover because none of the other pages /sites are linking to it, Google can possibly reach it through the Sitemaps file.

Sitemaps are created primarily for search engines but since they also have text, they may even show-up in regular search results thus competing with your other web pages. To see this for yourself, try a search query like filetype:xml inurl:sitemap on Google.

You can easily prevent HTML based web pages from getting indexed through the use of META tags but such a feature is not available for XML based Sitemaps.

Prevent your Site’s XML Sitemaps from showing up in Google Search Results

Prevent Google from Indexing your XML Sitemaps

John Mueller of Google, in a tweet, suggested an alternate solution – serve sitemap files with a “noindex x-robots-tag” to prevent them from getting indexed.

It should take no more than a minute to implement John’s suggestion. Here’s how:

Switch to the directory on your Apache web server where the sitemap files are (they are generally located in the top-level folder of the blog) and open the .htaccess file for editing.

If you have only one XML Sitemap file, add this code to your .htaccess file.

 <IfModule mod_rewrite.c>
  <Files sitemap.xml>
   Header set X-Robots-Tag "noindex"

Do replace the word “sitemap.xml” in the code with the actual name of your Sitemap file.

If you have more than one sitemap (like for imagesvideos or in the case of Blogger blogs that support multiple sitemaps), use the following code instead. Remember to replace sitemap1, sitemap2, etc. with the actual file names.

 <IfModule mod_rewrite.c>
  <Files ~ "^(sitemap1|sitemap2|sitemap3)\.xml$">
   Header set X-Robots-Tag "noindex"

Test your .htaccess changes

Once you have saved the changes in your .htaccess file, go to URI Valet and copy-paste the full URL of any of your XML Sitemaps.

If you have the X-Robots Tag with noindex listed in the Server Response, the implementation was successful and your sitemaps should no longer make an appearance in Google and other search engines.

Also see: Google Experts Answer your SEO Questions