http://www.labnol.org/forums/topic/new-information-added-in-robotstxt-of-blogger-based-blogs Help Center Topic: New information added in robots.txt of Blogger based blogs en-US Fri, 12 Mar 2010 21:27:10 +0000 http://bbpress.org/?v=1.0.1 q http://www.labnol.org/forums/search.php http://www.labnol.org/forums/topic/new-information-added-in-robotstxt-of-blogger-based-blogs#post-471 Sat, 01 Sep 2007 00:32:21 +0000 mahendra 471@http://www.labnol.org/forums/ <p>lolz ok... </p> http://www.labnol.org/forums/topic/new-information-added-in-robotstxt-of-blogger-based-blogs#post-470 Fri, 31 Aug 2007 23:48:03 +0000 Anonymous 470@http://www.labnol.org/forums/ <p>well, I am answering your question.<br /> NO for first question. YES for second question. </p> http://www.labnol.org/forums/topic/new-information-added-in-robotstxt-of-blogger-based-blogs#post-468 Fri, 31 Aug 2007 18:44:41 +0000 mahendra 468@http://www.labnol.org/forums/ <p>I have a question for amit,</p> <p>can we use our own robots.txt for a blogger blog..</p> <p>i would like my search labels to get indexed by google also.. as they give lot of traffic..</p> <p>any way around this..</p> <p>User-agent: Mediapartners-Google<br /> Disallow: .. So is this for letting adsense bot crawl the labels ?</p> <p>cheers </p> http://www.labnol.org/forums/topic/new-information-added-in-robotstxt-of-blogger-based-blogs#post-464 Fri, 31 Aug 2007 17:00:17 +0000 Anonymous 464@http://www.labnol.org/forums/ <p>I just noticed that a new information has been added in the robots.txt (<code>yourblogname.blogspot.com/robots.txt</code>) of any Blogger based blogs:-</p> <p>User-agent: Mediapartners-Google<br /> Disallow: </p> <p>This is to let bots crawl for AdSense content. </p>