Controlling crawling and indexing now documented on code.google.com

Webmaster level: All

Do you know how Google's crawler, Googlebot, handles conflicting directives in your robots.txt file? Do you know how to prevent a PDF file from being indexed? Do you know Googlebot’s favorite song? The answers to these questions (except for the last one :)), along with lots of other information about controlling the crawling and indexing of your site, are now available on code.google.com:

Controlling crawling and indexing



Now site owners have a comprehensive resource where they can learn about robots.txt files, robots meta tags, and X-Robots-Tag HTTP header directives. Please share your comments, and if you have questions you can post them in our Webmaster Help Forum.

Posted by Jonathan Simon, Webmaster Trends Analyst

Tweet

Jangan sampai ketinggalan postingan-postingan terbaik dari SEO CHips. Berlangganan melalui email sekarang juga:

Atau sobat juga bisa follow SEO CHips dengan mengklik tombol di bawah ini:

Artikel keren lainnya:

Ditulis oleh laki pada tanggal Selasa, 24 November 2015