[Python-Dev] Googlebot and the mail.python.org python-dev?archive (original) (raw)
A.M. Kuchling amk at amk.ca
Sat Feb 28 15:05:02 CET 2009
- Previous message: [Python-Dev] Googlebot and the mail.python.org python-dev archive
- Next message: [Python-Dev] What type of object mmap.read_byte should return on py3k?
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
On Sat, Feb 28, 2009 at 10:37:09AM +0000, Antoine Pitrou wrote:
have the following HTML tag in them: which explicitly instructs Web spiders not to index contents nor follow links.
I believe this makes spiders not index this page, but does follow links. Individual messages have "index,nofollow".
--amk
- Previous message: [Python-Dev] Googlebot and the mail.python.org python-dev archive
- Next message: [Python-Dev] What type of object mmap.read_byte should return on py3k?
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]