Subj : Last couple of days with my binkp mailer..
To   : Janis Kracht
From : mark lewis
Date : Wed Sep 29 2010 08:02 pm


 JK> Wow Baidubot is surely hitting on BBBS' webserver over here :(  Oh
 JK> well.. off to inet.bbb to get rid of them haha  

 ml> you could just deny them access to those links in BBBS that lead to
 ml> places like the messages, files and games... they are robots.txt
 ml> friendly... 

just to follow up and clarify this... an example is my gallery site... i may 
not want the spiders trapsing thru the exhibits and looking at the actual 
photos available... only the microthumbs and thumbs, sure so i just set up a 
disallow for the exhibits area and let them wander over the others... then, if 
i see them in the exhibits areas, then i know that they're not following 
robots.txt and i can then contact their admins with my complaint or just block 
them at the perimeter and deny them access to everything... depending on my 
mood, of course ;)

)\/(ark

 
* Origin:  (1:3634/12)