|
RoboGen
Information
BORIS
THE SPIDER STRIKES AGAIN (Boardwatch
Sept. 1999). RoboGen is mentioned on the second page of
this article on web spiders by Thom Stark.
Free
Webmaster Tools, Issue 11. The editors for this eZine
selected RoboGen as the #1 web site for the week in the July
issue.
RoboGen
Help File, the documentation for the RoboGen software is
also available online.
General
ROBOTS.TXT Information
Martijn
Koster's site about the Robot Exclusion Protocol.
|
