THE SPIDER STRIKES AGAIN (Boardwatch
Sept. 1999). RoboGen is mentioned on the second page of
this article on web spiders by Thom Stark.
Webmaster Tools, Issue 11. The editors for this eZine
selected RoboGen as the #1 web site for the week in the July
Help File, the documentation for the RoboGen software is
also available online.
Koster's site about the Robot Exclusion Protocol.