Trapped and Banned Robots

The following robots have either been trapped by the the robot trap or behaved unreasonably according to the access logs of this server. This page shows robots trapped after May 1999.

non table version
Name misuse of robots.txt ignores robots.txt gathers email addresses heavy load on server reads cgi scripts useless banned
imagelock ? - host
cyveillance - - - host
205.227.165.* - - ? host
216.32.64.10 - ? - - ? host
Microsoft URL Control - ? - - ? robot

Name	misuse of robots.txt	ignores robots.txt	gathers email addresses	heavy load on server	reads cgi scripts	useless	banned
imagelock	?		-				host
cyveillance	-		-	-			host
205.227.165.*	-	-	?				host
216.32.64.10	-		?	-	-	?	host
Microsoft URL Control	-		?	-	-	?	robot

Description:

misuse of robots.txt
This indicates that the robot used the file robots.txt to access directories that are not mentioned anywhere else. Such a robot is almost always immediatly banned from this site.

ignores robots.txt
A robot should always read robots.txt and ignore all directories listed there. Some don't and may therefore be banned from this site.

gathers email addresses
Such a robot uses those email addresses usually to send out spam and will be banned from further accessing this site.

heavy load on server
A robot should not try to download all files as fast as possible but rather take its time. Otherwise it may slow down users that want to access the site at the same time.

reads cgi scripts
Robots should never read cgi scripts, as the files generated by them is usually very dynamic and cannot be stored, cached or used for searches.

useless
If someone uses a robot to get text and images from my site and then makes some money from it WITHOUT that I get anything back (like more visitors) I take the liberty to block access to my site.

banned
True if the robot or host has been banned from accessing this site.

The imagelock.com company

This company offers the service to find illegal copies of images and text files that have been stolen from a website. To do this they regularly try to find all images stored on the web. They sell this service and therefore make money from accessing my site without actually giving back anything. As their robot also reads cgi scripts and traveresses the site as fast as possible all accesses from the imagelock servers (using IP numbers 209.133.111.*) have been banned.

The cyveillance.com company

This company offers the same service as imagelock.com, plus has programs that scans newsgroups and chat rooms for "bad" comments about a company. They sell this service and therefore make money from accessing my site without actually giving back anything. As their robot also reads cgi scripts all accesses from the imagelock servers (using IP numbers 207.87.178.6.*) have been banned.

The 205.227.165.* Robot

I was not able to figure out why this robot was indexing my site. It did it with an amazing speed (450 hits in 50 seconds) and accessed lots of cgi scripts not meant for robots. All accesses from the IP number 205.227.165.* have been blocked.

The Microsoft URL Control Robot

This robot seems to be used by different sites. At least some of them ignore the robots.txt file, and consequently this robot has been banned from this site.