Kevin crawler Information Page

This page explain why Kevin appears in your web log activities.

Kevin is collecting useful web information for our search engine.
Kevin respect the Robot Guidelines created to protect the internet users.

Kevin is crawling the internet searching and indexing web pages. If a given web site is active Kevin will retrieve it
and also retrieve some of its links as long as a robots.txt file is not found.

Kevin is documenting web sites which have the "title", "description" and "keywords" metatags in it.

You can prevent Kevin from accessing your web site by having a robots.txt file in your "/" directory.


You can get more information about robots.txt here.

If you have any questions or comments please contact us at dznet.com.