[LMB] Re: Listees by Nation/Bios/WhoWhere Project

Jim Toth jtoth at megrez.org
Fri, 22 Feb 2002 11:50:10 -0500


On Fri, Feb 22, 2002 at 09:46:07AM -0500, David Klecha
(dklecha at i2k.com) said:
> Yay.  Replying to my own post!
> 
> Okay, the mirror is also up now at:
> 
> http://www.daveman.org/bujold-list/

Thanks for putting it up!

[snip]
> Both my sites are fairly low-traffic and out in the backwaters of the
> internet, so for the most part, these pages are as "quiet" (if not private)
> as you get on the 'net.  (FWIW, I did a Google search using some of the
> keywords on the site and got nada.  I've never submitted to search engines
> and don't plan on it.)  E'en so, some people have expressed concern and in
> such an event, I will cheerfully and willingly edit whatsoever you want me
> to edit.

You may want to create a robots.txt file.  If you want to tell all
robots to go away, then

  http://www.daveman.org/robots.txt

should contain something like:

  User-agent: *
  Disallow: /

Google and most of the other reputable search engines pay attention to
this.  I suspect your site will get linked to 'ere long, and thus it
will be found.

-- 
Jim Toth
jtoth at megrez.org