robots.txt file

[Spiritsoul]

I've noticed in my Stats, that I keep getting an error msg because of a file called robots.txt
So I created one in notepad, and placed in the root file of my web page.
My question is: What should be in this file. Right now, it's blank.

Behind The Truth
wolfer@charter.net

Also, if you've looked at this site before, you may see some major changes.

[racefan20]

Re: robots.txt file

The robots look for robots.txt to find if you've set any "rules" for what they can and cannot access on your server. In most instances, you don't really need to worry about any of this, but you can read more here:

A Standard for Robot Exclusion

[Spiritsoul]

Re: robots.txt file

Thanks Racefan. That answered my question. Looked at your site. Nice...

[racefan20]

Re: robots.txt file

Thank you, I have a lucky spurt now and then.