PDA

View Full Version : accidentally deleted robots.txt


CipherMeister
09-21-2002, 06:42 AM
Hi;
I just accidentally deleted robots.txt -- preparing to uplaod a revised copy of my site. www.TheCipher.com
Could someone either mail me a copy or drop a copy in my domian directory?

agent1@thecipher.com would work.

I had just saved copies of 404.html and fquestindex.html, my stats folder is still there too. Did I loose anything else in my bulk delete?

Thanks
Roger

CipherMeister
09-21-2002, 06:44 AM
make that index.fquest

Arthur
09-21-2002, 06:59 AM
Hi Roger,

The default robots.txt contains just two lines;
User-agent: *
Disallow: /stats/

Which tells robots that are crawling your site not to index you stats folder (only well-behaved robots adhere to this). If you have password protected your stats directory (which it isn't by default), then no robot would be able to get in.

More about robots.txt can be found at http://www.robotstxt.org/wc/robots.html

Arthur

PS Request Site Owner status for forums (http://www.futurequest.net/Community/SiteOwner/)

CipherMeister
09-21-2002, 07:16 AM
alrighty, thanks.
I just created a text file with those two lines and uploaded it.

I also requested SiteOwner status.

Thanks much
Roger