Robots.txt

[Stecyk]

Hi Everyone,

I have a quick question. If at my root directory (www) I have several paths, some of which I do not want to have indexed, how do I indicate that in the robots.txt file?

For example, let's say I have...

www/subdirectory1/ - I want indexed
www/subdirectory2/ - I want indexed
www/subdirectory3/ - I don't want indexed
www/subdirectory4/ - I don't want indexed

I am assuming that once I don't index sudirectory3, all of its further subdirectories are also not indexed. If I incorrect in that assumption, please let me know.

[CDarklock]

I have a quick question. If at my root directory (www) I have several paths, some of which I do not want to have indexed, how do I indicate that in the robots.txt file?

Simple...

User-agent: *
Disallow: /path1
Disallow: /path2

Two things to keep in mind.

1. There is no way to explicitly *allow* a directory.

2. Disallowing a directory also disallows all directories it contains.

[Stecyk]

CDarklock,

Thank you very much!