Wassercrats
10-12-2003, 11:27 PM
I'm ridiculously close to completing the link checking function (full site and more) of LinkStructor, and I planned to announce it in my 1000th post, but what the heck...
I got the whole spidering thing done, but I'm wondering how to handle the with/without www issue. This is the option I'm considering (keep in mind that there will be a simple, default link checking function, but the following is for power users)Spider Locally
To indicate that the only extracted links used as pointers to other documents to process should be links within the same website as the source document, select a method for identifying a local link.
Off
Specified
Follow links pointing to the same domain as the source document.
Smart
Compare the documents at the source document address with and without the www. If identical, the www of extracted links will be ignored when comparing them with the source document.The source document is the document that the configurer specifies as a starting point for the spidering. If the same home page could be retrieved with and without the www, is that a good enough indication that other pages of the site also work with and without the www?
I got the whole spidering thing done, but I'm wondering how to handle the with/without www issue. This is the option I'm considering (keep in mind that there will be a simple, default link checking function, but the following is for power users)Spider Locally
To indicate that the only extracted links used as pointers to other documents to process should be links within the same website as the source document, select a method for identifying a local link.
Off
Specified
Follow links pointing to the same domain as the source document.
Smart
Compare the documents at the source document address with and without the www. If identical, the www of extracted links will be ignored when comparing them with the source document.The source document is the document that the configurer specifies as a starting point for the spidering. If the same home page could be retrieved with and without the www, is that a good enough indication that other pages of the site also work with and without the www?