Best way to redirect

[MPaul]

I'm underway with my website redesign, and I'm almost finished. Since so much has changed (page name, extensions, location), I'll need to redirect the users (and bots). So I'm looking for the best redirection method, one that would be good for redirecting more than 50 pages.

Right now I know 3 ways of redirecting:

1) With the meta tag:

PHP Code:

<meta  http-equiv = "refresh" content="0;url=http://www.mydomain.com/newpage.php"> 


2) With javascript:

PHP Code:

<script language="javascript">
<!--
location.replace("some_folder/newpage.php")
//-->
</script> 


3) With .htaccess

Now for the user, the redirection method doesn't really matter (except the Javascript method which for users who don't have Javascript enabled will not work). But I'm more worried about Google (and other search engines). Which redirection method makes the bot realise that: "Oh, this page will no longer be updated, it has been permanently been moved, and that we should remove it from our database and replace it with the new one."

I also don't want to make Google mad

[Mandi]

There are several good approaches - but I wanted to add that whenever I do this, I always make sure my custom 404 page makes reference to the recent redesign and points straight to a current site map, and also features an "Email us if you still can't find something" email link.

[MPaul]

Thanks Mandi, I was thinking about doing a custom 404 to help users. But about the sitemap, is there any script that can automatically create a sitemap for me, or do I have to do it myself?

[Mandi]

I believe there are some scripts that will spider your site and produce some sort of map - but I really prefer to do it by hand. Here's one I did that works well:

http://www.kerrysherbals.com/sitemap.shtml

An improvement to that would be to use anchor tags with name=___ and the # thingy in the URL, so the person could jump straight to the item on the appointed page.

[Wassercrats]

Hey, I almost missed my cue! I wrote a script that could create a site map for you, but I have to run it from my own computer. It's under development, but it's useable. Here's a sample from a few months ago: http://www.polisource.com/PublicMisc..._Site_Map.html

The boxed parts on the bottom are easy to edit out if you don't want them. I'm looking for volunteers who will let me test it on their web sites, but I'm pretty aware of its abilities and issues.

[MPaul]

Is that the spider you've been talking about?

Well, I'll do both. I'll do a sitemap by hand now , because some parts in my redesign are still missing. And when I'm finished with everything, you can try your spider on my site

I'll give you the ok, when I'm finished with everything (in about 2-3 weeks, things should be finished)

[Wassercrats]

Yeah, that's the spider. In two weeks it will probably obey robots.txt (forgot to mention it doesn't yet) and pretty up maps of sites that have a bunch of directories with only one file in each. It's surprising how many websites have that.

[Randall]

A redirect permanent in .htaccess will tell Google that the page has been moved.

Randall

[Chipmunk]

I always use .htaccess, which, if you use "Redirect permanent" (as Randall pointed out), works fine with Google. I've done a few reorgs, and I find Google quickly picks up on things.

I'm sooooo fed up with other bots, that two weeks ago I rewrote our robots.txt so Google is the only one allowed. Unfortunately, it turns out that Google's image bot is too stupid to realize that it's barred, and their tech "support"'s reply to my bug report was "explicitly exclude it". [ we need a "grumble, grumble, grumble" smiley ]

Mandi, excellent point about the custom 404 page! We've still got a few external links that are wrong, so I regularly monitor that, and our 404 has direct links to all the moved pages.

The easiest way I know of to create a site map is to do a word grep (including subdirs) of "title" from the root dir of your local site image, redirect the output to a file, then use a text transformation tool to change the contents of each title tag pair and filename into a link. If you have the right utilities, it takes about 5 minutes to set up the first time, then just a few seconds to re-run.

That gives you a simple and complete one, but the real time consuming part is deciding what to exclude, and how to lay it out.

MPaul, I'm sure you'll totally make Barry's day to let his spider onto your site. Barry, I might even let you spider mine, some day, purely to help you stress-test it (we're pushing 700 pages). We don't have a site map (our navigation is too easy), but do have a complete "species list" that I used to generate using the technique above.

[Andilinks]

In two years my site has grown from 12 to over 80 pages. Google has always kept up well without any special prompting though I seldom abandon file names, branching and linking new pages from existing names. This keeps other, non-search-engine external links intact as well.

But when I have used redirects several times, I have used meta-tag redirects which seem to work much faster and Google has worked just fine for me with those.

When you move content and while Google is still pointing to the old page, the redirect will still send visitors to the new page from Google--it shouldn't matter. Google always catches up on indexing quickly though PageRank takes a little longer.

Just be sure the new location is generously linked everywhere using the new file name and Google will reindex at its usual speedy pace.

Andi