I'm concerned about reliability

[russell]

For years, almost ten years, I've been an enormous FQ advocate. I encourage all my clients to use FQ services (and all but one have) and I do this on the basis that my personal experience with FQ has been that they are reliable. That rock solid reliable was worth paying a few dollars more.

That said, in the four days there have been two outages. Some of my sites are still down now. Speaking only about my personal sites, one of them is really taking off. These outages make me and the site look amateurish. My clients see these outages and they reflect back on me.

I very much appreciate the status blog. At least I can give updates and sounds reasonably informed but I would like to know what the problem has been? Will this be an ongoing issue or just very bad luck?

After almost a decade of flawless service, two stumbles in a short period of time has me asking questions. I look forward to any response FQ staff can post that will help me understand more about what's going on.

Thank you.

[kennylucius]

I, too, am concerned about this. The new status blog feels like a step taken in preparation for a bad time. Compared to other hosts, the FQ technical team is wonderfully open and honest about things--but I can't help being suspicious about the coincidence.

Are there growing pains a-comin'?

[Snarpy]

I, too, am concerned about this. The new status blog feels like a step taken in preparation for a bad time. Compared to other hosts, the FQ technical team is wonderfully open and honest about things--but I can't help being suspicious about the coincidence.

Are there growing pains a-comin'?It is absolutely clear that the status blog was implemented purely through client demand. FQ plans very carefully for any growth and I am confident there will be no "growing pains" that may come when a host bites off more than it can chew.

P.S. This post (http://www.aota.net/forums/showthread.php?postid=157821#post157821) has info that might help you.

[kitchin]

Have to agree that's FUD about the new status blog. No offense, it's just the first word that came to mind! People wanted the status blog and FQ put it up.

I'm mainly worried that shared hosting, and the web in general, is besieged by malicious traffic. The last outage was rouge user script. This one is a router? But the level of traffic is probably higher than any system could have handled ten years ago? Just a guess.

[russell]

Thanks but that actually does not address the issue at all. It's just the text of the status blog post. Which, while helpful, isn't meaningful because I'm still seeing problems with email on a number of different servers.

For instance, when I access this page, (http://www.visionquest.net/Uptime.php) I see that everything has 100% uptime on it. Well, whatever the cause is, it has not been up for 100% of the time.

As I said, my concern is that in my experience, this is UNusual. I want to know what has changed. That's all. Just saying such and such router is being power cycled doesn't tell me much.

[kitchin]

Yeah that's a good point. More info would be good.

[Joseph]

For instance, when I access this page, (http://www.visionquest.net/Uptime.php) I see that everything has 100% uptime on it. Well, whatever the cause is, it has not been up for 100% of the time.

The reason for that is that the page you reference has to be updated manually, it's not generated automatically or in real time. So we generally do not update that page until we are sure everything is back to normal, as that's the only time we can assess exactly how much downtime there was, and generate a new percentage. It would be a frivolous exercise to spend time updating that page during an outage, rather than responding to client emails and of course fixing the problem. Once the problem is solved and everything is back to normal, we are able to review, and update the necessary percentages accurately.

I also would just like to reiterate what a few others have said, in that the Status Blog was implemented solely due to client requests for such a page, and should not be considered an "ominous sign" of any kind.

We also acknowledge that some of you are still experiencing problems, and rest assured that we are still working to solve all of the problems to the best of our ability.

[kennylucius]

FUD is a pretty accurate description, though I'm not doing it with a purpose. I'm just naturally curious about coincidences. I'm a Cold War kid.

It still feels like a preparatory step (note the emphasis).

It would feel pretty good if it were such a step. The recent availability of PHP5 got me to wondering about how FQ planned to handle the extra strain caused by more and more clients upgrading over time. Now the routers are being upgraded rather suddenly. I would feel better about the recent outages if I thought they were not unexpected.

If there was a preexisting knowledge of the need to upgrade the routers, then the status page may not have been merely a response to client requests. It may have been a good, reasoned move.

[kennylucius]

Okay, I hadn't read Joseph's post before I posted my explanation. Sorry to hear that the router problems came out of nowhere.

I didn't mean to sound ominous.

[russell]

Normally, I wouldn't have time to post in this forum. I'm busy working. That said, what I'm mostly doing right now is watching the email access blink and on off like a morse code signaller on PCP. And fielding calls from clients asking if there is email is working. Yes it is. Ten minutes later, no it's not.

I look like an idiot here and frankly, if I told them their email would be working intermittently for the rest of the day and then be down intermittently throughout the this evening, I'd get an earful.

A long time ago a friend once told me that the art of cooking was learning to fix your mistakes. And someone even wiser said that we're all human. We all make mistakes. We all have accidents. Sometimes they're not even our fault. But how we handle those instances is what really defines our character.

[Andilinks]

The sad fact is that clients never demanded an offsite status page in the past because there was never a need for one until these recent few months. That is what has me disturbed. I would gladly pay three times what I pay now for the same level of service I got here 2002-2006... Perhaps the state of the internet now is such that better service will cost more, I'm ok with that.

I'm hoping I can get it here.

[Wassercrats]

The uptime page should be all you need to know how reliable Futurequest is, but I wish it was maintained by an independent monitoring service that had a reliable way to know the status of the servers. It would have to be a large, trusted company that specializes in this. I don't think there is such a service that's used by web hosts and I don't think it would be worth it for Futurequest to be the only one to make arrangements with some outside company just for a more impartial and frequently updated status page, but I wish some company would start to offer this and get a bunch of web hosts to use it so it would become a standard quality assurance measure.

[Andilinks]

There is this. (http://uptime.netcraft.com/perf/reports/Hosters)

[Wassercrats]

There is this. (http://uptime.netcraft.com/perf/reports/Hosters)

That website says:If you represent a hosting company and would like to be included in the table, we charge 1200 GBP per annum for inclusion....Interpreting the Tables: Using the performance of a hosting provider's own site to determine the performance of the hosting companies network, is only indicative.So Netcraft charges and all they do is check one server, which might not even be hosting any accounts.

Something closer to what I had in mind would be if Futurequest made a deal with one of the companies that checks webmasters' own websites, so each Futurequest client would have that service for free and Futurequest would pay a reduced rate. There would be results for each of our websites as well as statistical results for each server and a single overall result for Futurequest servers, all published on a public webpage by special arrangement with the monitoring service.

[Matt]

I have seen numerous concerns raised over this and similar issues. The question that I am increasingly asking myself is whether FQ really cares about the concerns that are voiced here. Putting up a blogspot that must be manually updated (pretty low tech) doesn't really count, in my mind, as a genuine response to user concerns. If FQ does care, does it have the resources to address these concerns? -Matt

[JoeRT]

For YEARS people have been begging for a third-party status page that just had simple messages like "There is a network problem and technicians are working on it." This is not something that came up the last few months, but something I've seen since shortly after I got here in July of 1999. There were even recommendations, from clients, it be at Blogspot. So now it's here. But now it's not enough and FQ's efforts to satisfy the masses are being met with criticism and called amateurish.

Over the last nine years I've had experiences with MANY hosts as people come to me asking how to improve their site's reliability. EVERY ONE that I've brought over to FQ has said things are SO much better. And that opinion has not changed the last few months. Even a client I brought over the end of last year said the recent outages were no where NEAR as bad as they had at their previous host.

Even with the recent problems I haven't had ONE client leave or ask for a change. Because they know what else is out there.

For those that want high reliability, mirror your site at another host and use DNS failover. It's cheap at zoneedit.com and dnsmadeeasy.com. I've done it for a couple clients that required ULTIMATE reliability. If you're willing to pay three times the price here's a way to do it for roughly twice the price. ;)

FQ's communication and ability FAR exceeds any other host I've seen. I've been with them through a few ups and downs. The downs have been VERY few and VERY short, while the ups have been extensive. This will pass. FQ is at the heart of one of my most critical programs and I have no intention of taking it elsewhere. My confidence remains in them.

[Andilinks]

For those that want high reliability, mirror your site at another host and use DNS failover. It's cheap at zoneedit.com and dnsmadeeasy.com. I've done it for a couple clients that required ULTIMATE reliability. If you're willing to pay three times the price here's a way to do it for roughly twice the price. ;)I would be concerned that the mirrored site would be indexed and counted as duplicate by Google. If I could somehow be assured that traffic would only go there as a fail-safe and it would not be visited by Googlebot I surely would do this. Robots.txt has already failed me in a similar situation, perhaps if I could deny all Google IP's to the mirror site. Even so, there's a risk there, maybe more risk than just having the current situation.

Thanks for the suggestion, I'll look into it.

Andi

added: Yes, a few hours traffic during a weekday brings in enough revenue to justify doubling my hosting costs.

[Matt]

For anyone concerned about ultimate reliability, here's the reality: shared hosting is not the solution. Andi, why don't you look into dedicated solutions?

FQ has come up with an answer to the demand for an external status page. I hope that it proves to be viable over the longer term. I imagine that long-term viability will necessitate a certain level of automation.

The subject of this thread is in regards to reliability. While FQ does a lot of things right and has been a very good decision for me, I am concerned as well about the overall reliability seen during the past 6 months - 1 year. I certainly think less communication is taking place between FQ and its clients here on the forums than has been the case historically.

[russell]

Thanks Matt. You managed to state my concerns better than I did. I'm feeling that FQ is slipping and all I want to hear from them is what's up. I know that there other ways to insure uptime. I'd prefer to stay here. Give me a good reason is all I am asking.

[JoeRT]

Dedicated servers are still subject to router failures, hardware failures, other servers on the network flooding networks, "backhoe fades" when cables get cut and other hazards. I maintain five dedicated servers for a firm that needs to be on no matter what. So they're at four major data centers around the country. Utilizing a fairly smart and self healing load balancing system, the country could be cut down the middle and people would never know... theoretically. ;) This last weekend one server had a hard drive failure and another had a network problem for about an hour. But no one outside of my partner and I knew there was a problem... not one call and no dip in traffic when the numbers were all totaled. If either of those servers would have been their only server there would have been a problem.

Bottom line... reliability still requires redundancy. :)

Oh... and they have absolutely no problems with their Google rankings. ;)

[Andilinks]

Yes, I will likely have a dedicated server next month, only a temporary glitch with Google kept me from upgrading this current month. But as Joe points out a dedicated server is not the end of redundancy and I will be continually be seeking additional reliability. No matter how well designed there are always vulnerabilities...

And I still have to keep adding content too. :)

This thread is a valuable discussion.

[Terra]

I am concerned as well about the overall reliability seen during the past 6 months - 1 year.
As are we!

I certainly think less communication is taking place between FQ and its clients here on the forums than has been the case historically.
That has been sort of a quandary for us as we've been trying to wrap our brains around the problem and find solutions that doesn't make it worse by fixing one thing that in turn breaks something else unforeseen... However, some of the comments make it seem like we've had problems every day, when in fact it has been far and few between once you look at the bigger picture... I can only surmise that it feels that way is because the problems have been clustered together, yet the time when everything is working perfectly well is cast aside... It is human nature to focus on the negative things and randomly discard the good or normal things... One of the hardest things in life is to not take things for granted... Problems will happen, as that is a simple fact of life for technology (more so when it is malicious intent causing it) yet it is within human nature to strive to reach some sort of semblance to perfection as is humanly possible in solving those problems...

I have posted up a response to the various threads that have been created as a result of the network instability...

http://www.aota.net/forums/showthread.php?t=22822

Hopefully it will better explain what is happening from an inside perspective... I am not trying to find excuses, but rather provide an honest perspective of what we are faced with and the plans underway to try and mitigate the problems that are within our grasp to ameliorate...

--
Terra
sysAdmin
FutureQuest

[manfred]

First of all… I think FQ is doing a very good job!

They are doing their jobs fast as they can. The hosters here in Europe should travel to FQ and learn how to get their webhosting to the FQ-quality level.

Have you ever asked other hoster to implement a special tool to their packages? I think 95% would say “NO CHANCE”.
The answer of FQ: “We are going to test the tool, if it’s ok we will activate it”

All my clients are from Austria, looking for a good and reliable hoster. Ok, here in Austria, there are some “good” hosters, BUT I didn’t get a fast, meaningful and helpful answer to a service ticket in less then 1 hour.

FQ clients thronged to start up an external status page. Now, here it is. And what’s wrong now? The written text on the external status page?
Clients now asking why there is this external status page!? I really don’t understand this thing!?

YOU, as the users of unsecure scripts, unsecure personal computers are a part of the problems in the whole www. Not updating your OS and your Anti-Virus-Software and the installed Webscripts…
“…Do not throw stones while you are sitting in a glasshouse…”

Maybe we all are spoiled childs?

A big and honest “THANK YOU” to the whole FQ-team for the unique service and support!!

Manfred

P.s.: English is not my native tongue, so please forgive grammatical errors ;)

[Andilinks]

I am not trying to find excuses, but rather provide an honest perspective of what we are faced with and the plans underway to try and mitigate the problems that are within our grasp to ameliorate...Once again when Terra speaks he puts all doubts to rest. My peculiar operating margins give me a greater flexibilty than most in my quest for reliability. Even so, I won't be going anywhere. Reliability was my first concern when choosing FutureQuest and remains so to this day.

I am sorry if I sounded alarmist but this recent series of outages this week following the outages earlier this year after several years of near perfection is an alarming pattern that needed to be addressed. My own concerns about reliability have been put to rest and I'd like to join Manfred in thanking the FQ team for their outstanding work.

[DogAndPony]

I certainly think less communication is taking place between FQ and its clients here on the forums than has been the case historically.I'm not seeing this at all. Seems to me that the amount and quality of communication has been consistent in the years since I got here.

But I do know there's been a huge increase in things like botnets and various flavors of malware.

I'd love to be able to look at some sort of historic comparison of connectivity and server uptime at the top shared hosts, to see how FQ has fared in relation to the rest of the world.

I would bet that while there have been increased problems at FQ, those problems have increased elsewhere as well.

[kennylucius]

Thanks for your Network State of the Union (http://www.aota.net/forums/showthread.php?t=22822), Terra. And for you who thought I was being too paranoid--I told you so! :rasberry:

[Randall]

I can only surmise that it feels that way is because the problems have been clustered together, yet the time when everything is working perfectly well is cast aside... I do have a sense that these things seem to happen in clusters, much like the spam attacks we're all familiar with. The bad moments have a tendency to stick out when we look back.

My main worry at the moment isn't the existence of a status blog -- we've been clamoring for it for a long time -- but the fact that I don't know how to report a problem when the whole network is inaccessible. I noticed the problem when I checked my email at 10:00 Eastern Time, but the first blog posting didn't come until 45 minutes later (glacial by FQ standards :wink:).

I don't know if the situation would have been handled any quicker if I knew how to contact FQ without using FQ, but the thought bothers me. :hrmm:

Randall

[sheila]

My main worry at the moment isn't the existence of a status blog -- we've been clamoring for it for a long time -- but the fact that I don't know how to report a problem when the whole network is inaccessible. I noticed the problem when I checked my email at 10:00 Eastern Time, but the first blog posting didn't come until 45 minutes later (glacial by FQ standards :wink:).
We have added contact information to the Status Blog page (http://FQStatus.blogspot.com), including an email address that will work when you are unable to access the FutureQuest network directly.

[Randall]

We have added contact information to the Status Blog page (http://FQStatus.blogspot.com), including an email address that will work when you are unable to access the FutureQuest network directly. Perfect. :yeah:

As far as those Other Hosts go, I discovered a security problem at my non-FQ host last week, which they fixed. It's the sort of thing Terra would have caught himself -- or never allowed to happen in the first place -- but they're still pretty good folks. Even upped my diskspace limit (try to imagine running a web site for eight years and suddenly discovering that you've got a hard limit of 50MB ... yes, 50).

Randall

[Jeff]

I remember my first host had a hard limit of 25 MB for $25/month -- if you wanted logs turned into graphical stats the cost was an additional $25/month.

We are quite demanding now as customers in comparison ;)