FQuest Alert: Scheduled Downtime

Justin · 01-13-1999, 04:28 PM

Quote:

The number of writes (log files) to the hard drive are more than double to the number of reads (web pages) - by splitting off the read/writes to seperate drives will help to optimize our hardware...

Are you telling me that there's more logging than actual page viewing? I always expected it to be drastically the other way!

Just wondering...

------------------
Justin Nelson, SFE Inc.
http://www.vdj.net

**Deb** · 01-13-1999, 04:46 PM

That's exactly right... but if you think about it, it makes sense...

One page with 10 images....

One person views this page = 1 viewing

10 images get logged separatly, we reverse resolve the visitor, log the page hit, check the referrer etc etc etc etc all because a single person opened the single page... a great deal happens everytime a single file is viewed....

It's all actually pretty amazing when you get to see everything from this side of the virtual fence.

Deb

Justin · 01-13-1999, 06:55 PM

Sounds interesting. My question is: more disk accesses or more data written to the disk? I would easily believe that there are more write accesses, but more data being written than read from? That's hard to fathom.

I'm not saying I don't believe you, just that it's hard to comprehend a server doing more logging than serving! I've always looked at a server as just something that stores information, although with these cool power toys (that I am still trying to figure out), that idea is changing fast

I find it really neat just being able to see the stats with so much detail. I'm still trying to figure out who from microsoft.com would visit my site and why. It's a little odd, to say the least. They don't have dialups with that domain, do they? It was just one visit, but they visited every page and downloaded my program. Scary.

Justin

**Terra** · 01-13-1999, 07:39 PM

2 points)

1) I have changed the downtime to 1/15/99, and have edited the above message to reflect.

2) Hard Drive access requests - not amount of data transfer...

Disk3 handles the /big/dom tree...
Disk1 handles the logfiles from 5 of our largest domains...

Current Status:
disk1: 423905r 3604973w
disk3: 3910470r 8214582w

If you want to get technical about the *total* amount...
8214582w
3604973w
============
118189555w Total access from log file writing

disk3 will be shifted over to disk4 for handling, which is only used by myself for developing...

--
Terra
--..- ..-- .-. .--...
FutureQuest.net

*Rich* · 01-13-1999, 07:41 PM

>>The number of writes (log files) to the hard drive are more than double to the number of reads (web pages) - by splitting off the read/writes to seperate drives will help to optimize our hardware...

via separate controllers, too?

>>Eventually we will upgrade to a complete SCSI based architecture...

A server without SCSI is like...

well, you get the idea

Hope this will be standard odering procedure for all new hardware? Duplexed controllers with RAID 1 or 5 also helps improve efficiency and provide an extra level of redundancy.

------------------
Rich

"What time is it in _____?"
www.timezoneconverter.com

**Terra** · 01-13-1999, 08:11 PM

SCSI is awesome - and wishful for us to migrate to..

2 Major negatives:

1) SCSI related downtime - if something goes wrong in the SCSI sub-system, it is *much* more difficult to isolate and fix the problem...

2) SCSI is very cost prohibitive for us right now... We looked at a Raid 5 sub-system for TAZ and would have cost us around $14,000 - 18,000 for the setup...

I do my best to balance the load on our UDMA (33Mb/s) EIDE primary/secondary ports, for which this is an effort to do...

Right now the only advantage of SCSI over our current setup is 'disconnects/command-queuing/scatter-gather'... In raw transfer power UDMA EIDE is easily keeping up with SCSI...

I am also waiting for Linux 2.2 kernel to stabalize, as it has built in hooks for the hardware based DPT SCSI controllers RAID solutions... I will probably utilize a modified RAID 2-5 architecture for the domain serving...

I would say that server 'wako.FutureQuest.net' will be the first to go SCSI based...

--
Terra
--forgot what I was going to write--
FutureQuest.net

Justin · 01-13-1999, 09:42 PM

This might be bad for my visitors - I am releasing the new version of VDJ at 12:00 AM (1:00 AM your time) on the 15th - I have had a lot of emails regarding the upcoming release, and if they are downloading right at that time...

No biggie, I guess. Maybe I'll put my ISP's link only for the first day, so if they started the DL, it will still finish. Just funny how my timing always works out

Besides, if it will help the overall performance of the server, so be it! (not that I've ever noticed a problem using my 28.8...)

Justin, whose used up his 2 cents for the next three weeks all in one day

**Terra** · 01-14-1999, 12:21 AM

At 2:00am, 1/17/99 - the Apache primaries will be taken offline for apprx 1 hour...

All other systems will be online: Email / Telnet / SSH / FTP / MySQL...

This is an effort to further enhance our speed of serving web pages...

I will be upgrading the Apache Daemon to version 1.3.4 (Over 90 new features, many of which I have been waiting for)

I am also splitting off all Apache/domain log files to a seperate drive, to help balance the load...

The number of writes (log files) to the hard drive are more than double to the number of reads (web pages) - by splitting off the read/writes to seperate drives will help to optimize our hardware...

STATS will also be delayed for a few hours as I rework them to properly follow the new symbolic link setup...

Domain owners should not notice anything different about their log files - other than the directory's symbolic link will point to '/big2/dom'... This is where the shadow log file directory tree will be stored...

Eventually we will upgrade to a complete SCSI based architecture (for which this begs for) - but for now, we are utilizing UDMA (Very Fast) EIDE hard drives...

--
Andrew Gillespie
Systems Administrator
FutureQuest.net

[This message has been edited by ccTech (edited 01-13-99).]

[This message has been edited by ccTech (edited 01-13-99).]

[This message has been edited by ccTech (edited 01-14-99).]

**Terra** · 01-14-1999, 10:19 AM

Since this is a *non-critical* upgrade, I have decided to shift the downtime to 1/17/99 at 2:00am...

The traffic will be much lower for us, so I hope that this will accomodate everyone...

It will also give me some more time to stress test Apache 1.3.4 a bit more - before I bring it to the forefront...

--
Andrew Gillespie
Systems Administrator
FutureQuest.net

Jacob Stetser · 01-14-1999, 10:43 AM

But....

Wait. Never mind, I don't even have my site finished yet

No conflicts for me here.

Jake