[FQuest Notice] PHP ZendOptimizer upgrade

[Terra]

Within the next 30 minutes, I will be taking all Apache engines offline momentarily in order to upgrade the PHP Zend Optimizer to 2.5.0...

This will address the issue noted in:
http://www.aota.net/forums/showthread.php?threadid=16584

This task should take no more than 60 seconds to complete...

--
Terra
sysAdmin
FutureQuest

[Terra]

The new ZendOptimizer is now in full production...

--
Terra
sysAdmin
FutureQuest

[dank]

I don't know if it's due to this, one of the other server upgrades, or something completely different, but I'm unable to access either of my phpBB forums this evening.

http://run-down.com/forum/
http://abledesign.com/forum/

Both are on SEVEN.

The rest of both sites seems to be working fine, including pages accessing MySQL.

Dan

[Joe]

I can see both without any problem Dan.

[dank]

Odd. I still can't load either one, but here's another that's working fine which I believe is also on SEVEN:

http://www.foodandhealth.com/forum/

I've never had this happen before...

Dan

[Terra]

We have received several reports that the 'phpBB' forums are no longer working properly... So far, I have not seen any other issues, with major (high traffic) PHP forums such as vBulletin...

On deeper investigation, the new Zend Optimizer is causing the Apache child to segfault and die... From what I can see, at this time, is a variable is being set from data retrieved through the mysql functions... When the data is being assigned to that variable, Zend Optimizer takes a left turn and moves the data to memory it does not have access too...

From another 'phpBB' report tonight, the problem may be related to how cookies are handled by that message forum as visitors with cookies being denied where allowed to surf the forums, yet those with cookies turned on would crash the Apache child serving the request...

In light of this problem, I am rolling this upgrade back and reinstating the previous Zend Optimizer version 2.1.0...

The rollback will begin within the next 10 minutes...

--
Terra
sysAdmin
FutureQuest

[dank]

Thanks for the full report and deep investigation. Much appreciated.

Sorry it causes you extra work, but at least it's nice to know I was on the right track. :)

Dan

p.s. One difference of potential note between the 3 installations is that the working one is the latest version, whereas the 2 I can't view are a couple versions old.

[Terra]

The rollback has been completed and all Apache engines are back online...

Dan, please retest your phpBB boards and let me know if they back in full operation...

Sorry about the confusion this has caused...

--
Terra
sysAdmin
FutureQuest

[dank]

Yep, all three are working now. Thanks.

Dan

[Terra]

p.s. One difference of potential note between the 3 installations is that the working one is the latest version, whereas the 2 I can't view are a couple versions old.
That is an important item of information, and should assist my investigation later...

Even if it is an older board, it still uses the functions that PHP provides to it... None of those offered functions should cause an Apache child to segfault and crash...

Debugging a live Apache/PHP engine is utterly mind bending, so this will most likely take me a great deal of time to isolate the offending PHP/ZendEngine source code... It also does not help matters that the ZendOptimizer is binary only... :(

--
Terra
sysAdmin
FutureQuest

[Joe]

I obviously didn't try posting or anything like that, but I could see both Dan's forums without any problem prior to the rollback. :dunno:

[dank]

I obviously didn't try posting or anything like that, but I could see both Dan's forums without any problem prior to the rollback.
Presumably because of the cookie thing Terra described.

Debugging a live Apache/PHP engine is utterly mind bending
I feel for ya... Let me know if I can be of any further testing help.

Dan
- crash test dummy

[sheila]

Originally posted by Joe:
I obviously didn't try posting or anything like that, but I could see both Dan's forums without any problem prior to the rollback. :dunno: Yes, I could too, and other clients' forums as well. However, I never logged into these forums prior to the Zend Optimizer upgrade and had no pre-existing cookies.

Don't know why that should make a difference, but it is the only difference I can come up with, and another client who was also experiencing this issue earlier this evening indicated that he believed it was an issue with the forum's cookies.

[Terra]

Looking at the traces which were captured during the event, I have isolated where it is crashing...

ZO == ZendOptimizer

phpBB causing a segfault with new ZO 2.5.0:
read in the replacement word list: phpbb_words
**crash**

phpBB working properly with old ZO 2.1.0:
read in the replacement word list: phpbb_words
getcwd("/big/dom/xexample/www/phpBB2/")
lstat("/big/dom/xexample/www/phpBB2/includes/page_header.php")
open("/big/dom/xexample/www/phpBB2/includes/page_header.php")

In both cases, the mysql read on 'phpbb_words' fully completed... Somewhere right after the MySQL query, and where PHP/Zend goes to do the 'getcwd()', it crashes...

I also do not see how this could be Cookie related, unless PHP/Zend is doing a bundle of work right before it starts to handle the 'page_header.php' file...

It could also be Session related, but that is yet to be determined...

From a cursory standpoint, I'd say the problem is in one of 5 areas:
1) the closing of the MySQL connection
2) variable assignment
3) Session related
4) Cookie related
5) OOP related

Dan, can you provide a second set of eyes, and look in your problematic (caused by the new ZO) phpBB boards source code and see if you can find anything between those two processing spots that jumps out at you?

--
Terra
sysAdmin
FutureQuest

[dank]

One thing I forgot to mention, I was logged into the new installation that was working after the change, so it isn't a straight cookie issue. Must be some combination of code and cookies/sessions.

can you provide a second set of eyes, and look in your problematic (caused by the new ZO) phpBB boards source code and see if you can find anything between those two processing spots that jumps out at you?
I can take a look, but I'm not entirely sure what I should be looking for. Just anything cookie/session related? Or 1, 2, and 5 also?

Dan

[Terra]

Dan, nevermind, I am going to look at the phpBB urls you posted earlier to see what is different between the versions that worked and failed...

--
Terra
sysAdmin
FutureQuest

[mromero]

Notice a couple of things that might be related. My PHPADS New seems slow and is not delivering ads as it usually does. This prevented many pages not loading for some reason of the other. I intend to take it offline later today as I was planning to do so anyway.

Also when logging in to my LINKSSQL it takes like forever to load some pages from the admin panel. LINKSSQL is cgi but it does have some PHP function somewhere I believe.

I thought my site was hosed today (I was so concerned I missed my lunch) and did a full restore of my data but it's something at FQuest I believe.

Taking PHP ADS NEW offiline now. Notice that to re-build my pages now is taking like 10 minutes now where it used to take like 20 secs.

Regards

[dank]

LINKSSQL is cgi but it does have some PHP function somewhere I believe.
That's only the optional front end, which is completely separate from the admin area.

Dan

[Neo541]

I had a problem that I notified Sheila about last night regarding my phpBB forum and cookies.

To expand on it a bit...

My admin account on the forum, which was the only one I had ever used on that computer, went down, and every time I tried to log in, it would crash. If I deleted all cookies, and closed browsers, it would let me go back, but only to crash if I tried to log in.

So, I thought it was the cookies, but I actually had an old account that I decided to try, and it worked fine, no problems.

I'm a little perplexed, but it doesn't sound like it's necessarily a cookie problem, but I can't figure out why some accounts worked fine and others didn't.

[Phil Chaney]

While looking for some phpBB mods, I happened to bump into a thread regarding a fix for the Zend upgrade - have a look...

Phil

http://www.phpbb.com/phpBB/viewtopic.php?f=14&t=177594

[squillo]

Any chance (that with the passage of time) the Zend Optimizer can be upgraded? I'm trying out the trial version of pMachine's new ExpressionEngine which I think relies on the Optimizer to protect the code. More on the issue here (www.pmachine.com/forum/threads.php?id=15356_0_20_0_C).

If an upgrade is still not possible that's fine, I'll probably purchase the full version anyway.

best
squillo

[mromero]

Originally posted by squillo:
Any chance (that with the passage of time) the Zend Optimizer can be upgraded? I'm trying out the trial version of pMachine's new ExpressionEngine which I think relies on the Optimizer to protect the code. More on the issue here (www.pmachine.com/forum/threads.php?id=15356_0_20_0_C).

If an upgrade is still not possible that's fine, I'll probably purchase the full version anyway.

best
squillo

I ran the latest copy of Expression Engine without any problem recently. If I am allowed to sell my license I could do so as I have no need for EE. I am preferring these days to run mainly Perl-based programs on my websites - the only PHP program I cannot yet replace with a Perl based one is VBulletin which is rally outstanding in its niche.