[FQuest Notice] PHP Upgrade 4.1.2 ==> 4.3.1

[Terra]

When: 4/13/2003 (Sunday)
Window: 1:00am - 5:00am EDT
Duration: 5 - 30 minutes

FutureQuest will be upgrading the core PHP release from: 4.1.2 to 4.3.1

We are providing a 5 day notice so that you have time to study the following document and can be prepared to make any changes necessary to your PHP scripts:
http://www.php.net/ChangeLog-4.php

This will also be the first step of ushering in FutureQuest's exclusive PHP Secure_Mode(TM) offering, which will shatter the current limitations imposed by PHP's Safe_Mode. Though PHP Secure_Mode(TM) will be embedded into this release, it will remain dormant until we are confident that PHP 4.3.1 survives the acid test of a full production environment.

This will also be the first stage of multiple incremental feature offerings that site owners have been requesting such as:
1) mbstring
2) XSLT
3) IMAP
4) libmcrypt (We are however still not confident with this modules stability)

The proposed order of events:
1) Upgrade to baseline 4.3.1, derived from 4.1.2 feature set
2) Monitor for stability and resolve any problems that may arise (~2 weeks)
3) Announce PHP Secure_Mode(TM) deployment plan
4) Deploy PHP Secure_Mode(TM) on a limited number of servers
5) Monitor for stability and resolve any problems that may arise (~1 week)
6) Deploy PHP Secure_Mode(TM) to remaining servers

We are excited to be offering this new updated PHP core, and are confident that this transition will go smoothly.

--
Terra
sysAdmin
FutureQuest

<EDIT: updated Maintenance window from Saturday to Sunday morning>

[kitchin]

Onward and upward!

Does this apply?

Version 4.2.0
22-Apr-2002

* ATTENTION!! register_globals defaults to 'off' now !!

[Terra]

register_globals will default to 'on' for us...

We will maintain this setting until it is deprecated in a future release of PHP... There has been some rumors that this will come about somewhere in PHP 5.x, but nothing conclusive...

Once the ability to change this item is gone, then it's pretty much game over and everyone will need to scramble to update their scripts to ensure compliance...

--
Terra
-- http://www.dailywav.com/0999/gameover.wav --
FutureQuest

[Terra]

Jeff has written a brief synopsis on potential impact with those running vBulletin...

Please view:
http://www.aota.net/forums/showthread.php?postid=85334#post85334

--
Terra
--Thanks Jeff--
FutureQuest

[hobbes]

Terra -

Is PHP 4.3.1 already loaded on the systems somewhere where it can be tested against critical scripts before going live?

Is there an expectation that Secure_Mode(TM) might break scripts? Or will scripts continue to work normally but we'll now have additional options (a la safe_mode) and functionality?

Thx.

[Terra]

Hobbes:
1) Yes, but in private testing on select accounts... The site owners currently running on the Secure_Mode(TM) beta currently have exposure to 4.3.0 - with a shift to 4.3.1 by tomorrow evening... There are no plans to expand the reach of 4.3.1, as I have to start breaking it down for production transition...

2) To lead off by saying that I would like to keep all Secure_Mode(TM) discussion held to a minimum until we are ready to officially announce it and open discussions...

To answer your questions:
2a) No - except in the case of file uploads where the umask is currently very restrictive... I am already working on relaxing this stipulation...
2b) Scripts should continue to work normally, but will reserve judgment on this until it has been in full production for a few weeks... There is no possible way we can possibly test every PHP script available, or have a clue as to what might be thrown at it by clever programmers doing clever things... I will say that our current targets have been reached...

--
Terra
--with great power comes great responsibility--
FutureQuest

[SneakyDave]

thanks for the update. I'm getting excited.

[Terra]

It appears that a couple domains have been able to segfault the new PHP 4.3.1 release... This issue did not show up in prerelease testing, so I need to see what features they are using that is triggering this...

Therefore, I am reverting back to the 4.1.2 release for now until I can remedy this problem...

Hopefully I will be able to take another shot at this tomorrow night... If I pin it down quickly, I might do a mid morning push as I'd rather get this out stable sooner than later...

--
Terra
sysAdmin
FutureQuest

[Terra]

I have pinned down the source of the problem, which led to aggressive C compiler optimizations used on PHP not agreeing with the new ZendOptimizer library...

I have made the necessary adjustments and have deployed it already to the HC01 server where I have been observing it... Incidentally, that was the location for one of the sites that triggered the problem to begin with...

I will update the Deployment Window for tonight in original post above...

--
Terra
sysAdmin
FutureQuest

[Terra]

Work is now underway to perform this upgrade... You may notice your web site being down during the upgrade procedure, however all other services will remain online for your use...

--
Terra
sysAdmin
FutureQuest

[Terra]

All Community Servers have been upgraded, with the exception of the SSL servers which will be rolled out once we are sure this is stable...

Let the bug hunting begin... %)

--
Terra
sysAdmin
FutureQuest

[Terra]

Ha - found one... :)

I will be pushing out a PHP maintenance release in a couple minutes... All Apaches will be taken offline for ~2 min...

--
Terra
sysAdmin
FutureQuest

[hobbes]

Terra -

I had a PHP script break but was able to fix it. Just bringing it up to make you aware in case it was an unintended config change:

A script under www/ included another PHP script outside www/, which in turn included a third script in its same dir:/www/script.php:
include "../script2.php";

/script2.php:
include "script3.php";
All worked mostly well (see recent support requests) prior to last night, but not this morning. The fix was to use the full path to script3.php in the script2.php include. Apparently ./ was part of the include path in the prior config?

[Terra]

I'm showing that the '.' entry is there:


directive local_value master_value
include_path .:/usr/local/lib/php .:/usr/local/lib/php


I just double checked, and '.' is perfectly valid as well as './'...

What does your include_path look like?

--
Terra
sysAdmin
FutureQuest

[Terra]

Hobbes:
Thinking about it, there is a good chance they updated/changed the way they do their VCWD (Virtual Current Working Directory)...

PHP 4.1.2, most likely changed it's VCWD when 'include()' was initiated, therefore any scripts you included from script2.php would have used the '.' include_path setting...

However, I feel that is non-intuitive behavior and most likely the PHP developers did as well... Just because you include a file, does not mean that the VCWD is changed as well then reset back to starting point once all the includes were done... It would be like leap frogging all around your directories pulling in files...

Personally, I feel the new PHP 4.3.1 behavior does the RightThing(TM), by asking specifically for the file you want to include... Includes that are relative to your VCWD are now much more predictable and can be more tightly controlled...

This does not prevent you from setting your own 'include_path' tailored to your site's needs... It is prudent, security wise, to do so anyway for your private PHP script library...

--
Terra
sysAdmin
FutureQuest

[dank]

I found with previous versions that it is necessary to specify full server paths when dealing with multiple includes from different directories, otherwise you tend to get incorrect paths. Not sure how hobbes' script was working previously, because that looks like exactly the sort of thing that didn't work for me...

Dan

[dank]

Uh-oh, big problem... Seems run-down.com has been down since last night, with several hundred identical entries in the log file starting at:

[13-Apr-2003 03:12:55] PHP Fatal error: Cannot redeclare unixdate() in /big/dom/xrun_down/cgi-bin/links/admin/Links/PHP/adodb/adodb.inc.php on line 1219

10 hours of downtime could make the natives restless...

Dan

[dank]

Follow-up: line 1216 in the above referenced file is:

function UnixDate($v)
Apparently named with capitalization to avoid naming overlap with the unixdate() function (of which I have never heard of). Does something in 4.3.1 ignore case in function names?

edit: On the other hand... That function appears to be declared 3 times within the file, which looks problematic. I'm assuming each declaration is conditional based on the portion of the script, but that will take a while to trace through its logic to figure out... Probably worth heading over to Gossamer Threads' forum and posting a warning of this potential problem...

Dan

[Terra]

Dank:
Good job, looks like you nailed this one rather quickly...

Looks like this came about in PHP 4.2.1, and from what I can find, the solution is to use proper coding techniques to ensure you are not re-declaring functions... PHP used to parse and run the script at the same time, but PHP4 is following the way of Perl by compiling to bytecode first and then executing... This is why the errors are found before the script has had a chance to execute...

The most prevalent solution is to reorganize your library functions and use either:
require_once();
or
include_once();

If you are using a 3rd Party script, then I would highly recommend that you see if any updates or patches are available...

--
Terra
sysAdmin
FutureQuest

[Terra]

Ok, it is confirmed that the 'Cannot redeclare' error is the intended behavior...

The PHP developers are basically saying to those bitten by this to "Fix your broken scripts, it is not a bug in PHP"....

http://bugs.php.net/search.php?search_for=redeclare&limit=10&order_by=&direction=ASC&cmd=display&status=All&bug_type=Any&php_os=&phpver=&assign=&bug_age=0

--
Terra
sysAdmin
FutureQuest

[dank]

Thanks for looking into it.

Good job, looks like you nailed this one rather quickly...
Small consolation, seeing as how I have a broken site as a result. :( I've submitted the bug to Gossamer Threads (http://www.gossamer-threads.com/perl/gforum/gforum.cgi?post=239746) (and through their support form) and will have to investigate it more on my own in the meantime...

Dan

[Terra]

Dan:
It appears that PHP has been very strict about this since 4.2.1... I'm shocked that no one else has brought it up over there as PHP 4.2.1 has been out for close to a year now... :(

--
Terra
sysAdmin
FutureQuest

[dank]

Surprising indeed. The user base and support of the optional PHP frontend has not been what I had expected or hoped for. Seems the lack of timely support has dwindled the user base to the point where I wouldn't be surprised to see development of the PHP frontend get dropped at some point... :(

With not many people using it, I suppose it's possible, although unlikely, that no one has run it under 4.2.1 or newer.

Dan

[dank]

The problem was solved by an updated ADOdb library, which I didn't realize is a 4th (?) party library used within G-T's Links SQL.

Dan

[Terra]

4th (?) party library
Yes, that is correct terminology and can be nested as deeply as dependencies demand...

Glad to see you got it all worked out, as this upgrade was a pretty good sized leap forward even though PHP 4.1.2 was on a stability island between some really crappy releases filled with problems...

:QTbeach:

--
Terra
sysAdmin
FutureQuest

[dank]

Who would the 2nd party be? %)

Dan

[Terra]

Who would the 2nd party be?
That would be you Dan... :P

--
Terra
--even if you do speak of yourself in the 3rd person--
FutureQuest

[dank]

Ok then, who's the 1st party? I had assumed that was me...

Dan

[Terra]

Ok then, who's the 1st party? I had assumed that was me
That depends on who was the initiator...

When a client requests support from FutureQuest for a script provided to the client by an outside vendor, then we refer to that as a 3rd party script... The 1st party has no direct contact with the 3rd party, and only the 2nd party has access to both....


FutureQuest <==> client <==> software vendor
1st Party <==> 2nd Party <==> 3rd Party


In your 4th party case using right-to-left due to HTML formatting:

|==>3rd Party<==> 2nd Party <==> 1st Party
|==>4th Party<======^

As there is the option that you either use the 3rd Party as a middle man to obtain the fix, or you deal directly with the 4th Party to obtain the updated components that are dependent upon the software supplied by the 3rd Party...

In neither case does the 1st Party become involved in the overall transaction, other than direct communication with the 2nd Party...

Hopefully that helps to explain the interrelationship between the client and vendors...

--
Terra
--it's as easy as a<==>b<==>c--
FutureQuest

[dank]

Did I already say %) ?

Seems to me the 1st party doesn't even deserve a place at the table, let alone top spot in the numerical pecking order.

Dan
- party of one?

[Terra]

Well, it's not really a 'pecking' order per se, but rather a positional order that shows the interrelationship more clearly...

In this case, being in 2nd place is King (or Queen) and trumps all...

:bounce:

--
Terra
--One day silver may be worth more than gold--
FutureQuest

[Renaud]

Was this upgrade really necessary? Now because of this upgrade my vBulletin Version 2.0.3 has problems:

vBulletin versions before 2.2.6 have problems with php 4.2.x . The two most common problems reported by users are:

admin/user.php - when modifying users you receive a SQL error #1064
admin/template.php - when modifying templates the list only shows your custom templates followed by a long list of "[change original]" links.

To fix these problems you need to upgrade to vBulletin 2.2.6

[Terra]

Renaud:
I'm sorry you have experienced problems with the upgrade...

The upgrade was necessary to bring forward exciting new features that have been often requested... If we had never upgraded, then we would still be running PHP3... It is simply the natural evolution of technology, and you can only get so far behind before the old technology becomes so deeply entrenched that it stymies forward mobility...

We tried to minimize the impact by allowing site owners enough time to research the transition to PHP 4.3.1 and how it would affect their site...

I would highly recommend that you visit the following post and join the QuestReport mailing list so that you can stay abreast of important Server News and Announcements...

http://www.aota.net/forums/showthread.php?s=&threadid=11557

Overall, it is definitely in your best interest to upgrade your version of vBulletin (2.0.3) as it has several security flaws...
http://www.securiteam.com/exploits/6B00C2060A.html
and
Item #4: http://groups.google.com/groups?hl=en&lr=lang_en&ie=UTF-8&oe=UTF-8&frame=right&th=5a5ba597798c9e0b&seekm=Pine.LNX.4.43.0205100832290.18396-100000%40mail.securityfocus.com#link1

--
Terra
--software in many ways plays just like an arms race--
FutureQuest

[TVB]

Renaud,

When Terra announced this upgrade, at first I had a near heart attack, because like you my vbulletin was seriously in need of upgrading. I was so scared that I was willing to pay someone to do it for me. Since no one offered their services, I figured they either thought I was really cheap and it wouldn't be worth it or they thought, "get a grip, it isn't that hard". Turns out, it was probably that because it was really easy and I have a really steep learning curve.

So, I upgraded to the latest release following the so-so directions from vbulletin, and it went without a problem. I made sure I got everything I figured I needed, and just dived in. Last night I held my breath, and lo and behold, everything was still there. I lost a minor hack but that was it. Since I saved the documentation for it when I did it, it will be easy to reinstall it.

It's really easy to upgrade...and as a bonus, your boards will be secure.

Betsy

[jeep]

Hi,

here's a problem we had with the upgrade which might catch others out.

Due to MSIE's weird handling of cookies we had a workaround in our cookie code:


$cookie_time = time()+2995200;
if ($msie) {
$cookie_time = date("l, d-M-y H:i:s", $cookie_time);
}
setcookie("id",$cookie_value, $cookie_time,"/","www.swingdigital.net",0);


This worked great. Until the PHP upgrade. The PHP boys have changed the handling of dates in the cookie code meaning that the workaround was no longer needed. With it still in place nobody could log into our sites if using MSIE - everyone else was ok.

That'll teach 'em! %)

Jason

[SneakyDave]

Post Nuke uses the same ADODB library, it it has the same problems. If you run Post Nuke (I don't think PHP Nuke uses this library), and get a blank page, this is probably the problem.

[13-Apr-2003 14:36:28] PHP Fatal error: Cannot redeclare unixdate() in xxxxxx/xxxxxx/adodb.inc.php on line 1477

Here's the link to adodb...
http://php.weblogs.com/ADODB

[hobbes]

Originally posted by Terra (long ago now it seems):
This does not prevent you from setting your own 'include_path' tailored to your site's needs... It is prudent, security wise, to do so anyway for your private PHP script library... What do you recommend as the best approach for doing this on FQ? Using ini_set or in .htaccess?

[SneakyDave]

Regarding the adodb fix for Post Nuke, I'd recommend upgrading the adodb add-on, but the authors say that you can also just remove the first declaration of the UnixDate function, and it should work. I verified that it does:

http://phplens.com/lens/lensforum/msgs.php?LeNs#LensBM_forummsg

These are the lines to delete:

function UnixDate($d)
{
return ADORecordSet::UnixDate($d);
}

[dank]

I can't find the thread now, but there was a discussion maybe a year ago about an oddity with PHP mail() being sent from certain scripts always showing as being from anonymous@seven.futurequest.net. Never did figure out what was causing that... Anyway, I just noticed that the 4.3.1 upgrade did away with that problem. Or if it wasn't that, at least some change that took place between 4/11 8:35am and 4/13 3:43pm.

Dan

[Terra]

Yes, PHP mail() lockdown is an ongoing project, and you are seeing some of the initial benefits... There is more work to do in this area, as Bruce is currently looking at embedding some solutions into our server queuing system instead of it being hacked into PHP itself...

Looking at my Apache/PHP configuration changelog, those new mail settings were put into place on: 2/11/2003 at 8:07:42 AM EST

You will also notice that we are pipelining directly into 'qmail-inject' now, instead of the depending on the 'sendmail' wrapper... This will add a small perfomance boost and provide more fine grained engineering/design control...

--
Terra
--it is nice feeling to see that someone finally noticed--
FutureQuest

[jeep]

array_search, which I use occassionally, returns FALSE on failure now whereas before 4.2.0 it returned NULL. Depending on how you check the return from array_search this may break your code.

More at http://www.php.net/manual/en/function.array-search.php

Jason

[Terra]

Thanks Jason, that is good to know!
:QTthumb:

--
Terra
--this one could be an elusive one to catch--
FutureQuest