Help setting correct path on custom software

[sheila]

I'm getting a 500 Internal server error on a Python script.
I've installed Python 2.0 in my own space, and apparently am having trouble with the path settings. Here is a snippet from my script error log:


%% [Sun Mar 18 14:52:25 2001] POST /cgi-bin/listsettings.py HTTP/1.0
%% 500 /big/dom/xthinkspot/cgi-bin/listsettings.py
%request
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, image/png, */*
Accept-Charset: iso-8859-1,*,utf-8
Accept-Encoding: gzip
Accept-Language: en,de-DE,fr,ru,ko,zh,zh-CN,zh-TW,cs
Connection: Keep-Alive
Content-length: 88
Content-type: application/x-www-form-urlencoded
Host: www.thinkspot.net (http://www.thinkspot.net)
Referer: http://www.thinkspot.net/sheila/computers/listManagement.html
User-Agent: Mozilla/4.61 [en] (Win98; U)

require-subscriberEmail=bob%40kserver.org&require-command=NODIGEST&Submit=Submit+Changes
%response
%stderr
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
'import site' failed; use -v for traceback
Traceback (most recent call last):
[nbsp][nbsp]File &quot;listsettings.py&quot;, line 5, in ?
[nbsp][nbsp][nbsp][nbsp]import cgi
ImportError: No module named cgi



I think that the whole difficulty is that it can't find the modules. I have Python-2.0 installed in my xdomain directory, which is one level above my $HOME directory.

I tried going to my .bash-profile and adding the following to my PATH line:
/big/dom/xthinkspot/Python-2.0

But this doesn't seem to be helping things.

Help?

I know that the cgi library is installed, because in a telnet session, if I telnet in and change to the directory
/big/dom/xthinkspot/Python-2.0
and type at the prompt
$python
and then type (in the python interpreter)
>>>import cgi

I get no error message. So, apparently all I need to do is help the script FIND the directory.

I do have a correct path at the top of my script, namely:

#!/big/dom/xthinkspot/Python-2.0/python

[sheila]

In seeking help elsewhere on this problem, I got the following advice from someone:


It looks like it has messed up its path, is the path on the webserver
the same as the path you installed it to??

Maybe try to do a shell wrapperscript that looks around, before
starting up python?

Could someone please translate that for me? I don't know how to write shell scripts. Or, how to make one &quot;look around&quot;.

(The above comments are based on my invoking the -v option and getting a more detailed log report, which I haven't shared here because it is very long. But if someone really needs to see it, just shout...)

[sheila]

OK, I seem to have found some sort of work-around with the paths problem for Python-2.0, thanks to D-Man (a regular on the Python Tutor mailing list).

(Additional boo-hoo-ing and background on this problem can be found here: http://www.aota.net/ubb/Forum3/HTML/001565-1.html
http://mail.python.org/pipermail/tutor/2001-March/004309.html
http://mail.python.org/pipermail/tutor/2001-March/004323.html )

Diagnosing the problem involved the following steps:

(1) in a telnet session, invoke my private installed version of the Python-2.0 interpreter, and determine the system paths, as follows:


Python 2.0 (#1, Jan 15 2001, 01:09:04)
[GCC 2.7.2.3] on linux2
Type &quot;copyright&quot;, &quot;credits&quot; or &quot;license&quot; for more information.
>>> import sys
>>> for path in sys.path:
...[nbsp][nbsp][nbsp][nbsp] print path
...

/big/dom/xthinkspot/Python-2.0/Lib
/big/dom/xthinkspot/Python-2.0/Lib/plat-linux2
/big/dom/xthinkspot/Python-2.0/Lib/lib-tk
/big/dom/xthinkspot/Python-2.0/Modules
>>> import cgi
>>> print sys.modules[&quot;cgi&quot;]
<module 'cgi' from '/big/dom/xthinkspot/Python-2.0/Lib/cgi.pyc'>
>>>[nbsp][nbsp]


(2) Then, write a cgi script which prints out the system paths under which it is running. Although I could not import modules in general in my Python-2.0 running as a cgi script, for some reason the sys module is always available and can be imported. So, using the following script:


#! /big/dom/xthinkspot/Python-2.0/python

import sys
print &quot;Content-type: text/plain\n\n&quot;

print sys.version

print
print
for path in sys.path:
[nbsp][nbsp][nbsp][nbsp][nbsp]print path

print
print &quot;Those are the paths&quot;


I get this output:


2.0 (#1, Jan 15 2001, 01:09:04)
[GCC 2.7.2.3]

/usr/local/lib/python2.0/
/usr/local/lib/python2.0/plat-linux2
/usr/local/lib/python2.0/lib-tk
/usr/local/lib/lib-dynload

Those are the paths


Conclusion: Apparently, when a script runs called by cgi, rather than at a command-line prompt, it is not finding the correct paths, despite the shebang-path at the top of the script.

Perhaps there is some way that I can fix my installation of Python-2.0 in order to set it so that it knows the correct paths when it is invoked. In the meantime, the following work-around appears to solve the problems with the paths:

At the beginning of each script, under the shebang line and before importing any other modules, first import the sys module and fix the paths, something like this:


import sys

sys.path.insert( 0 , &quot;/big/dom/xthinkspot/Python-2.0/Lib&quot; )
sys.path.insert( 0, &quot;/big/dom/xthinkspot/Python-2.0&quot; )
sys.path.insert( 1, &quot;/big/dom/xthinkspot/Python-2.0/Lib/plat-linux2&quot;)
sys.path.insert( 1, &quot;/big/dom/xthinkspot/Python-2.0/Lib/lib-tk&quot;)
sys.path.insert( 1, &quot;/big/dom/xthinkspot/Python-2.0/Modules&quot; )

for path in sys.path:
[nbsp][nbsp][nbsp][nbsp][nbsp]if path.startswith(&quot;/usr/local/lib&quot;):
[nbsp][nbsp][nbsp][nbsp][nbsp][nbsp][nbsp][nbsp][nbsp][nbsp]k = sys.path.remove(path)


I am, of course, always interested in comments, remarks, and insights as others may have on this situation.

[sheila]

OK, this problem has been brought to resolution.

I didn't set the install paths properly when I installed the package.

More details available here:
http://mail.python.org/pipermail/tutor/2001-March/004344.html

Public apology to Terra here:
http://www.aota.net/ubb/Forum3/HTML/001564-1.html

[sheila]

I just want to say, this had been a great thread. I've thoroughly enjoyed the discussion with myself. I am rather witty and clever.

[Terra]

I think the main problem is that no one really uses Python that much in a production environment on the FutureQuest servers...[nbsp][nbsp]This would sort of place you on an island with seeking help with development issues from the FutureQuest Community...

Myself included...[nbsp][nbsp]Most of my programming is system level in nature, and Python just does not fit very well in to that scheme of things...[nbsp][nbsp]I can write extremely powerful and idiomatic perl code in less time and accomplish far more than dealing with an OOP style language...

Personally, I like Perl's expressive power which gives me the freedom to create wonderful art or the darker side of shooting myself in the foot with it...

An oxymoron, but I think that Python is just too clean and pristine for the grungy side of system/administrative coding...[nbsp][nbsp]I've found that I can do almost 100% of chores in combination of Bash/Awk/Sed/egrep/C/Perl...[nbsp][nbsp]Also consider that a lot of the programs I write are throw away in nature designed to do a very specific run-once job...[nbsp][nbsp]These are the tools that carry forward the evolution of the server's mannerisms...

In short, I just haven't felt the need yet to master Python...[nbsp][nbsp]That is why I can't help you as deeply as I could with the other languages, leaving you to feel all alone...

One day, when I have some free time, I may take a dive into Python and see it's capabilities in a more panoramic view...[nbsp][nbsp]Until them, I'm pretty much useless to you...

--
Terra
--Always tries to find the right hammer for the wrong nail--
FutureQuest

<EDIT: Trying to not create a language Holy War>

[This message has been edited by ccTech (edited 03-20-01@01:56 am)]

[sheila]

OK, first a humorous post...

High School Perl Test:
http://www.bbspot.com/News/2001/03/perl_test.html

To follow...more serious comments...

[sheila]

Python is very quick to learn for someone who already programs. You could probably sit down for about two hours one afternoon with the Tutorial:
http://www.python.org/doc/current/tut/tut.html
(caution: this tutorial assumes knowledge of programming in other languages.)

Python has many tools for automating system tasks:
general operating system services:
http://www.python.org/doc/current/lib/allos.html

Unix specific services:
http://www.python.org/doc/current/lib/unix.html

Internet protocols:
http://www.python.org/doc/current/lib/internet.html

Internet data handling:
http://www.python.org/doc/current/lib/netdata.html

String services, including a regular expression module:
http://www.python.org/doc/current/lib/strings.html

and much more. But these are all included in the regular install. You don't have to go out to CPAN or something like that, and find and install the modules you need. They are part of the regular distribution. And very cross-platform. (Obviously, not something like the Unix specific services.)

For me, I was able to mess around with Python and send out e-mails using the smtplib in a single day. Wow. Perl: I bought the O'Reilly book, and read the first two or three chapters and after two days of looking at it, couldn't imagine doing anything useful in it any time soon. Python:[nbsp][nbsp]a couple of days working in the language and I'm bursting with all kinds of ideas for projects I'd like to try (eventually).

One thing about Python: It is scalable. You can write small, throw-away scripts in it. But you can also write full-blown apps. Take a look at this app, written in Python:
http://boa-constructor.sourceforge.net/

Plus, there are several cross-platform GUI bindings, for developing GUI apps (something I hope to get into in the not too distant future):

Tkinter
http://starbase.neosoft.com/~claird/comp.lang.python/python_GUI.html#Tkinter

PyQT
http://starbase.neosoft.com/~claird/comp.lang.python/python_GUI.html#PythonQt

wxPython
http://www.wxpython.org/

are just a few of the available GUI toolkits for working with Python.

What finally turned me on to Python, even though I'd heard about it before, was this personal testimonial I read on a personal web page that I happened across by chance (I was looking for links on LaTeX):
http://pantheon.yale.edu/~pmm34/programming.html

After reading that, I decided I really had to spend at least a few hours looking at it. And after a few hours, I felt like I could already accomplish something meaningful in the language.

More Python Evangelism:
http://www.aota.net/ubb/Forum3/HTML/001567-1.html

[sheila]

One last pointer...

quoted from http://linuxworld.com/linuxworld/expo/lw-python.html, an article which expounds on Python, and compares it to other programming languages


For all its strengths, Python has been overlooked in the floodlight glare surrounding Perl, which has flourished of late largely because so many Unix hackers use it to write CGI scripts for Web servers. But don't accept the impression that Python never gets used.

Several major Web sites, including InfoSeek and Four11, are supportive users. Other corporations are closet users, visible only when they goof in their code writing and get debugging &quot;traceback&quot; messages from their scripts. These tracebacks are displayed in your Web browser (which, by the way, brings up the positive point that Python checks segment boundaries and the like very closely, and Van Rossum guarantees that Python never just bombs -- you always get a traceback). The most recent traceback &quot;outing&quot; I've heard of was at the American Greetings site

Much more at the article linked above.

[sheila]

<EDIT: Trying to not create a language Holy War>
I really wasn't starting a Holy War. I just thought, it is true (as you later wrote), that Python is not that much used here at FQ. Certainly nothing wrong with talking it up, right? It is installed on all the servers. Maybe people are just not aware of it.

Anyhow, I've enjoyed using it. For anyone else, I would recommend it, but, of course, YMMV.