Perl Question (Del?)...

[Justin]

I know there is a way... but I don't know how. I want to have a script send some html to the browser, wait a few, send some more, wait a few more, etc. How can this be done? Sort of like a progress indicator for something that might take a few minutes to complete. You would see:

Please wait...

a few seconds later:

10% complete...

and so on, until it is done. I tried just printing the html a chunk at a time, using the sleep() function to simulate that something is happening, but I only see "Website found, waiting for reply...". And it's not in a table. I know this can be done, I just don't remember how.

What I'm afraid of is that the browser will time out after a minute or so, and although the operation is still in progress, the user will not know what is going on. I don't want to go crazy with refresh tags like the UBB does when updating threads, but if that's the only way...

A URL or simple answer or whatever would be awesome http://www.aota.net/ubb/smile.gif


------------------
Justin Nelson
FutureQuest Tech Support

[Stephen]

Justin,

I think what you want to do is unbuffer the output by placing the following line at the top of your program (presuming it's Perl you're using):

$|=1; # unbuffer output so it appears in right order

I have to admit that I've never tried to output html piecewise, but I THINK this is how you do it.

[Del]

Hi Justin

I know exactly what you're talking about, but I've never really played with it myself. It sounds like what you're after is 'server push', which unfortunately is only available to Netscape users (3+ I think). Server Push causes the browser to keep an open connection with the script running on the server. As you can guess, this can really rack up the server load...

You're halfway there using the sleep() function, server push totally relies on it. What you're missing is the special content-type header (which I can't find right now! ggrrrrr 9 perl books sitting in front of me, for the life of me I can't remember where I read that blippin header. Found it, it's "multipart/x-mixed-replace;boundary=$boundary", where you've predefined $boundary as some random string that's guaranteed not to appear anywhere in the text you're outputting. Anyway, it also requires that you generate your own complete headers, otherwise the browser would see it as an error. This means you have to rename the script as nph-scriptname.pl (the nph- at the beginning means something like non-parsed-headers). This also means that in addition to sending the content-type header, you also have to send "HTTP/1.0 200 OK". All in all it's a big hairy thing http://www.aota.net/ubb/smile.gif

Stephen is correct, you do have to flush the buffer. However, it's a lot deeper than that if you get in to the realtime serverpush stuff. The easiest way is indeed to go with a meta refresh tag. If you want to look deeper into the server push stuff though, here's some URLs (I know this answer was longer than you wanted, sorry http://www.aota.net/ubb/smile.gif )

http://www.extropia.com/scripts/animation.html

http://www.cgi-resources.com/Programs_and_Scripts/Perl/Animations/ (these mainly deal with images instead of text)

Del

[Justin]

Actually, I appreciate the detailed explanation. I didn't know there was so much to that though. I will probably end up doing the refresh thing anyway, as the whole point of the idea was to ease the server load. I figured doing it a chunk at a time, showing the output to the browser, would keep from trying to take up the whole server for a long period of time.

I realized one other advantage to the refresh tag - the operation can be canceled by clicking stop. I don't think you can do that from the client side once the script has started it's thing otherwise. Only this way the browser is required to remain open, at that URL, until it's completed, where the other way you could conceivably close the browser (I'm probably wrong there too http://www.aota.net/ubb/smile.gif)

Oh, well, I need some sleep http://www.aota.net/ubb/smile.gif Thanks for the replies. I probably will check out those URL's tomorrow (today) just for the heck of it.


------------------
Justin Nelson
FutureQuest Tech Support

[jenili]

You don't have to use server push, actually, unless you're sending multiple documents. If you're just sending chunks of content as the results of a CGI, you need non-parsed headers but you don't need server push (with the multipart/mixed content header and the boundaries). And the browser won't time out, but doesn't have to wait if the user decides to cancel out (but in this event, the CGI will halt execution).

Here's what you'd need to do, Justin. It's actually pretty minor.

1. Put Stephen's line
$| = 1;
in the perl script, near the top, to keep perl from buffering its output.

2. Prepend Del's line
HTTP/1.0 200 OK
to your headers. No blank line afterward. This is the HTTP protocol/status header. So your headers would look something like this:
HTTP/1.0 200 OK
Content-type:text/html

<html>....

3. Name the script
nph-script.cgi
(where script is whatever you want, of course).

Normally, when a Web server receives a call for a CGI, the Web server forks the CGI and waits on its output and termination. There's some error-checking there: The Web server looks to make sure the CGI is returning HTTP headers as required, and if the CGI doesn't respond in a timely fashion the Web server assumes it bombed. If all goes well, the Web server sends the browser an HTTP status header and then sends it the CGI's output (actually, it sends the status header and some content in any event, since the result of a bad CGI is gonna be HTTP/1.0 500 Server Error and an error message in HTML).

When you name the CGI nph-something (#3 above), you're telling the Web server to get out of the way and let your CGI talk directly to the browser. Then your CGI has to do the server's job WRT the HTTP status header (#2 above), and in the case of a time-sensitive thing you also have to make sure its output isn't being buffered by the interpreter (#1 above).

That's all you need to do if you're just sending time-delayed chunks of the same type of content. If you were sending multiple types of content -- say, a PDF form and HTML instructions for filling it out -- you'd have to use the multipart/mixed header and boundaries. But in your case, you don't, unless you want each time-delayed chunk to completely overwrite the chunk that preceded it.

Ooooooh... If this doesn't work for you, you might need to check Apache docs to see what hoops you have to jump through to get Apache to recognize an nph script as such. It might be different from naming it nph-whatever. I've done nph's on Netscape Enterprise, but not on Apache. I'm sure the functionality is there, though.


------------------

jeni


[This message has been edited by jenili (edited 03-14-99).]

[Del]

I've run nph's on Apache before, and it handled em fine. I think it's built to understand them by default, but not totally sure.


But in your case, you don't, unless you want each time-delayed chunk to completely overwrite the chunk that preceded it.


That's what I interpreted Justin's request as (if I worded that sentence properly, my grammar stinks). It sounded like he wanted it to say "10%" when 10% done, the replace that with "20%" when 20% was done, etc... therefore needing the mixed-replace header. Jeni's way (skipping the mixed-replace header) would work dandily if you wanted (for example) a growing status bar or something though, since it wouldn't be replacing what was already there, but appending to it instead (eg tack a new # onto the end of a line of ###'s every few seconds or something).

Thanks Jeni for reminding me of the way not to use server push, I'd forgotten all about it http://www.aota.net/ubb/smile.gif

Del

[jenili]

Del... Everything was there in your message and Stephen's already http://www.aota.net/ubb/smile.gif (that smiley would be shades if I could find the right string to type :p

[This message has been edited by jenili (edited 03-14-99).]

[Terra]

From what I understand of the Apache 1.3.x series, nph- is a thing of the past... I believe that Apache handles all CGI scripts like nph- script, but also pre-pending with the header tag...

The most important aspect is to unbuffer your CGI script '$|=1;'...

Off the top of my head, I am not sure that you can create a page that can keep adding '#' '##' '###' and so on, because the browser rendering engine must interpret the page if it's in HTML, and displays what is between the <HTML></html> tags...

Now if this was 'Content-Type: text/plain\n\n', then you should be able to do what you are looking for... I did some nph-blah.pl scripts awhile back, but it had a bad tendency write the new output to a new line...
#
##
###
####

All I can say at this point, is that Tinkering will rule the spotlight here... Try different 'tiny one-shot' scripts, then when you get the desired affect - roll it into a function within the primary script and call it whenever needed...

One other *off-the-wall* sidenote - see if you can make 'KeepAlive' to be your friend on this, since we are fully based on HTTP/1.1 protocol... http://www.aota.net/ubb/wink.gif

Hope these ramblings somehow help... This is pretty much the overview that I would start with to zone in on the solution...

PS: nph- scripts are pretty much dead now...

--
Terra
--What do you want to push around today?--
FutureQuest

[Justin]

I probably should have been a little more specific. Jenili is right, though - I don't need to replace what you are seeing in the browser, just add to it. I was thinking something like the progress indicator (####) - or have a 1 or 2 pixel wide gif and keep adding them http://www.aota.net/ubb/smile.gif Maybe more like 10 pixels. Then you don't have to worry about it writing to a new line each time - as long as there's no
it will continue in line.

But either way, that's what I'll have to do, just play around with it until it works.


------------------
Justin Nelson
FutureQuest Tech Support

[Stephen]

calling all CGI gurus...

i finally have a need to generate nph pages, but the implementation escapes me. i dug up this thread and put the theory to practice, but no go. i've tried the following script locally and on FQ, both times under Apache--but the server seems to store the page and spit it out in one chunk at the end, instead of piece-wise.

does anyone know how to tweak the script to get it to print out one line per second? is there something that needs to be done to tell Apache to recognize it as an nph? i named it nph-clock.cgi, but i don't think that's helped any.


#!/usr/local/bin/perl

$| = 1;

my $CRLF = &quot;\n&quot;;[nbsp][nbsp]# tried &quot;\r\n&quot;, &quot;\015\012&quot; too

my @headers;

push[nbsp][nbsp][nbsp][nbsp][nbsp]@headers,
[nbsp][nbsp][nbsp][nbsp][nbsp]&quot;HTTP/1.0 200 OK&quot;,
[nbsp][nbsp][nbsp][nbsp][nbsp]&quot;Content-type: text/plain&quot;;

my $headers[nbsp][nbsp]= join $CRLF, @headers, $CRLF;

print $headers;

print qq|
header check:
-------------
$headers
-------------
|;
my $i = 1;

while ($i <= 5) {

[nbsp][nbsp][nbsp][nbsp][nbsp]print &quot;time[$i] = &quot;.time.&quot;\n\n&quot;;
[nbsp][nbsp][nbsp][nbsp][nbsp]++$i;
[nbsp][nbsp][nbsp][nbsp][nbsp]sleep 1;
}

[Stephen]

resolved.

apparently it's not the code so much as the non-buffering. at some level it isn't happening. after finding a suggestion on some newsgroup that you could &quot;fill the buffer&quot; by adding 2048 space characters to the end of each print statement i added the filler and voila, staggered output...