Is this possible at all?

Jeepers · 01-15-2002, 02:34 PM

Say I have a script that goes looks up a certain webpage and captures sport's scores(ie. xml).

The problem is, my script would have to direct to this certain webpage for the info, and because of this, it may be slow at times depending on how fast that webpage loads.

So my question is,

Is it possible to have a file that automatically captures another webpage every hour or so?
This way, I can direct my script to that one file instead of directing it to the webpage.

thanx

bturner · 01-15-2002, 02:48 PM

You bet! Check this thread for how to use the Perl LWP module to copy a file from another site: http://www.aota.net/forums/showthrea...threadid=10115

Set up a script to grab the XML file, and then set up a cron job to run every hour. I know there's a FAQ for cron somewhere on the FQ site.

Good luck!
Bill

MikeGnitecki · 01-15-2002, 02:53 PM

The FQ page on crontab is at:

http://www.aota.net/Script_Installat.../cronhelp.php3

-Mike Gnitecki

**Kevin** · 01-15-2002, 03:12 PM

You can also use the wget command (in a cron job) to accomplish this.

Just make sure that your script checks that the copy it downloads is complete and valid before overwriting the previously cached copy.

*Rich* · 01-15-2002, 05:56 PM

You should also make sure that the information you are cacheing is not copyrighted or if it is, you have permission to do this if you don't own the "other" site.

Rich

Jeepers · 01-16-2002, 01:55 AM

Its a partner site.

I was thinking that its better to do an FTP push where they send upload us the xml file, but that's a lot more hassle.

Would it be good to have a cron job update every 15 minutes? or would this be too much....the file i'm getting is around 25k.

**Terra** · 01-16-2002, 04:38 AM

Every 15 is fine... The ones that cause me concern are those that fire off every minute...

--
Terra
sysAdmin
FutureQuest

Jeepers · 01-16-2002, 03:51 PM

Would this be correct to have the script run every 15 minutes?

0,15,30,45 * * * * /big/dom/xdomain/cgi-bin/scriptname.cgi

Not sure if I should put anything in the hour column.

**Arthur** · 01-16-2002, 03:58 PM

That's correct. You may also write it like this;
*/15 * * * * /big/dom/xdomain/cgi-bin/scriptname.cgi