Initiating cronjobs from PHP?

[okidom]

Hi all,

I have a requirement for initiating tasks from my site that requires long processing times, i.e. uploading/downloading some files to remote nodes. And, to avoid timeouts and hence to be able to provide instant responses to the user (user does not need to get the result right away), I was thinking about handling this with cronjobs, i.e. when I get this request, I will insert an entry into the crontab with exact time information (which is like 2 minutes from current time) and return the response to the user right away, and the task will be invoked right away (again 2 minutes from that time) by the cron daemon...

Do you think this is a viable approach? Can you guys recommend a better way of handling this?

Thanks,

[Wassercrats]

Sounds like a nice, sneaky way to get around the script timeout of under one minute (20 or 30 seconds, last I heard). I think there's still a two minute timeout for cronjobs, so it's not unlimited. If that's not long enough, you need a different solution.

[sheila]

Keep in mind that the processor time limits are seconds of CPU time, not seconds of wall-clock time. There is a huge difference. A process that uses 30 seconds of CPU time may appear to actually take much longer to complete, since the CPU runs many different processes, swapping out time to different tasks as it goes...

see
Difference between CPU time and wall time (http://service.futurequest.net/index.php?_a=knowledgebase&_j=questiondetails&_i=532&nav=+%26gt%3B+%3Ca+href%3D%5C%27index.php%3F_a%3Dknowledgebase%26 _j%3Dsubcat%26_i%3D39%5C%27%3EScripting+and+Coding%3C%2Fa%3E+%26g t%3B+%3Ca+href%3D%5C%27index.php%3F_a%3Dknowledgebase%26_j%3Dsubc at%26_i%3D60%5C%27%3ECGI%2FPerl%3C%2Fa%3E)
for more info.

CGI CPU time limit is 30 seconds. (http://service.futurequest.net/index.php?_a=knowledgebase&_j=questiondetails&_i=179&nav=+%26gt%3B+%3Ca+href%3D%27index.php%3F_a%3Dknowledgebase%26_j% 3Dsubcat%26_i%3D39%27%3EScripting+and+Coding%3C%2Fa%3E+%26gt%3B+% 3Ca+href%3D%27index.php%3F_a%3Dknowledgebase%26_j%3Dsubcat%26_i%3 D60%27%3ECGI%2FPerl%3C%2Fa%3E)

I don't see anything sneaky about what he is proposing to do. Sounds like a good way to return pages immediately to the site visitor, since they do not require the results of the process immediately. :yeah:

[Wassercrats]

How would CPU time be calculated if the task is split between two processors, making it faster? If it currently takes 30 seconds of CPU time and Terra starts doing parallel processing, or whatever it's called, cutting the processing time in half, would you count 15x2 or just 15? Maybe the total microprocessor calculations is what should be limited.

[Wassercrats]

Maybe the total microprocessor calculations is what should be limited.By the way, how would you count those if you wanted to?

[Bruce]

How would CPU time be calculated if the task is split between two processors, making it faster?A single process is limited to a single CPU. If you created multiple processes to split the time across multiple CPUs, the amount of CPU time consumed would still be calculated the same as if there was a single CPU.
Maybe the total microprocessor calculations is what should be limited.
By the way, how would you count those if you wanted to?On x86 CPUs, you can use the "rdtsc" assembly instruction to read a 64-bit count of the number of CPU clock ticks since boot. I believe there are also counters available that detail how many instructions have been issued, but they are harder to get at (and may be protected opcodes).

[okidom]

First of all, thanks for prompt replies. And, Sheila, thanks for NOT misunderstanding this! I really am not trying to do something sneaky. In fact, trying to put the least required load on the CPU...

My main worry/question was in the line of the underlying concept actually. I could not think of something better to take care of these jobs, and cronjob is pretty much the only solution that I can come up with. But, security part of this solution scares me a bit. Although I would require authentication for hitting the URL to setup this task, still it is a bit risky, and I was asking if there is any better way of solving it.

Thanks..

[dank]

A cronjob can run a PHP script, but I'm drawing a blank on how you would get a PHP script to run a cronjob, which as far as I'm aware, is purely a scheduled event. Command line actions, maybe?

Dan

[Joseph]

A cronjob can run a PHP script, but I'm drawing a blank on how you would get a PHP script to run a cronjob, which as far as I'm aware, is purely a scheduled event. Command line actions, maybe?

Dan

PHP can run command line... commands using either exec() (http://us2.php.net/manual/en/function.exec.php) or system() (http://us2.php.net/manual/en/function.system.php) functions. Using one of these functions, you can set a cron to run another script in a couple minutes or instantly. :wink:

[okidom]

That I took care of. What I do is: when the request for a task comes in, I lock a file (for avoiding concurrent access), clean out old/expired entries in cron, insert the new entry, and unlock the file. And, as told before, I am using system() to update cron.

[sheila]

Since you are only executing these commands a single time (as I understand it?), I wonder whether you might want to look into using the at command, rather than cron. To use cron you have to edit and save etc... your cron file. You could avoid this by using at.

See the man page for at.
But, security part of this solution scares me a bit. Although I would require authentication for hitting the URL to setup this task, still it is a bit risky,
I assume you are using end-user-entered data to determine the file or command that gets executed (since you bring up the security risk concern...?). I'm just trying to follow the concern here.

I would guess, that you are therefore validating all the user-entered data (anything that is read in and not hard coded in the script). I don't see how you avoid this security concern, regardless of whether you use cron or other mechanisms to execute the script in question.

[okidom]

Hi Sheila,

I might be a bit paranoid here, but what I am scared of is a DoS type of attack. Not that it is very likely, but if someone exploits this URL somehow, they can repeatedly call the URL and insert lots of entries into cronjob which would put the server in a bad situation.

And as for the parameters, I don't pass anything from the request to the process to avoid more security hazards. Everytime this process is invoked, it checks the mysql db to find out if it needs to do something, and does it if so.

What I am thinking as a solution is to check previous jobs in the cron to find out when this script was last executed, schedule accordingly to avoid too frequent invocations of this process.

Thanks for the suggestions...

[Bruce]

I might be a bit paranoid here, but what I am scared of is a DoS type of attack. Not that it is very likely, but if someone exploits this URL somehow, they can repeatedly call the URL and insert lots of entries into cronjob which would put the server in a bad situation.A good way to avoid this is to set up the cron job to run frequently and control it with two files. If the first one is present, the job will not run (locking). Otherwise if the second one is present, it creates the first one, runs, and deletes both of them. That way there will never be more than one job running at once.

And as for the parameters, I don't pass anything from the request to the process to avoid more security hazards.Good plan. Never trust user-supplied input until it is 100% validated.

[okidom]

I did not want to run this frequently, because, as you know no matter how light the process is, even the burden of spawning a process is considerable. Besides, as I mentioned, I don't know when this thing will happen, it might happen 5 times a day, or once in 5 days...

What I did was something similar to what Bruce said, basically this PHP gets an exclusive lock on a lock file, does its thing with cron, then unlocks the file. Furthermore, it does not insert anything if there is already a job within a 30 minute window, i.e. if there is a job scheduled within next 30 minutes.

Thanks for all your help, suggestions and support. It is very much appreciated.