Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I want the ability to schedule commands to be run in a FIFO queue. I DON'T want them to be run at a specified time in the future as would be the case with the "at" command. I want them to start running now, but not simultaneously. The next scheduled command in the queue should be run only after the first command finishes executing. Alternatively, it would be nice if I could specify a maximum number of commands from the queue that could be run simultaneously; for example if the maximum number of simultaneous commands is 2, then only at most 2 commands scheduled in the queue would be taken from the queue in a FIFO manner to be executed, the next command in the remaining queue being started only when one of the currently 2 running commands finishes.

I've heard task-spooler could do something like this, but this package doesn't appear to be well supported/tested and is not in the Ubuntu standard repositories (Ubuntu being what I'm using). If that's the best alternative then let me know and I'll use task-spooler, otherwise, I'm interested to find out what's the best, easiest, most tested, bug-free, canonical way to do such a thing with bash.

UPDATE:

Simple solutions like ; or && from bash do not work. I need to schedule these commands from an external program, when an event occurs. I just don't want to have hundreds of instances of my command running simultaneously, hence the need for a queue. There's an external program that will trigger events where I can run my own commands. I want to handle ALL triggered events, I don't want to miss any event, but I also don't want my system to crash, so that's why I want a queue to handle my commands triggered from the external program.

share|improve this question

5 Answers 5

Apart from dedicated queuing systems (like the Sun Grid Engine) which you can also use locally on one machine and which offer dozens of possibilities, you can use something like

 command1 && command2 && command3

which is the other extreme -- a very simple approach. The latter neither does provide multiple simultaneous processes nor gradually filling of the "queue".

share|improve this answer
    
Updated my question: Simple solutions like ; or && from bash do not work. I need to schedule these commands from an external program, when an event occurs. I just don't want to have hundreds of instances of my command running simultaneously, hence the need for a queue. –  Andrei Apr 10 '13 at 15:44

Use ;

For example:
ls ; touch test ; ls

That will list the directory. Only after ls has run it will run touch test which will create a file named test. And only after that has finished it will run the next command. (In this case another ls which will show the old contents and the newly created file).

Similar commands are || and &&.

; will always run the next command.

&& will only run the next command it the first returned success.
Example: rm -rf *.mp3 && echo "Success! All MP3s deleted!"

|| will only run the next command if the first command returned a failure (non-zero) return value. Example: rm -rf *.mp3 || echo "Error! Some files could not be deleted! Check permissions!"

If you want to run a command in the background, append an ampersand (&).
Example:
make bzimage &
mp3blaster sound.mp3
make mytestsoftware ; ls ; firefox ; make clean

Will run two commands int he background (in this case a kernel build which will take some time and a program to play some music). And in the foregrounds it runs another compile job and, once that is finished ls, firefox and a make clean (all sequentially)

For more details, see man bash


[Edit after comment]

in pseudo code, something like this?


Program run_queue:

While(true)
{
   Wait_for_a_signal();

   While( queue not empty )
   {
       run next command from the queue.
       remove this command from the queue.
       // If commands where added to the queue during execution then
       // the queue is not empty, keep processing them all.
   }
   // Queue is now empty, returning to wait_for_a_signal
}
// 
// Wait forever on commands and add them to a queue
// Signal run_quueu when something gets added.
//
program add_to_queue()
{
   While(true)
   {
       Wait_for_event();
       Append command to queue
       signal run_queue
   }    
}
share|improve this answer
    
As updated in my question: Simple solutions like ; or && from bash do not work. I need to schedule these commands from an external program, when an event occurs. I just don't want to have hundreds of instances of my command running simultaneously, hence the need for a queue. –  Andrei Apr 10 '13 at 15:41
    
Ah. Yes, that changes it a bit. :-) How do you detect the event? Someone clicking on a button (real of web-based) ? Is a wrapper script an option? ( if(button_pressed && !script_running) run_my_script. script: set the flag script running, run command, clear flag ) –  Hennes Apr 10 '13 at 15:50
    
The program is external and will trigger an event where I can add my own command, and basically I want to handle ALL triggered events, I don't want to miss any event, just that I also don't want my system to crash, so that's why I want a queue to handle my commands triggered by the external program. –  Andrei Apr 10 '13 at 16:09
    
Added some pseudo code. add_to_queue() can be modified for multiple queus. But... If you are going to spent that much effort then something pre-build might be better. –  Hennes Apr 10 '13 at 16:52
    
Well I'm a programmer myself so I could write my own solution; perhaps my question was misleading. Since I want to run this on a Pogoplug, I had hoped there was something similar to the "at" command that would have done what I wanted. The only "bash" part of the question referred to my hopes that such a solution could be found as close as possible to the shell (so as to be cheap resource-wise, reliable/tested and readily available on almost everything Linux). –  Andrei Apr 11 '13 at 11:18

The easiest way would be to simply run the commands sequentially:

cmd1; cmd2; cmd3; cmdN

If you want the next command to run only if the previous command exited successfully, use &&:

cmd1 && cmd2 && cmd3 && cmdN

That is the only bash native way I know of doing what you want. If you need job control (setting a number of parallel jobs etc), you could try installing a queue manager such as TORQUE but that seems like overkill if all you want to do is launch jobs sequentially.

share|improve this answer
    
Updated my question: Simple solutions like ; or && from bash do not work. I need to schedule these commands from an external program, when an event occurs. I just don't want to have hundreds of instances of my command running simultaneously, hence the need for a queue. –  Andrei Apr 10 '13 at 15:41
    
Have a look at TORQUE then, also you may be able to script something. Perhaps a combination of pgrep foo | wc to count running processes and act accordingly. –  terdon Apr 10 '13 at 16:17

You are looking for at's twin brother: batch. It uses the same daemon but instead of scheduling a specific time, the jobs are queued and will be run whenever the system load average is low.

share|improve this answer
    
But I want to have exact control over this, I don't want batch to decide for me. task-spooler seems like it might do the job, but it's a program that's a bit obscure. –  Andrei Apr 10 '13 at 15:45
up vote 3 down vote accepted

Task Spooler:

http://vicerveza.homeunix.net/~viric/soft/ts/

https://launchpad.net/ubuntu/+source/task-spooler/0.7.3-1

Does the trick very well. Hopefully it will be included in Ubuntu's package repos.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.