Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

do you know if there is a simple way to run FOR command in batch file on multiple threads? What's the point of having 4 cores if I can't run my tasks in 4 parallel threads?

For example, if I am optimizing PNGs with PNGOUT, the command I would use is

for %i in (*.png) do pngout "%i"

But this is highly paralellizable task in which the sub-tasks do not depend on each other at all.

To run this in 4 'queues' I'd write something like

for -thread 4 %i in (*.png) do pngout "%i"

Do I need to write my own for-like app that would be able to do this or is there available free solution?

share|improve this question
add comment

2 Answers

up vote 6 down vote accepted

I have written a batch file that executes only a maximum number of commands a while ago on Stack Overflow: Parallel execution of shell processes:

@echo off
for /l %%i in (1,1,20) do call :loop %%i
goto :eof

:loop
call :checkinstances
if %INSTANCES% LSS 5 (
    rem just a dummy program that waits instead of doing useful stuff
    rem but suffices for now
    echo Starting processing instance for %1
    start /min wait.exe 5 sec
    goto :eof
)
rem wait a second, can be adjusted with -w (-n 2 because the first ping returns immediately;
rem otherwise just use an address that's unused and -n 1)
echo Waiting for instances to close ...
ping -n 2 ::1 >nul 2>&1
rem jump back to see whether we can spawn a new process now
goto loop
goto :eof

:checkinstances
rem this could probably be done better. But INSTANCES should contain the number of running instances afterwards.
for /f "usebackq" %%t in (`tasklist /fo csv /fi "imagename eq wait.exe"^|wc -l`) do set INSTANCES=%%t
goto :eof

It spawns a maximum of four new processes that execute in parallel and minimized. Wait time needs to be adjusted probably, depending on how much each process does and how long it is running. You probably also need to adjust the process name for which tasklist is looking if you're doing something else.

There is no way to properly count the processes that are spawned by this batch, though. One way would be to create a random number at the start of the batch (%RANDOM%) and create a helper batch that does the processing (or spawns the processing program) but which can set its window title to a parameter:

@echo off
title %1
"%2" "%3"

This would be a simple batch that sets its title to the first parameter and then runs the second parameter with the third as argument. You can then filter in tasklist by selecting only processes with the specified window title (tasklist /fi "windowtitle eq ..."). This should work fairly reliable and prevents too many false positives. Searching for cmd.exe would be a bad idea if you still have some instances running, as that limits your pool of worker processes.

You can use %NUMBER_OF_PROCESSORS% to create a sensible default of how many instances to spawn.

You can also easily adapt this to use psexec to spawn the processes remotely (but wouldn't be very viable as you have to have admin privileges on the other machine as well as provide the password in the batch). You would have to use process names for filtering then, though.

share|improve this answer
    
this is some serious black batch magic! I wish I could upvote more times! –  Axarydax Jan 25 '11 at 19:36
    
Nah, that's fairly basic stuff once you get the hang of it ;-) –  Јοеу Jan 25 '11 at 21:33
add comment

Batch files are for extremely basic scripting and do not support any kind of multithreading. You would need to write your own utility to do what you are wanting to accomplish.

Alternatively, Windows PowerShell may provide more capabilities to get you closer to your goal.

If you simply want to run multiple operations at the same time, you can use the command start to launch the PNGOUT utility in a new window. The FOR loop will then continue without waiting on each operation to finish.

The modified line would look like this:

for %i in (*.png) do start pngout "%i"

However, note that this will effectively launch PNGOUT for ALL the files in the directory at the same time, which is most likely not desirable.

share|improve this answer
    
This is a naive approach and as you mention is probably not desireable. You only want to launch as many processes as you have cores. If you have 4 cores and 1,000 files, you really want to process only 4 at a time until all 1,000 are processed. If you try to process all 1,000 at once on 4 cores, your computer will slow to a useless crawl. –  Adisak Aug 25 '11 at 19:38
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.