Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Here is the situation:

I am processing images, transforming them from tif to ecw and reprojecting them.

The software I am using one uses one process, and only one core it seems. When I run a batch operation all I see is 13% of the cpu used.

My specs are the following:

i7 3770K 16 GB of ram 2TB sata 7200 HD

The program processes file by file, and I have around one million files to transform.

To try and be more efficient, I have separated all the files in 8 folders and ran the batch operations separately. This way I am able to hit 100% on the CPU.

Since each process also reads the file, and writes a new one, I was afraid I/O might be an issue, but from what I understand in the task manager, this is not a problem? (not sure)

My question is, is this the best way to go? I am actually slowing myself down by running 8 batches or should I run even more?

Thanks.

att. task manager: 1 2 3 4

share|improve this question
1  
You're CPU-bound, not IO-bound, so there's no point increasing the number of concurrent processes you're running. –  Aaron Miller Jun 28 '13 at 17:44
    
cpu bound with 8 processes, I would probably stick with one less than 100%, just so i could have some for myself. When I did the same thing I found some Programs were 10or more times slower at doing the same thing as others. So I hope you started with the right program for the job. For the very specific changes I was making Xnview took DAYS less time than than photoshop did, so much faster I did not worry about full utalisation. –  Psycogeek Jun 29 '13 at 7:57

2 Answers 2

up vote 3 down vote accepted

If the program you're using does not natively support multithreading, and only a single thread does all the work (in this case, the conversion of the image), then you would see approximately 1/8 of your processor being used (or 12.5%) in the best-case.

Indeed if this is the case, you can achieve a much higher throughput by spawning several of these processes in parallel, assuming that they don't do too much disk I/O (which is affected by accessing several files simultaneously) - but in the screenshot you provided, it appears as though the tasks are CPU limited and not disk limited; no process goes over 0.5 MB/s read/write rate.

Just note that in your particular case, this will also cause the average RAM usage for the program to increase eight-fold, so you may have to run less if memory usage becomes a limiting factor. If you find your system experience becomes too slow, you can also set the processes to run at a lower priority.

share|improve this answer

Looking at those graphs, I agree with Aaron Miller that it looks like it's more of a CPU bottleneck now than an IO bottleneck. I would say to remove batches so you'd be at most 90% utilized on the CPU, and you should be good. The throughput on the drive looks good, even though that response time is terrible, but the only way to really help that is to run fewer batches, or split the work across several drives (Which you don't appear to have).

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.