0

I am using FFMPEG to merge some videos into a single video. For some reason, it is running extremely slowly, barely using any computing power (see picture below). I am not recompiling the video. (See below for script that is running.) Normally it runs very quickly. Indeed, at first it was going at >1000 fps. Now it's down around 50.

Importantly, I am running three sessions simultaneously. That is, I have made three copies of the bin folder (which contains the executables) and am running each one separately. Note, though, that three instances totalling around 140fps is significantly worse than a single instance giving more like 1200!

Is there some way that I can get it to fully utilise the processing power? I'm using Windows 10.

FFMPEG_Slow

These three questions seem relevant.

  • Is mergeall a batch or script file? What is your actual ffmpeg command? – Mokubai Feb 27 '20 at 9:21
  • If I were running 3 different sessions of FFMPEG at the same time, on the same hard disk, I would expect about the same running frames per second also, or less. That is a lot of work for your system, especially your memory and harddisk. I bet the disk is hot when your done. – vssher Feb 27 '20 at 9:21
  • What is the bitrate of the files? Is your disk saturated by seeks or bandwidth? – Mokubai Feb 27 '20 at 9:22
  • 1
    Are you running 3 concurrent conversions from a single rotating disk to that same disk? That seems quite suicidal! – Eugen Rieck Feb 27 '20 at 9:53
  • @vssher yes, I would expect them to be all the same as each other (up to some small variation). It just seems odd that the sum is ~140fps when an individual one running at >1000fps, almost 10x time speed – Sam OT Feb 27 '20 at 10:04
3

Assuming that you run those conversions from and to a rotating disk, you are very likely to make a CPU-bound job into a disk-bound one.

A conversion process consist of three tasks:

  1. Read the original file from disk
  2. Do some calculation on the data
  3. Write the result file to the disk

A rotating disk is good at sequential reads or writes, but extremly bad at random IO - so even a single conversion can be hampered by the concurrency between 1. and 3. This implies that a conversion from one physical disk to another is likely to be faster than a conversion from a disk to itself.

If you now multiply this concurrency by three, you are very likely to end up in a scenario, where the seek and wait-on-rotation times of the disk by far outweigh the actual read times - this can easily lead to throughput going down by orders of magnitude: It is not uncommon for a disk, that can reach over 100MB/s sequential read, to reach less than a single MB/s random reads.

A usually seen pattern is very fast inital performance, while the writes are buffered in RAM, but dropping of a cliff, when the cache is full and the writes really need to hit the disk.

Recommendations: - First of all get rid of spinning rust - it is 2020. - If that is no option, then try to limit IO concurrency by using different disks for read and write. The best way might be to create a RAM disk as a target device (as usual in the broadcast industry). In fact since RAM is so cheap it might be a good idea to convert from a RAM disk into a RAM disk. - Carefully chose the number of concurrent conversion jobs to find the sweet spot between IO saturation and CPU/GPU saturation.

  • Thanks for this description; it is very clear! I do relatively little video stuff, so don't want to splash out big £££ on a large SSD to be used infrequently. I think I shall try to set it up to read from one disk and write to another. In fact, my videos are rarely >10GBs, so I can read it from my large HDD and write it to my smaller SSD! – Sam OT Feb 27 '20 at 11:21
  • Neat benchmark to prove the point: hdd.userbenchmark.com/Seagate-Barracuda-1TB-2016/Rating/3896 sequential reads/writes = fast, small (4k) reads/writes = painfully slow. Once the head starts seeking across the drive the performance of the disk is dominated by that time and read/write speeds can fall down to less than 1% of the peak performance of a spinning rust disk. – Mokubai Feb 27 '20 at 11:23
  • Here's a benchmark of my machine (same site) from a few weeks ago. Hard drives are pretty bad in it, for some reason. (It was a gaming machine back in 2010 when I got it! Look at the poor thing now...) – Sam OT Feb 27 '20 at 22:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.