So I am trying to find out what the best way to transcode using FFMPEG would be on a commercial scale. I currently just have very CPU heavy cloud instances that run FFMPEG with parameters set as a balance between speed and quality. However I am now looking into a long term solution that will allow me to really transcode hundreds of video files daily maybe a few dozen at one time. So I looked into GPU acceleration and it seems due to the way the compression algorithms are structured CPU is generally faster than GPU anyway (more on that here). So next up is dedicated hardware, is there some sort of dedicated h.264 / h.264 encoder I could get and physically install in my server that would make FFMPEG transcoding blazing fast?
You need to understand that hardware encoders generally produce slightly lower quality images at the same bitrate (or larger files at the same quality). If that’s ok, your best options are quick sync on intel CPUs, or nvenc on Nvidia GPUs. If you go Nvidia, make sure you check the specs on the card, as most of them are limited to 2 parallel encodes.
Hardware encoders are generally used for live encoding, providing a lot more stability than software which relies on an general purpose operating system.
Also Hardware encoders are limited to a certain set of options while software encoders are open for any thinkable option. Developing a hardware encoder costs multiple million dollars and it is not a flexible thing at all. E.g. Nvidia NVENC chips do not yet support any 4:2:2 colorspace profile (but future generations will do). Why is Nvidia not building encoder boards without all the graphics features? ...that is because because there is no money in this business.
Assumed what you want is to store the encoded result as a "file on harddrive", hardware encoders is not the way to go for you. Any professional hardware encoder that i know is built for network distribution and "live" usage only.
So, as long as you are in a cloud environment, you better use CPU encoding and just go for the CPU with highest frequency if you want speed.