This is now possible, but on Linux via VAAPI.
Here is a gist that shows you how to generate a working build on Ubuntu 16.04LTS.
For the QSV-based encoders, as tested with the opensource Intel Media SDK (on an Apollolake testbed), only the H.264/AVC and the HEVC encoders are available. Newer generations will typically expose more encoding capabilities.
Note that QuickSync H/W acceleration requires the presence of the media-driver package on the system. Some Linux distributions, such as Ubuntu 19.04LTS (and above) package it in their repositories.
Update: VP9 encoding is now enabled for IceLake (ICL+) hardware through QuickSync, and FFmpeg can take advantage of it.
Examples demonstrating the use of FFmpeg's vp9_vaapi
and vp9_qsv
encoder wrappers:
You will need to use FFmpeg directly, where you may optionally add the vp9_superframe and the vp9_raw_reorder bitstream filters in the same command line if you enable B-frames in the vp9_vaapi
encoder.
Example:
ffmpeg -vaapi_device /dev/dri/renderD128 -hwaccel vaapi \
-i http://server:port -vf 'format=nv12|vaapi,hwupload' \
-c:v vp9_vaapi -global_quality 50 -bf 1 \
-bsf:v vp9_raw_reorder,vp9_superframe \
-f segment -segment_time 5 -segment_format_options movflags=+faststart output%03d.mp4
Adjust your input and output paths/urls as needed.
What this command does:
It will create 5 second long mp4 segments, via the segment muxer.
See the usage of the movflags=+faststart
, and how it has been passed as a format option to the underlying mp4 muxer via -segment_format_options
flag above.
The segment lengths may not be exactly 5 seconds long, as each segment begins (gets cut on) (with) a keyframe.
However, I'd not recommend enabling B-frames in that encoder, as these bitstream filters have other undesired effects, such as mucking around with the encoder's rate control and triggering bugs like this one. This is not desirable in a production environment. This is why the scripts below do not have that option enabled, and instead, we define a set rate control mode directly in the encoder options.
If you need to take advantage of 1:N encoding with VAAPI, use these snippets:
- If you need to deinterlace, call up the
deinterlace_vaapi
filter:
ffmpeg -loglevel debug -threads 1 \
-init_hw_device vaapi=va:/dev/dri/renderD128 -hwaccel vaapi -hwaccel_device va -filter_hw_device va -hwaccel_output_format vaapi \
-i 'http://server:port' \
-filter_complex "[0:v]format=nv12|vaapi,hwupload,deinterlace_vaapi,split=3[n0][n1][n2]; \
[n0]scale_vaapi=1152:648[v0]; \
[n1]scale_vaapi=848:480[v1];
[n2]scale_vaapi=640:360[v2]" \
-b:v:0 2250k -maxrate:v:0 2250k -bufsize:v:0 360k -c:v:0 vp9_vaapi -g:v:0 50 -r:v:0 25 -rc_mode:v:0 2 \
-b:v:1 1750k -maxrate:v:1 1750k -bufsize:v:1 280k -c:v:1 vp9_vaapi -g:v:1 50 -r:v:1 25 -rc_mode:v:1 2 \
-b:v:2 1000k -maxrate:v:2 1000k -bufsize:v:2 160k -c:v:2 vp9_vaapi -g:v:2 50 -r:v:2 25 -rc_mode:v:2 2 \
-c:a aac -b:a 128k -ar 48000 -ac 2 \
-flags -global_header -f tee -use_fifo 1 \
-map "[v0]" -map "[v1]" -map "[v2]" -map 0:a \
"[select=\'v:0,a\':f=segment:segment_time=5:segment_format_options=movflags=+faststart]$output_path0/output%03d.mp4| \
[select=\'v:1,a\':f=segment:segment_time=5:segment_format_options=movflags=+faststart]$output_path1/output%03d.mp4| \
[select=\'v:2,a\':f=segment:segment_time=5:segment_format_options=movflags=+faststart]$output_path2/output%03d.mp4"
- Without deinterlacing:
ffmpeg -loglevel debug -threads 1 \
-init_hw_device vaapi=va:/dev/dri/renderD128 -hwaccel vaapi -hwaccel_device va -filter_hw_device va -hwaccel_output_format vaapi \
-i 'http://server:port' \
-filter_complex "[0:v]format=nv12|vaapi,hwupload,split=3[n0][n1][n2]; \
[n0]scale_vaapi=1152:648[v0]; \
[n1]scale_vaapi=848:480[v1];
[n2]scale_vaapi=640:360[v2]" \
-b:v:0 2250k -maxrate:v:0 2250k -bufsize:v:0 2250k -c:v:0 vp9_vaapi -g:v:0 50 -r:v:0 25 -rc_mode:v:0 2 \
-b:v:1 1750k -maxrate:v:1 1750k -bufsize:v:1 1750k -c:v:1 vp9_vaapi -g:v:1 50 -r:v:1 25 -rc_mode:v:1 2 \
-b:v:2 1000k -maxrate:v:2 1000k -bufsize:v:2 1000k -c:v:2 vp9_vaapi -g:v:2 50 -r:v:2 25 -rc_mode:v:2 2 \
-c:a aac -b:a 128k -ar 48000 -ac 2 \
-flags -global_header -f tee -use_fifo 1 \
-map "[v0]" -map "[v1]" -map "[v2]" -map 0:a \
"[select=\'v:0,a\':f=segment:segment_time=5:segment_format_options=movflags=+faststart]$output_path0/output%03d.mp4| \
[select=\'v:1,a\':f=segment:segment_time=5:segment_format_options=movflags=+faststart]$output_path1/output%03d.mp4| \
[select=\'v:2,a\':f=segment:segment_time=5:segment_format_options=movflags=+faststart]$output_path2/output%03d.mp4"
- Using Intel's QuickSync (on supported platforms):
On Intel Icelake and above, you can use the vp9_qsv
encoder wrapper with the following known limitations (for now):
(a). You must enable low_power mode
because only the VDENC decode path is exposed by the iHD
driver for now.
(b). Coding option1 and extra_data are not supported by MSDK.
(c). The IVF header will be inserted in MSDK by default, but it is not needed for FFmpeg, and remains disabled by default.
See the examples below:
- If you need to deinterlace, call up the
vpp_qsv
filter:
ffmpeg -nostdin -y -fflags +genpts -init_hw_device qsv=hw \
-hwaccel qsv -hwaccel_output_format qsv \
-filter_hw_device hw \
-threads 1 -vsync 1 -async 1 -reinit_filter 1 \
-i 'http://server:port' \
-filter_complex "[0:v]format=nv12|qsv,hwupload=extra_hw_frames=64,vpp_qsv=deinterlace=2:async_depth=4,split[n0][n1][n2]; \
[n0]vpp_qsv=w=1152:h=648:async_depth=4[v0]; \
[n1]vpp_qsv=w=848:h=480:async_depth=4[v1];
[n2]vpp_qsv=w=640:h=360:async_depth=4[v2]" \
-b:v:0 2250k -maxrate:v:0 2250k -bufsize:v:0 360k -c:v:0 vp9_qsv -g:v:0 50 -r:v:0 25 -low_power:v:0 2 \
-b:v:1 1750k -maxrate:v:1 1750k -bufsize:v:1 280k -c:v:1 vp9_qsv -g:v:1 50 -r:v:1 25 -low_power:v:1 2 \
-b:v:2 1000k -maxrate:v:2 1000k -bufsize:v:2 160k -c:v:2 vp9_qsv -g:v:2 50 -r:v:2 25 -low_power:v:2 2 \
-c:a aac -b:a 128k -ar 48000 -ac 2 \
-flags -global_header -f tee -use_fifo 1 \
-map "[v0]" -map "[v1]" -map "[v2]" -map 0:a \
"[select=\'v:0,a\':f=segment:segment_time=5:segment_format_options=movflags=+faststart]$output_path0/output%03d.mp4| \
[select=\'v:1,a\':f=segment:segment_time=5:segment_format_options=movflags=+faststart]$output_path1/output%03d.mp4| \
[select=\'v:2,a\':f=segment:segment_time=5:segment_format_options=movflags=+faststart]$output_path2/output%03d.mp4"
- Without deinterlacing:
ffmpeg -nostdin -y -fflags +genpts -init_hw_device qsv=hw \
-hwaccel qsv -hwaccel_output_format qsv \
-filter_hw_device hw \
-threads 1 -vsync 1 -async 1 -reinit_filter 1 \
-i 'http://server:port' \
-filter_complex "[0:v]format=nv12|qsv,hwupload=extra_hw_frames=64,split=3[n0][n1][n2]; \
[n0]vpp_qsv=w=1152:h=648:async_depth=4[v0]; \
[n1]vpp_qsv=w=848:h=480:async_depth=4[v1];
[n2]vpp_qsv=w=640:h=360:async_depth=4[v2]" \
-b:v:0 2250k -maxrate:v:0 2250k -bufsize:v:0 2250k -c:v:0 vp9_qsv -g:v:0 50 -r:v:0 25 -low_power:v:0 2 \
-b:v:1 1750k -maxrate:v:1 1750k -bufsize:v:1 1750k -c:v:1 vp9_qsv -g:v:1 50 -r:v:1 25 -low_power:v:1 2 \
-b:v:2 1000k -maxrate:v:2 1000k -bufsize:v:2 1000k -c:v:2 vp9_qsv -g:v:2 50 -r:v:2 25 -low_power:v:2 2 \
-c:a aac -b:a 128k -ar 48000 -ac 2 \
-flags -global_header -f tee -use_fifo 1 \
-map "[v0]" -map "[v1]" -map "[v2]" -map 0:a \
"[select=\'v:0,a\':f=segment:segment_time=5:segment_format_options=movflags=+faststart]$output_path0/output%03d.mp4| \
[select=\'v:1,a\':f=segment:segment_time=5:segment_format_options=movflags=+faststart]$output_path1/output%03d.mp4| \
[select=\'v:2,a\':f=segment:segment_time=5:segment_format_options=movflags=+faststart]$output_path2/output%03d.mp4"
Note that we use the vpp_qsv filter
with the async_depth
option set to 4. This massively improves transcode performance over using scale_qsv
and deinterlace_qsv
. See this commit on FFmpeg's git.
Note: If you use the QuickSync path, note that MFE (Multi-Frame encoding mode) will be enabled by default if the Media SDK library on your system supports it.
References:
- See the encoder options, including rate control methods supported:
ffmpeg -h encoder=vp9_vaapi
- See the deinterlace_vaapi filter usage options:
ffmpeg -h filter=deinterlace_vaapi
- On the
vpp_qsv
filter usage, see:
ffmpeg -h filter=vpp_qsv
For instance, if you want field-rate output rather than frame-rate output from the deinterlacer, you could pass the rate=field
option to it instead:
-vf=vaapi_deinterlace=rate=field
This feature, for instance, is tied to encoders that support MBAFF. Others, such as the NVENC-based ones in FFmpeg, do not have this implemented (as of the time of writing).
Tips on maximizing performance with hardware-based encoders in FFmpeg:
Where possible, infer to the built-in docs, as with the examples shown above.
They can uncover potential pitfalls that you may be able to avoid by understanding how filter chaining and encoder initialization works, unsupported features, etc, and the impact on performance.
Secondly, where possible, take advantage of 1:N transcoding where multiple outputs are required. That way, inputs and filter chain products can be re-used.
For example, you'll see that in the snippets above, we call up the deinterlacer only once, then split its' output via the split
filter to separate scalers. This is done to lower the overhead that would be incurred had we called up the deinterlacer more than once, and it would've been wasteful.
decoders: vp8 libvpx vp8_cuvid vp8_qsv
- that's a clear statement, so why should there be anything wrong with it? Also note FFmpeg's page on QuickSync-support