The ffmpeg version in OBS has the flag.

But just for fun I tried the ffmpeg command line anyway with my dnf package of ffmpeg and it seems to work!

Here's the command line I used:
 
ffmpeg -loglevel debug -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -i /dev/video0 -vf 'format=nv12,hwupload' -y -f matroska -acodec copy -b:v 12500k -vcodec h264_vaapi output.mkv

And here's the first lines of the output:

Spoiler, click me

 
ffmpeg version 4.4.1 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 11 (GCC)
  configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --docdir=/usr/share/doc/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 --optflags='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection' --extra-ldflags='-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 ' --extra-cflags=' -I/usr/include/rav1e' --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libvo-amrwbenc --enable-version3 --enable-bzlib --enable-chromaprint --disable-crystalhd --enable-fontconfig --enable-frei0r --enable-gcrypt --enable-gnutls --enable-ladspa --enable-libaom --enable-libdav1d --enable-libass --enable-libbluray --enable-libbs2b --enable-libcdio --enable-libdrm --enable-libjack --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libilbc --enable-libmp3lame --enable-libmysofa --enable-nvenc --enable-openal --enable-opencl --enable-opengl --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librav1e --enable-librtmp --enable-librubberband --enable-libsmbclient --enable-version3 --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-version3 --enable-vapoursynth --enable-libvpx --enable-vulkan --enable-libglslang --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libxml2 --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-avfilter --enable-avresample --enable-libmodplug --enable-postproc --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-stripping --shlibdir=/usr/lib64 --enable-lto --enable-libmfx --enable-runtime-cpudetect
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Splitting the commandline.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
Reading option '-hwaccel' ... matched as option 'hwaccel' (use HW accelerated decoding) with argument 'vaapi'.
Reading option '-vaapi_device' ... matched as option 'vaapi_device' (set VAAPI hardware device (DRM path or X11 display name)) with argument '/dev/dri/renderD128'.
Reading option '-i' ... matched as input url with argument '/dev/video0'.
Reading option '-vf' ... matched as option 'vf' (set video filters) with argument 'format=nv12,hwupload'.
Reading option '-y' ... matched as option 'y' (overwrite output files) with argument '1'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'matroska'.
Reading option '-acodec' ... matched as option 'acodec' (force audio codec ('copy' to copy stream)) with argument 'copy'.
Reading option '-b:v' ... matched as option 'b' (video bitrate (please use -b:v)) with argument '12500k'.
Reading option '-vcodec' ... matched as option 'vcodec' (force video codec ('copy' to copy stream)) with argument 'h264_vaapi'.
Reading option 'output.mkv' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option loglevel (set logging level) with argument debug.
Applying option vaapi_device (set VAAPI hardware device (DRM path or X11 display name)) with argument /dev/dri/renderD128.
[AVHWDeviceContext @ 0x557a85552a40] libva: VA-API version 1.13.0
[AVHWDeviceContext @ 0x557a85552a40] libva: Trying to open /usr/lib64/dri/radeonsi_drv_video.so
[AVHWDeviceContext @ 0x557a85552a40] libva: Found init function __vaDriverInit_1_13
[AVHWDeviceContext @ 0x557a85552a40] libva: va_openDriver() returns 0
[AVHWDeviceContext @ 0x557a85552a40] Initialised VAAPI connection: version 1.13
[AVHWDeviceContext @ 0x557a85552a40] Format 0x3231564e -> nv12.
[AVHWDeviceContext @ 0x557a85552a40] Format 0x30313050 -> p010le.
[AVHWDeviceContext @ 0x557a85552a40] Format 0x36313050 -> unknown.
[AVHWDeviceContext @ 0x557a85552a40] Format 0x30323449 -> yuv420p.
[AVHWDeviceContext @ 0x557a85552a40] Format 0x32315659 -> yuv420p.
[AVHWDeviceContext @ 0x557a85552a40] Format 0x56595559 -> unknown.
[AVHWDeviceContext @ 0x557a85552a40] Format 0x32595559 -> yuyv422.
[AVHWDeviceContext @ 0x557a85552a40] Format 0x59565955 -> uyvy422.
[AVHWDeviceContext @ 0x557a85552a40] Format 0x41524742 -> bgra.
[AVHWDeviceContext @ 0x557a85552a40] Format 0x41424752 -> rgba.
[AVHWDeviceContext @ 0x557a85552a40] Format 0x58524742 -> bgr0.
[AVHWDeviceContext @ 0x557a85552a40] Format 0x58424752 -> rgb0.
[AVHWDeviceContext @ 0x557a85552a40] VAAPI driver: Mesa Gallium driver 21.3.7 for AMD Radeon RX 6700 XT (NAVY_FLOUNDER, DRM 3.44.0, 5.16.15-201.fc35.x86_64, LLVM 13.0.0).
[AVHWDeviceContext @ 0x557a85552a40] Driver not found in known nonstandard list, using standard behaviour.
Applying option y (overwrite output files) with argument 1.
Successfully parsed a group of options.
Parsing a group of options: input url /dev/video0.
Applying option hwaccel (use HW accelerated decoding) with argument vaapi.
Successfully parsed a group of options.
Opening an input file: /dev/video0.

As you can see towards the end, after the line `Applying option vaapi_device` everything seemed to be detected as expected (and the video produced was readable).

That said, when I forced the resolution to 1920x1080 it became sluggish (though not as bad as the video generated by OBS).

So it's still progress, but not usable yet for a good streaming setup.

It's either a bug in ffmpeg or in AMD's vaapi then (so Mesa). Dealing with ffmpeg bugs is a mess though, they use some ancient bug tracker.

Last edited by Shmerl on 21 March 2022 at 5:21 pm UTC

Well, I think I'll wait for the bug to fix itself at some point (I must not be the only one having issue streaming using HW encoding on an AMD GPU )

I wouldn't count on it. ffmpeg project is kind of a mess when it comes to fixing bugs especially if it's not even reported.

At least try to find the bug report.

Last edited by Shmerl on 21 March 2022 at 5:25 pm UTC

On the other hand, there is a high chance maybe the slowlyness (on the HW accelerated encoded video) is coming from a bad command-line on my side. My webcam should be able to do 1080p but on Windows... so maybe the Linux drivers are just bad. Or maybe it's an OBS issue, or maybe a Flatpak issue, or ...

I'm afraid there are just too many "maybe"s and I know right now that I won't have time to run the tests they'll ask me to run, so my ticket will be closed in 6 months or a year because there won't be any answer.

There are 8 pages of tickets about "AMD" and "vaapi". Let's hope my bug is listed in there somewhere

A simple test is to feed a video file first, without realtime video feed.

Don't even bother with VAAPI and OBS it's a waist of time.

Use the AMF encoder , Works flawlessly with barely any impact on framerate!



Here's Boarderlands 3 benchmark recorded with it @ 1440p


https://www.youtube.com/watch?v=ofbuvYRozjM

The issue is that AMF requires the proprietary drivers, which I don't have and don't even know how to install.

I read on GoL's Discord that Mesa 22 will get a bunch of optimizations for video encoding. It even gets VA-API to be faster than AMF apparently (from a small test a user published on Discord. So hopefully hardware encoding on RDNA GPUs will be solved!

You can always test Mesa main with vaapi. Just make sure to set LIBVA_DRIVERS_PATH to where your built radeonsi_drv_video.so is located (I also set LD_LIBRARY_PATH to that place).

Last edited by Shmerl on 27 March 2022 at 8:22 pm UTC

Quoting: CreakThe issue is that AMF requires the proprietary drivers, which I don't have and don't even know how to install.

I read on GoL's Discord that Mesa 22 will get a bunch of optimizations for video encoding. It even gets VA-API to be faster than AMF apparently (from a small test a user published on Discord. So hopefully hardware encoding on RDNA GPUs will be solved!

No clue on Fedora but I would expect it's the same as all other major distros, Go to package manager and find AMF-AMDGPU... (chose the 21.30 version as 21.50 broke )and should install like any other package. You will also need obs-steamfx installing.

Quoting: ShmerlYou can always test Mesa main with vaapi. Just make sure to set LIBVA_DRIVERS_PATH to where your built radeonsi_drv_video.so is located (I also set LD_LIBRARY_PATH to that place).

Thats a lot of messing for most people to be frank, Alot easier just to install a package or two .