[Libav-user] Using VAAPI to decode to RGB colors and to DRM PRIME fd

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

[Libav-user] Using VAAPI to decode to RGB colors and to DRM PRIME fd

M. Stoeckl
Hello,

I have the following problem: there is a program outside of my control
which uses and produces DRM PRIME fds only with a specific format (for
example, DRM_FORMAT_XRGB8888/AV_PIX_FMT_BGR0), and I'd like to decode
an input video format (such as H264, or VP9) in YUV colorspace and have
its frames be written to the DRM PRIME fds, without copying the entire
frame back to the CPU to do a color format conversion.

With the h264_vaapi encoder, I can use av_hwframe_map with
AV_PIX_FMT_DRM_PRIME frames (with format DRM_FORMAT_XRGB8888) to produce
AV_PIX_FMT_VAAPI frames with the software format AV_PIX_FMT_NV12, that
the encoder then uses successfully. [Unless I am mistaken, this calls
vaCreateSurfaces using the DRM frame data, and the VAAPI driver does
the format conversion work.]

However, I'm not sure how best to decode with VAAPI; a similar approach
trying av_hwframe_map before the decoder started produced
AVERROR(ENOSYS). When I configure AVVAAPIFramesContext to have the
frames be produced from an existing DRM PRIME file descriptor, the
decoder runs, but the original file descriptor contents aren't updated.
Alternatively, converting the frames produced by the decoder, with
av_hwframe_map, to DRM PRIME frames gives me data in NV12 format, which
isn't what I want.

Has anyone managed to get something similar working, or even any
GPU-only decoding pipeline that can decode YUV video into RGB images,
handling any conversions on-GPU?

Any additional advice (or pointers to relevant example code not already
in FFMPEG) would be appreciated.
_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Using VAAPI to decode to RGB colors and to DRM PRIME fd

M. Stoeckl
Problem solved; it turns out that VAAPI has a postprocessing module
which can do (among other things) format conversions. The FFMPEG
interface to this is the scale_vaapi filter in libavfilter. As
integrating the scale_vaapi filter would have been tricky for my
specific use case, I instead used the VAAPI interface directly to
set up a pipeline without any filters that copies the video output
surface onto a provided target DMABUF.

In case anyone is interested in the code, see [0].

[0]
https://gitlab.freedesktop.org/mstoeckl/waypipe/blob/77c92bb1a40b4ac779414d9ec2ace9841b283993/video.c#L207-363

On Tue, 9 Jul 2019 19:14:34 -0400
"M. Stoeckl" <[hidden email]> wrote:

> Hello,
>
> I have the following problem: there is a program outside of my control
> which uses and produces DRM PRIME fds only with a specific format (for
> example, DRM_FORMAT_XRGB8888/AV_PIX_FMT_BGR0), and I'd like to decode
> an input video format (such as H264, or VP9) in YUV colorspace and
> have its frames be written to the DRM PRIME fds, without copying the
> entire frame back to the CPU to do a color format conversion.
>
> With the h264_vaapi encoder, I can use av_hwframe_map with
> AV_PIX_FMT_DRM_PRIME frames (with format DRM_FORMAT_XRGB8888) to
> produce AV_PIX_FMT_VAAPI frames with the software format
> AV_PIX_FMT_NV12, that the encoder then uses successfully. [Unless I
> am mistaken, this calls vaCreateSurfaces using the DRM frame data,
> and the VAAPI driver does the format conversion work.]
>
> However, I'm not sure how best to decode with VAAPI; a similar
> approach trying av_hwframe_map before the decoder started produced
> AVERROR(ENOSYS). When I configure AVVAAPIFramesContext to have the
> frames be produced from an existing DRM PRIME file descriptor, the
> decoder runs, but the original file descriptor contents aren't
> updated. Alternatively, converting the frames produced by the
> decoder, with av_hwframe_map, to DRM PRIME frames gives me data in
> NV12 format, which isn't what I want.
>
> Has anyone managed to get something similar working, or even any
> GPU-only decoding pipeline that can decode YUV video into RGB images,
> handling any conversions on-GPU?
>
> Any additional advice (or pointers to relevant example code not
> already in FFMPEG) would be appreciated.

_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".