[Libav-user] H264 Packed RGB Decoding

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

[Libav-user] H264 Packed RGB Decoding

Alan Daniels
Hello,

Is there a way for FFmpeg to decode an H264 bitstream into packed RGB?
Ideally AV_PIX_FMT_BGR0 or AV_PIX_FMT_BGRA.

For the encoder, I am using libx264rgb with AV_PIX_FMT_BGR0.

For the decoder, I am using AV_CODEC_ID_H264. However, the frame comes back from the decoder as AV_PIX_FMT_GBRP. This seems to be the only option supplied to get_format().

I am trying to minimize encode/decode latency, so I am trying to avoid using swscale after decoding.

Thanks!
- Alan


_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: H264 Packed RGB Decoding

Devin Heitmueller
On Thu, Feb 6, 2020 at 8:25 PM Alan Daniels <[hidden email]> wrote:
> Is there a way for FFmpeg to decode an H264 bitstream into packed RGB?
> Ideally AV_PIX_FMT_BGR0 or AV_PIX_FMT_BGRA.
>
> For the encoder, I am using libx264rgb with AV_PIX_FMT_BGR0.
>
> For the decoder, I am using AV_CODEC_ID_H264. However, the frame comes back from the decoder as AV_PIX_FMT_GBRP. This seems to be the only option supplied to get_format().

The output of the decoder is whatever format the video was originally
encoded with.  Decoders generally don't do any sort of colorspace
conversion.  In this case the output of the decoder is a planar
version of RGB (since codecs typically operate on packed formats).
Presumably if you really did feed the encoder BGR0 (which is a packed
format), something converted it to planar format prior to encoding.
The ffmpeg program will automatically insert swscale into the filter
pipeline to convert packed to planar as would have been required by
the encoder.

> I am trying to minimize encode/decode latency, so I am trying to avoid using swscale after decoding.

It's probably unavoidable - if you need a packed format for your
output then you're going to have to convert planar to packed.  On the
upside, swscale isn't something that is typically a large source of
latency.  Hardware inputs/outputs tend to use packed formats (e.g.
cameras, SDI cards, etc), while codecs tend to use planar formats, and
yet people are still able to create very low latency solutions.

There are plenty of other places within your pipeline which would
likely be larger sources of latency (such as the numerous FIFOs
internal to ffmpeg).

Devin

--
Devin J. Heitmueller - Kernel Labs
http://www.kernellabs.com
_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: H264 Packed RGB Decoding

Carl Eugen Hoyos-2


> Am 07.02.2020 um 05:06 schrieb Devin Heitmueller <[hidden email]>:
>
>> On Thu, Feb 6, 2020 at 8:25 PM Alan Daniels <[hidden email]> wrote:
>> Is there a way for FFmpeg to decode an H264 bitstream into packed RGB?
>> Ideally AV_PIX_FMT_BGR0 or AV_PIX_FMT_BGRA.
>>
>> For the encoder, I am using libx264rgb with AV_PIX_FMT_BGR0.
>>
>> For the decoder, I am using AV_CODEC_ID_H264. However, the frame comes back from the decoder as AV_PIX_FMT_GBRP. This seems to be the only option supplied to get_format().
>
> The output of the decoder is whatever format the video was originally
> encoded with.

No.

The decoder output depends on development decisions that were hopefully driven by reasons such as speed and readability.

Carl Eugen
_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: H264 Packed RGB Decoding

Carl Eugen Hoyos-2
In reply to this post by Alan Daniels


Am 07.02.2020 um 02:25 schrieb Alan Daniels <[hidden email]>:

Is there a way for FFmpeg to decode an H264 bitstream into packed RGB?

No but as Devin correctly explained the software scaler should not add a lot of latency compared to the 4:4:4 decoding.

Carl Eugen

_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: H264 Packed RGB Decoding

Alan Daniels
In reply to this post by Devin Heitmueller
> Presumably if you really did feed the encoder BGR0 (which is a packed
> format), something converted it to planar format prior to encoding.
> The ffmpeg program will automatically insert swscale into the filter
> pipeline to convert packed to planar as would have been required by
> the encoder.

libx264rgb only advertises BGR0, BGR24, and RGB24 as supported formats.
I think it would have GBRP as a supported format if it really converted to that prior to encoding.

ffmpeg.exe would insert swscale, though I don't think libavcodec does. Could be wrong.

However, ffprobe detects the bitstream as GBRP..

- Alan



_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: H264 Packed RGB Decoding

Alan Daniels
In reply to this post by Carl Eugen Hoyos-2
> No but as Devin correctly explained the software scaler should not add a lot of latency compared to the 4:4:4 decoding.

Perhaps I am doing something wrong then. swscale always takes a long time for me:
~10 ms for GBRP -> BGRA
~30 ms for YUV420P -> BGRA

I haven't fully looked into minimizing decoder latency yet, though it varies from ~5-50 ms depending on content.
2560x1440 resolution.

- Alan


_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: H264 Packed RGB Decoding

Paul B Mahol
On 2/7/20, Alan Daniels <[hidden email]> wrote:

>> No but as Devin correctly explained the software scaler should not add a
>> lot of latency compared to the 4:4:4 decoding.
>
>
>
> Perhaps I am doing something wrong then. swscale always takes a long time
> for me:
>
>  ~10 ms for GBRP -> BGRA
>
> ~30 ms for YUV420P -> BGRA
>
>
>
> I haven't fully looked into minimizing decoder latency yet, though it varies
> from ~5-50 ms depending on content.
>
> 2560x1440 resolution.
>

Do you have assembly SIMD disabled? nasm/yasm is installed?
_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: H264 Packed RGB Decoding

Alan Daniels
> Do you have assembly SIMD disabled? nasm/yasm is installed?

I believe SIMD is enabled. What's a good way to check? In my config.h, SIMD, SSE, and AVX external defines are set to 1.

I have nasm and yasm.

- Alan


_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".