[Libav-user] Decoding h264 (NV12) stream from NVENC

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

[Libav-user] Decoding h264 (NV12) stream from NVENC

Michael IV
Hi! I am using NVENC encoder to create h264 stream.The YUV format is NV12.
Now, I need also to decode that stream back to YUV.I wrote a module for that,based on
this example:


But, because my YUV format is not 420P, but NV12 I am trying to force it on AVCodecContext when initializing it with this line:

        avcodec_register_all();
        mCodec = avcodec_find_decoder(AV_CODEC_ID_H264);
        mCodecContext = avcodec_alloc_context3(mCodec);
        mCodecContext->pix_fmt = AVPixelFormat::AV_PIX_FMT_NV12;


But when I am receiving the decoded AVFrame, I see that the AVFrame->format property is 
set to zero, which is enum for 420P. Also inspecting the planes and other props of this struct I see that the decoder tried to decode it as 420P. (3 planes , 3 line sizes etc.) .And of course,when trying to play the output of such a YUV frame , there is just a green screen.
I am trying to understand where is my error.Is there some data that decoder misses from the h264 stream and it doesn't allow it to guess the pixel format?Or maybe I need to set pixel format for the codec context in a some different way? 

_______________________________________________
Libav-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/libav-user
Reply | Threaded
Open this post in threaded view
|

Re: Decoding h264 (NV12) stream from NVENC

Hendrik Leppkes
On Mon, Jul 31, 2017 at 10:00 AM, Michael IV <[hidden email]> wrote:

> Hi! I am using NVENC encoder to create h264 stream.The YUV format is NV12.
> Now, I need also to decode that stream back to YUV.I wrote a module for
> that,based on
> this example:
>
> https://gist.github.com/roxlu/9329339
>
> But, because my YUV format is not 420P, but NV12 I am trying to force it on
> AVCodecContext when initializing it with this line:
>
>         avcodec_register_all();
>         mCodec = avcodec_find_decoder(AV_CODEC_ID_H264);
>         mCodecContext = avcodec_alloc_context3(mCodec);
>         mCodecContext->pix_fmt = AVPixelFormat::AV_PIX_FMT_NV12;
>
>
> But when I am receiving the decoded AVFrame, I see that the AVFrame->format
> property is
> set to zero, which is enum for 420P. Also inspecting the planes and other
> props of this struct I see that the decoder tried to decode it as 420P. (3
> planes , 3 line sizes etc.) .And of course,when trying to play the output of
> such a YUV frame , there is just a green screen.
> I am trying to understand where is my error.Is there some data that decoder
> misses from the h264 stream and it doesn't allow it to guess the pixel
> format?Or maybe I need to set pixel format for the codec context in a some
> different way?
>

You can't control the output pixel format of the H264 decoder. A video
is not encoded as "NV12", its encoded as 4:2:0 YUV, NV12 is just one
representation of this format.
As such, our decoder will always decode 4:2:0 YUV content to yuv420p,
not nv12. If you require NV12 in your application, you can convert
from yuv420p to nv12 using swscale (a lossless conversion).

- Hendrik
_______________________________________________
Libav-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/libav-user
Reply | Threaded
Open this post in threaded view
|

Re: Decoding h264 (NV12) stream from NVENC

Michael IV
So you basically say that the decoder 'knows' to detect interleaved UV in NV12 UV plane and decode it correctly to yuv420p without any hints from the application? 
Then why my YUV frame is corrupted? How can I detect where is the problem? I also found this SO answer regarding h264 streaming from NVENC.

And I currently don't set those params for encoder.Do I have to? 
 

On Mon, Jul 31, 2017 at 11:04 AM, Hendrik Leppkes <[hidden email]> wrote:
On Mon, Jul 31, 2017 at 10:00 AM, Michael IV <[hidden email]> wrote:
> Hi! I am using NVENC encoder to create h264 stream.The YUV format is NV12.
> Now, I need also to decode that stream back to YUV.I wrote a module for
> that,based on
> this example:
>
> https://gist.github.com/roxlu/9329339
>
> But, because my YUV format is not 420P, but NV12 I am trying to force it on
> AVCodecContext when initializing it with this line:
>
>         avcodec_register_all();
>         mCodec = avcodec_find_decoder(AV_CODEC_ID_H264);
>         mCodecContext = avcodec_alloc_context3(mCodec);
>         mCodecContext->pix_fmt = AVPixelFormat::AV_PIX_FMT_NV12;
>
>
> But when I am receiving the decoded AVFrame, I see that the AVFrame->format
> property is
> set to zero, which is enum for 420P. Also inspecting the planes and other
> props of this struct I see that the decoder tried to decode it as 420P. (3
> planes , 3 line sizes etc.) .And of course,when trying to play the output of
> such a YUV frame , there is just a green screen.
> I am trying to understand where is my error.Is there some data that decoder
> misses from the h264 stream and it doesn't allow it to guess the pixel
> format?Or maybe I need to set pixel format for the codec context in a some
> different way?
>

You can't control the output pixel format of the H264 decoder. A video
is not encoded as "NV12", its encoded as 4:2:0 YUV, NV12 is just one
representation of this format.
As such, our decoder will always decode 4:2:0 YUV content to yuv420p,
not nv12. If you require NV12 in your application, you can convert
from yuv420p to nv12 using swscale (a lossless conversion).

- Hendrik
_______________________________________________
Libav-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/libav-user


_______________________________________________
Libav-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/libav-user
Reply | Threaded
Open this post in threaded view
|

Re: Decoding h264 (NV12) stream from NVENC

jing zhang
You should use decoded AVFrame->format to check decoded yuv format.
Maybe H.264 bitstream encoded by NVENC is corrupted?

2017-07-31 16:27 GMT+08:00 Michael IV <[hidden email]>:
So you basically say that the decoder 'knows' to detect interleaved UV in NV12 UV plane and decode it correctly to yuv420p without any hints from the application? 
Then why my YUV frame is corrupted? How can I detect where is the problem? I also found this SO answer regarding h264 streaming from NVENC.

And I currently don't set those params for encoder.Do I have to? 
 

On Mon, Jul 31, 2017 at 11:04 AM, Hendrik Leppkes <[hidden email]> wrote:
On Mon, Jul 31, 2017 at 10:00 AM, Michael IV <[hidden email]> wrote:
> Hi! I am using NVENC encoder to create h264 stream.The YUV format is NV12.
> Now, I need also to decode that stream back to YUV.I wrote a module for
> that,based on
> this example:
>
> https://gist.github.com/roxlu/9329339
>
> But, because my YUV format is not 420P, but NV12 I am trying to force it on
> AVCodecContext when initializing it with this line:
>
>         avcodec_register_all();
>         mCodec = avcodec_find_decoder(AV_CODEC_ID_H264);
>         mCodecContext = avcodec_alloc_context3(mCodec);
>         mCodecContext->pix_fmt = AVPixelFormat::AV_PIX_FMT_NV12;
>
>
> But when I am receiving the decoded AVFrame, I see that the AVFrame->format
> property is
> set to zero, which is enum for 420P. Also inspecting the planes and other
> props of this struct I see that the decoder tried to decode it as 420P. (3
> planes , 3 line sizes etc.) .And of course,when trying to play the output of
> such a YUV frame , there is just a green screen.
> I am trying to understand where is my error.Is there some data that decoder
> misses from the h264 stream and it doesn't allow it to guess the pixel
> format?Or maybe I need to set pixel format for the codec context in a some
> different way?
>

You can't control the output pixel format of the H264 decoder. A video
is not encoded as "NV12", its encoded as 4:2:0 YUV, NV12 is just one
representation of this format.
As such, our decoder will always decode 4:2:0 YUV content to yuv420p,
not nv12. If you require NV12 in your application, you can convert
from yuv420p to nv12 using swscale (a lossless conversion).

- Hendrik
_______________________________________________
Libav-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/libav-user


_______________________________________________
Libav-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/libav-user



_______________________________________________
Libav-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/libav-user
Reply | Threaded
Open this post in threaded view
|

Re: Decoding h264 (NV12) stream from NVENC

Michael IV
Well,Hendrik is right. The decoder knows to detect that the YUV is NV12. I do get the decoded YUV correct now,but not for the first frame.For some reason the first frame has completely green color. I do tell the NVENC to set SPSPPS at the beginning, but still the first frame gets corrupted. Also, how do I flip the frame vertically on decoding ? 

On Jul 31, 2017 2:07 PM, "jing zhang" <[hidden email]> wrote:
You should use decoded AVFrame->format to check decoded yuv format.
Maybe H.264 bitstream encoded by NVENC is corrupted?

2017-07-31 16:27 GMT+08:00 Michael IV <[hidden email]>:
So you basically say that the decoder 'knows' to detect interleaved UV in NV12 UV plane and decode it correctly to yuv420p without any hints from the application? 
Then why my YUV frame is corrupted? How can I detect where is the problem? I also found this SO answer regarding h264 streaming from NVENC.

And I currently don't set those params for encoder.Do I have to? 
 

On Mon, Jul 31, 2017 at 11:04 AM, Hendrik Leppkes <[hidden email]> wrote:
On Mon, Jul 31, 2017 at 10:00 AM, Michael IV <[hidden email]> wrote:
> Hi! I am using NVENC encoder to create h264 stream.The YUV format is NV12.
> Now, I need also to decode that stream back to YUV.I wrote a module for
> that,based on
> this example:
>
> https://gist.github.com/roxlu/9329339
>
> But, because my YUV format is not 420P, but NV12 I am trying to force it on
> AVCodecContext when initializing it with this line:
>
>         avcodec_register_all();
>         mCodec = avcodec_find_decoder(AV_CODEC_ID_H264);
>         mCodecContext = avcodec_alloc_context3(mCodec);
>         mCodecContext->pix_fmt = AVPixelFormat::AV_PIX_FMT_NV12;
>
>
> But when I am receiving the decoded AVFrame, I see that the AVFrame->format
> property is
> set to zero, which is enum for 420P. Also inspecting the planes and other
> props of this struct I see that the decoder tried to decode it as 420P. (3
> planes , 3 line sizes etc.) .And of course,when trying to play the output of
> such a YUV frame , there is just a green screen.
> I am trying to understand where is my error.Is there some data that decoder
> misses from the h264 stream and it doesn't allow it to guess the pixel
> format?Or maybe I need to set pixel format for the codec context in a some
> different way?
>

You can't control the output pixel format of the H264 decoder. A video
is not encoded as "NV12", its encoded as 4:2:0 YUV, NV12 is just one
representation of this format.
As such, our decoder will always decode 4:2:0 YUV content to yuv420p,
not nv12. If you require NV12 in your application, you can convert
from yuv420p to nv12 using swscale (a lossless conversion).

- Hendrik
_______________________________________________
Libav-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/libav-user


_______________________________________________
Libav-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/libav-user



_______________________________________________
Libav-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/libav-user



_______________________________________________
Libav-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/libav-user
Reply | Threaded
Open this post in threaded view
|

Re: Decoding h264 (NV12) stream from NVENC

Nicolas George
In reply to this post by Michael IV
Le tridi 13 thermidor, an CCXXV, Michael IV a écrit :
> So you basically say that the decoder 'knows' to detect interleaved UV in
> NV12 UV plane and decode it correctly to yuv420p without any hints from the
> application?

No, he means that YV12 and YUV420P are just two ways of putting the
pixels in memory for the encoder, but once they are encoded as H.264
they are both the same. And the FFmpeg decoder always decodes them into
YUV420P.

> On Mon, Jul 31, 2017 at 11:04 AM, Hendrik Leppkes <[hidden email]>
> wrote:

Please do not top-post on these mailing-lists. If you do not know what
it means look it up.

Regards,

--
  Nicolas George

_______________________________________________
Libav-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/libav-user

signature.asc (849 bytes) Download Attachment