[Libav-user] avcodec_receive_frame returns duplicate buffers.

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[Libav-user] avcodec_receive_frame returns duplicate buffers.

Hugh Waite
Hello,
I have an application that decodes an input stream into AVFrames and stores these in a ring buffer. On a trigger, I can create an output file containing video from the past N seconds until I stop.
I am seeing a problem with the output video that it sometimes contains duplicate frames and sometimes contains frames from "the future" (i.e. the wrong point in the ringbuffer). I have added debug and it appears that avcodec_receive_frame() sometimes returns the same data buffer for multiple decoded frames, even though I have not unref'ed the previous AVFrame.
Specifically I am checking that the returned av_frame->data[0] is not being used by any other AVFrames and this is failing.

This is on Ubuntu bionic (18.04) using the bionic backports of v4.1.3 and v4.2.0 from https://launchpad.net/~jonathonf/+archive/ubuntu/ffmpeg-4.
The source of the video stream is a v4l2 device.

Cut-down example of the frame retrieve function:
void *reader_run()
{
  AVFrame *ff = NULL;
  while (is_running)
  {
    AVPacket;
    av_init_packet(&pkt);
    ff = av_alloc_frame();

    /* Get packet from input */
    ret = av_read_frame(reader_fctx, &pkt);
    /* error checking ... */
    ret = avcodec_send_packet(decoder_cctx, &pkt);
    /* error checking ... */
    ret = avcodec_receive_frame(decoder_cctx, ff);
    /* error checking ... */

    /* Debug to check buffers in other frames */
    for (int i = 0; i< ringbuffer->size; i++)
    {
      if ringbuffer->items[i]->data[0] == ff->data[0]
      {
        ERROR("The buffer at %u is a duplicate when inserting %u %p", i, ringbuffer->next, ff->data[0]);
      }
    } /* End of debug */

    av_packet_unref(&pkt);
    rb_push(ringbuffer, ff); // Push onto head of ringbuffer and free oldest frame
  }
  /* Exiting. Flush decoder ... */
}

The buffer at 0 is a duplicate when inserting 1 0x7f1eb881e000
The buffer at 0 is a duplicate when inserting 2 0x7f1eb881e000
The buffer at 1 is a duplicate when inserting 2 0x7f1eb881e000
The buffer at 3 is a duplicate when inserting 301 0x7f1eb8a55000

The AVFrame's are not duplicated (new one alloc'ed each time), but they are being populated with the same "data[0]" as other AVFrames. Is there anything I am misunderstanding about these buffers?
This usually happens 2-3 times as soon as the decoder starts, and then again when the buffer is full and these frames are being freed. It is not the same buffer pointers each time.

This code is being run in a thread, but it is the only thread using this decoder. It happens even when not passing the frames to an encoder.

Can anyone see something that would cause this?

I am going to build my own copy of the libraries, but if this exhibits the same issue what can I check further to find the source of this problem (e.g. any data that would be useful in a bug report)

Many thanks,
Hugh

_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: avcodec_receive_frame returns duplicate buffers.

Devin Heitmueller
Hi Hugh,

> I have an application that decodes an input stream into AVFrames and stores these in a ring buffer. On a trigger, I can create an output file containing video from the past N seconds until I stop.
> I am seeing a problem with the output video that it sometimes contains duplicate frames and sometimes contains frames from "the future" (i.e. the wrong point in the ringbuffer). I have added debug and it appears that avcodec_receive_frame() sometimes returns the same data buffer for multiple decoded frames, even though I have not unref'ed the previous AVFrame.
> Specifically I am checking that the returned av_frame->data[0] is not being used by any other AVFrames and this is failing.

Just a couple of observations which might point you in the right direction...

So I am not an expert on the specifics of libavdevice's implementation
for v4l2, but I did take a quick look at the code (and I've worked on
plenty of other v4l2 based apps over the years).  In order to avoid
memory copies, the avpackets data[0] points to the underlying mmap'd
buffer provided by the kernel.  In other words data[0] will only ever
point to a small number of specific memory addresses (e.g. actual
number dependent on the specific v4l2 device, but somewhere between 3
to 8 is typical).  This is generally a good thing, but if your
reference counter is screwed up then you will see results similar to
where you described since the buffers will be reused by the driver as
soon as they are released.

What decoder are you using?  If it's one that involves no format or
colorspace conversion, I wouldn't be shocked to find that it just
unwraps the AVPacket into an AVFrame and references the same
underlying buffer.  I would probably start by looking there and see
what gets done with the reference counting within the decoder.

In cases like this I've also found it helpful to be feeding video with
some form of on-screen timecode or incrementing frame counter.  That
makes it much more easy to spot out-of-order issues which might be
less obvious with real video.

Devin

--
Devin J. Heitmueller - Kernel Labs
http://www.kernellabs.com
_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: avcodec_receive_frame returns duplicate buffers.

Hugh Waite
Thanks Devin,
Your comment about the kernel buffers makes sense as there are only 8 addresses that turn up as being reused in my debug output.
There is some memory copying going on, because I only occasionally get duplicate buffers, even through there are 900 frames stored in my 30s ring-buffer.

I have fixed / worked around this issue by allocating new buffers and copying the frame contents (av_get_buffer + av_frame_copy) just after the avcodec_receive_frame.

Regards,
Hugh


On Mon, 26 Aug 2019 at 16:03, Devin Heitmueller <[hidden email]> wrote:
Hi Hugh,

> I have an application that decodes an input stream into AVFrames and stores these in a ring buffer. On a trigger, I can create an output file containing video from the past N seconds until I stop.
> I am seeing a problem with the output video that it sometimes contains duplicate frames and sometimes contains frames from "the future" (i.e. the wrong point in the ringbuffer). I have added debug and it appears that avcodec_receive_frame() sometimes returns the same data buffer for multiple decoded frames, even though I have not unref'ed the previous AVFrame.
> Specifically I am checking that the returned av_frame->data[0] is not being used by any other AVFrames and this is failing.

Just a couple of observations which might point you in the right direction...

So I am not an expert on the specifics of libavdevice's implementation
for v4l2, but I did take a quick look at the code (and I've worked on
plenty of other v4l2 based apps over the years).  In order to avoid
memory copies, the avpackets data[0] points to the underlying mmap'd
buffer provided by the kernel.  In other words data[0] will only ever
point to a small number of specific memory addresses (e.g. actual
number dependent on the specific v4l2 device, but somewhere between 3
to 8 is typical).  This is generally a good thing, but if your
reference counter is screwed up then you will see results similar to
where you described since the buffers will be reused by the driver as
soon as they are released.

What decoder are you using?  If it's one that involves no format or
colorspace conversion, I wouldn't be shocked to find that it just
unwraps the AVPacket into an AVFrame and references the same
underlying buffer.  I would probably start by looking there and see
what gets done with the reference counting within the decoder.

In cases like this I've also found it helpful to be feeding video with
some form of on-screen timecode or incrementing frame counter.  That
makes it much more easy to spot out-of-order issues which might be
less obvious with real video.

Devin

--
Devin J. Heitmueller - Kernel Labs
http://www.kernellabs.com
_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".

_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".