libav-users

Frame accurate seeking on H.264 video streams

Classic

List

Threaded

9 messages Options

mediasystems

Frame accurate seeking on H.264 video streams

Hello,

I use ffmpeg for about one year with MPEG-2 streams. Everything works fine
so far. Now I want to use H.264 coded videos as well. I'm trying to do a
frame accurate random seeking on H.264 video streams. But I cannot
accomplish to decode all single frames without any artefacts after calling
av_seek_frame. If I go through the stream from the beginning, it works. Is
it generally possible to do frame accurate seeking on H.264 video streams
and how?

Code snippet:

ret = av_seek_frame(pFormatCtx, videoStreamIndex, timestamp,
AVSEEK_FLAG_BACKWARD); avcodec_flush_buffers(pCodecCtx);

...
while (av_read_frame(pFormatCtx, pPacket) >= 0) { ...
bytesDecoded = avcodec_decode_video(pCodecCtx, pFrame, ref
frameFinished, packet.data, packet.size); ...

Any help would be appreciated!
Many thanks.

Florian Hallberg

_______________________________________________
libav-user mailing list
[hidden email]
https://lists.mplayerhq.hu/mailman/listinfo/libav-user

Scott Harper-5

Re: Frame accurate seeking on H.264 video streams

On Jun 26, 2009, at 10:57 AM, Mediasystems GmbH wrote:

> Hello,
>
> I use ffmpeg for about one year with MPEG-2 streams. Everything
> works fine
> so far. Now I want to use H.264 coded videos as well. I'm trying to
> do a
> frame accurate random seeking on H.264 video streams. But I cannot
> accomplish to decode all single frames without any artefacts after
> calling
> av_seek_frame. If I go through the stream from the beginning, it
> works. Is
> it generally possible to do frame accurate seeking on H.264 video
> streams
> and how?

I think for this you need to seek to an I-frame, as H.264 is a
progressive encoder, and some of the frames only contain the changes
from the last frame to the current one. Once you have the previous I-
frame, you can then keep grabbing and decoding frames until you reach
the desired frame. This is how I was able to implement frame seeking
in one of my simple projects from a while ago.

>
> Code snippet:
>
> ret = av_seek_frame(pFormatCtx, videoStreamIndex, timestamp,
> AVSEEK_FLAG_BACKWARD); avcodec_flush_buffers(pCodecCtx);
>
> ...
> while (av_read_frame(pFormatCtx, pPacket) >= 0) { ...
> bytesDecoded = avcodec_decode_video(pCodecCtx, pFrame, ref
> frameFinished, packet.data, packet.size); ...
>
>
> Any help would be appreciated!
> Many thanks.

_______________________________________________
libav-user mailing list
[hidden email]
https://lists.mplayerhq.hu/mailman/listinfo/libav-user

mediasystems

Re: Frame accurate seeking on H.264 video streams

Right, exactly that way I tried it. However, I assumed that av_seek_frame
with flag AVSEEK_FLAG_BACKWARD set should seek to the previous I-frame. Am I
wrong here? If av_seek_frame doesn't work like I thought, is there another
libav function to achieve this?

On Jun 26, 2009, at 10:57 AM, Mediasystems GmbH wrote:

ejlim

Re: Frame accurate seeking on H.264 video streams

Hi.

I recommend avformat_seek_file() in avformat/util.c, because this function
use new semantics of various demuxer.
And u will reference av_seek_frame_generic().

Internally, AVSEEK_FLAG_BACKWARD flag use
AVFormatContext->streams[idxVideo]->nb_index_entries.
BTW, some container have not this indexes. If this case, u can use
AVSEEK_FLAG_BACKWARD and sequenced searching method.

GL~~

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of Mediasystems GmbH
Sent: Wednesday, July 01, 2009 2:57 AM
To: 'Libav* user questions and discussions'
Subject: Re: [libav-user] Frame accurate seeking on H.264 video streams

Right, exactly that way I tried it. However, I assumed that av_seek_frame
with flag AVSEEK_FLAG_BACKWARD set should seek to the previous I-frame. Am I
wrong here? If av_seek_frame doesn't work like I thought, is there another
libav function to achieve this?

On Jun 26, 2009, at 10:57 AM, Mediasystems GmbH wrote:

_______________________________________________
libav-user mailing list
[hidden email]
https://lists.mplayerhq.hu/mailman/listinfo/libav-user

_______________________________________________
libav-user mailing list
[hidden email]
https://lists.mplayerhq.hu/mailman/listinfo/libav-user

_______________________________________________
libav-user mailing list
[hidden email]
https://lists.mplayerhq.hu/mailman/listinfo/libav-user

mediasystems

Re: Frame accurate seeking on H.264 video streams

Hi.

Thanks for your recommendations. I tried it with avformat_seek_file, but
couldn't find any difference to av_seek_frame. With avformat_seek_file I'm
also getting artifacts on decoding H.264 video streams. How I have to call
this function to seek to the right I-frame and to decode frames without any
artifacts? Is there perhaps a sample application with which I could test it?

What do you mean with sequenced searching method?

Thanks.

-----Ursprüngliche Nachricht-----
Von: [hidden email]
[mailto:[hidden email]] Im Auftrag von ejlim
Gesendet: Mittwoch, 1. Juli 2009 03:53
An: 'Libav* user questions and discussions'
Betreff: Re: [libav-user] Frame accurate seeking on H.264 video streams

Hi.

I recommend avformat_seek_file() in avformat/util.c, because this function
use new semantics of various demuxer.
And u will reference av_seek_frame_generic().

Internally, AVSEEK_FLAG_BACKWARD flag use
AVFormatContext->streams[idxVideo]->nb_index_entries.
BTW, some container have not this indexes. If this case, u can use
AVSEEK_FLAG_BACKWARD and sequenced searching method.

GL~~

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of Mediasystems GmbH
Sent: Wednesday, July 01, 2009 2:57 AM
To: 'Libav* user questions and discussions'
Subject: Re: [libav-user] Frame accurate seeking on H.264 video streams

Right, exactly that way I tried it. However, I assumed that av_seek_frame
with flag AVSEEK_FLAG_BACKWARD set should seek to the previous I-frame. Am I
wrong here? If av_seek_frame doesn't work like I thought, is there another
libav function to achieve this?

On Jun 26, 2009, at 10:57 AM, Mediasystems GmbH wrote:

Scott Andrew-3

Re: Frame accurate seeking on H.264 video streams

For MPEG2 what I did was do a seek to frame.. Then do a tight loop of
reading and decoding until I got a picture. Storing PTS when I finally got
my picture so my next seek didn't go beyond. I basically seeked to the
closest picture. Not necessarily frame accurate. Again this seek only gets
me to the closet picture. Frame accuracy was hard for me because the files I
was dealing with had re-ordered b-frames (so some bframes were behind the
iframe). So we settled for the closest picture we could decode. We didn't
need to be frame accurate.

One other thing... Make sure when seeking that you also flush your buffers.
Any residual data in you video decoder buffer will be considered when
decoding and will get you artifacts.

On 7/7/09 10:50 AM, "Mediasystems GmbH" <[hidden email]> wrote:

> Hi.
>
> Thanks for your recommendations. I tried it with avformat_seek_file, but
> couldn't find any difference to av_seek_frame. With avformat_seek_file I'm
> also getting artifacts on decoding H.264 video streams. How I have to call
> this function to seek to the right I-frame and to decode frames without any
> artifacts? Is there perhaps a sample application with which I could test it?
>
> What do you mean with sequenced searching method?
>
>
> Thanks.
>
>
> -----Ursprüngliche Nachricht-----
> Von: [hidden email]
> [mailto:[hidden email]] Im Auftrag von ejlim
> Gesendet: Mittwoch, 1. Juli 2009 03:53
> An: 'Libav* user questions and discussions'
> Betreff: Re: [libav-user] Frame accurate seeking on H.264 video streams
>
> Hi.
>
> I recommend avformat_seek_file() in avformat/util.c, because this function
> use new semantics of various demuxer.
> And u will reference av_seek_frame_generic().
>
> Internally, AVSEEK_FLAG_BACKWARD flag use
> AVFormatContext->streams[idxVideo]->nb_index_entries.
> BTW, some container have not this indexes. If this case, u can use
> AVSEEK_FLAG_BACKWARD and sequenced searching method.
>
> GL~~
>
> -----Original Message-----
> From: [hidden email]
> [mailto:[hidden email]] On Behalf Of Mediasystems GmbH
> Sent: Wednesday, July 01, 2009 2:57 AM
> To: 'Libav* user questions and discussions'
> Subject: Re: [libav-user] Frame accurate seeking on H.264 video streams
>
> Right, exactly that way I tried it. However, I assumed that av_seek_frame
> with flag AVSEEK_FLAG_BACKWARD set should seek to the previous I-frame. Am I
> wrong here? If av_seek_frame doesn't work like I thought, is there another
> libav function to achieve this?
>
>
> On Jun 26, 2009, at 10:57 AM, Mediasystems GmbH wrote:
>
>> Hello,
>>
>> I use ffmpeg for about one year with MPEG-2 streams. Everything
>> works fine
>> so far. Now I want to use H.264 coded videos as well. I'm trying to
>> do a
>> frame accurate random seeking on H.264 video streams. But I cannot
>> accomplish to decode all single frames without any artefacts after
>> calling
>> av_seek_frame. If I go through the stream from the beginning, it
>> works. Is
>> it generally possible to do frame accurate seeking on H.264 video
>> streams
>> and how?
>
> I think for this you need to seek to an I-frame, as H.264 is a
> progressive encoder, and some of the frames only contain the changes
> from the last frame to the current one. Once you have the previous I-
> frame, you can then keep grabbing and decoding frames until you reach
> the desired frame. This is how I was able to implement frame seeking
> in one of my simple projects from a while ago.
>
>>
>> Code snippet:
>>
>> ret = av_seek_frame(pFormatCtx, videoStreamIndex, timestamp,
>> AVSEEK_FLAG_BACKWARD); avcodec_flush_buffers(pCodecCtx);
>>
>> ...
>> while (av_read_frame(pFormatCtx, pPacket) >= 0) { ...
>> bytesDecoded = avcodec_decode_video(pCodecCtx, pFrame, ref
>> frameFinished, packet.data, packet.size); ...
>>
>>
>> Any help would be appreciated!
>> Many thanks.
> _______________________________________________
> libav-user mailing list
> [hidden email]
> https://lists.mplayerhq.hu/mailman/listinfo/libav-user
>
>
> _______________________________________________
> libav-user mailing list
> [hidden email]
> https://lists.mplayerhq.hu/mailman/listinfo/libav-user
>
> _______________________________________________
> libav-user mailing list
> [hidden email]
> https://lists.mplayerhq.hu/mailman/listinfo/libav-user
>
>
> _______________________________________________
> libav-user mailing list
> [hidden email]
> https://lists.mplayerhq.hu/mailman/listinfo/libav-user

_______________________________________________
libav-user mailing list
[hidden email]
https://lists.mplayerhq.hu/mailman/listinfo/libav-user

mediasystems

Re: Frame accurate seeking on H.264 video streams

> For MPEG2 what I did was do a seek to frame.. Then do a tight loop of
> reading and decoding until I got a picture.

I also do it like that. With MPEG2 streams it works.

> me to the closet picture. Frame accuracy was hard for me because the files
I
> was dealing with had re-ordered b-frames (so some bframes were behind the
> iframe). So we settled for the closest picture we could decode. We didn't

I think this is the point. It seems, that seeking functions disregards
re-ordered b-frames.

> One other thing... Make sure when seeking that you also flush your
buffers.
> Any residual data in you video decoder buffer will be considered when
> decoding and will get you artifacts.

Right. I'm calling avcodec_flush_buffers prior to decoding. Are there other
functions which I should call?

Now I probably have a proof of fact that frame accurate seeking on H.264
video (transport) streams is apparently not possible with ffmpeg until now:
I tried out VLC media player V 1.0.0 and noticed that VLC also has problems
with frame accurate seeking. When I open H.264 mts files and scroll the
slider most frames are not decoded properly and show artifacts. Under
assumption that VLC uses ffmpeg to decode H.264 I can assume that frame
accurate seeking on H.264 doesn't work with ffmpeg. Can anybody confirm
that?

-----Ursprüngliche Nachricht-----
Von: [hidden email]
[mailto:[hidden email]] Im Auftrag von Scott Andrew
Gesendet: Dienstag, 7. Juli 2009 20:00
An: Libav* user questions and discussions
Betreff: Re: [libav-user] Frame accurate seeking on H.264 video streams

For MPEG2 what I did was do a seek to frame.. Then do a tight loop of
reading and decoding until I got a picture. Storing PTS when I finally got
my picture so my next seek didn't go beyond. I basically seeked to the
closest picture. Not necessarily frame accurate. Again this seek only gets
me to the closet picture. Frame accuracy was hard for me because the files I
was dealing with had re-ordered b-frames (so some bframes were behind the
iframe). So we settled for the closest picture we could decode. We didn't
need to be frame accurate.

One other thing... Make sure when seeking that you also flush your buffers.
Any residual data in you video decoder buffer will be considered when
decoding and will get you artifacts.

On 7/7/09 10:50 AM, "Mediasystems GmbH" <[hidden email]> wrote:

> Hi.
>
> Thanks for your recommendations. I tried it with avformat_seek_file, but
> couldn't find any difference to av_seek_frame. With avformat_seek_file I'm
> also getting artifacts on decoding H.264 video streams. How I have to call
> this function to seek to the right I-frame and to decode frames without
any
> artifacts? Is there perhaps a sample application with which I could test
it?

>
> What do you mean with sequenced searching method?
>
>
> Thanks.
>
>
> -----Ursprüngliche Nachricht-----
> Von: [hidden email]
> [mailto:[hidden email]] Im Auftrag von ejlim
> Gesendet: Mittwoch, 1. Juli 2009 03:53
> An: 'Libav* user questions and discussions'
> Betreff: Re: [libav-user] Frame accurate seeking on H.264 video streams
>
> Hi.
>
> I recommend avformat_seek_file() in avformat/util.c, because this function
> use new semantics of various demuxer.
> And u will reference av_seek_frame_generic().
>
> Internally, AVSEEK_FLAG_BACKWARD flag use
> AVFormatContext->streams[idxVideo]->nb_index_entries.
> BTW, some container have not this indexes. If this case, u can use
> AVSEEK_FLAG_BACKWARD and sequenced searching method.
>
> GL~~
>
> -----Original Message-----
> From: [hidden email]
> [mailto:[hidden email]] On Behalf Of Mediasystems GmbH
> Sent: Wednesday, July 01, 2009 2:57 AM
> To: 'Libav* user questions and discussions'
> Subject: Re: [libav-user] Frame accurate seeking on H.264 video streams
>
> Right, exactly that way I tried it. However, I assumed that av_seek_frame
> with flag AVSEEK_FLAG_BACKWARD set should seek to the previous I-frame. Am

> wrong here? If av_seek_frame doesn't work like I thought, is there another
> libav function to achieve this?
>
>
> On Jun 26, 2009, at 10:57 AM, Mediasystems GmbH wrote:
>
>> Hello,
>>
>> I use ffmpeg for about one year with MPEG-2 streams. Everything
>> works fine
>> so far. Now I want to use H.264 coded videos as well. I'm trying to
>> do a
>> frame accurate random seeking on H.264 video streams. But I cannot
>> accomplish to decode all single frames without any artefacts after
>> calling
>> av_seek_frame. If I go through the stream from the beginning, it
>> works. Is
>> it generally possible to do frame accurate seeking on H.264 video
>> streams
>> and how?
>
> I think for this you need to seek to an I-frame, as H.264 is a
> progressive encoder, and some of the frames only contain the changes
> from the last frame to the current one. Once you have the previous I-
> frame, you can then keep grabbing and decoding frames until you reach
> the desired frame. This is how I was able to implement frame seeking
> in one of my simple projects from a while ago.
>
>>
>> Code snippet:
>>
>> ret = av_seek_frame(pFormatCtx, videoStreamIndex, timestamp,
>> AVSEEK_FLAG_BACKWARD); avcodec_flush_buffers(pCodecCtx);
>>
>> ...
>> while (av_read_frame(pFormatCtx, pPacket) >= 0) { ...
>> bytesDecoded = avcodec_decode_video(pCodecCtx, pFrame, ref
>> frameFinished, packet.data, packet.size); ...
>>
>>
>> Any help would be appreciated!
>> Many thanks.
> _______________________________________________
> libav-user mailing list
> [hidden email]
> https://lists.mplayerhq.hu/mailman/listinfo/libav-user
>
>
> _______________________________________________
> libav-user mailing list
> [hidden email]
> https://lists.mplayerhq.hu/mailman/listinfo/libav-user
>
> _______________________________________________
> libav-user mailing list
> [hidden email]
> https://lists.mplayerhq.hu/mailman/listinfo/libav-user
>
>
> _______________________________________________
> libav-user mailing list
> [hidden email]
> https://lists.mplayerhq.hu/mailman/listinfo/libav-user

Stephan Assmus

Re: Frame accurate seeking on H.264 video streams

Hi,

On 2009-07-08 at 19:42:01 [+0200], Mediasystems GmbH <[hidden email]>
wrote:
> Now I probably have a proof of fact that frame accurate seeking on H.264
> video (transport) streams is apparently not possible with ffmpeg until
> now: I tried out VLC media player V 1.0.0 and noticed that VLC also has
> problems with frame accurate seeking. When I open H.264 mts files and
> scroll the slider most frames are not decoded properly and show
> artifacts. Under assumption that VLC uses ffmpeg to decode H.264 I can
> assume that frame accurate seeking on H.264 doesn't work with ffmpeg. Can
> anybody confirm that?

From what I heard, H.264 can reference frames completely outside of the
current GOP, which may be the problem here. If those are flushed with the
rest of the buffers, or haven't even been decoded yet, then it should be a
problem. I am using the API to give the time stamp for a keyframe backwards
of a requested time stamp, but as far as I can tell, the code searches the
index to find the next timestamp, it doesn't seem to take into account more
advanced requirements of the current decoder.

Best regards,
-Stephan
_______________________________________________
libav-user mailing list
[hidden email]
https://lists.mplayerhq.hu/mailman/listinfo/libav-user

SyRenity

Re: Frame accurate seeking on H.264 video streams

Hi.

My 2 cents on this that many other decoder implementations seems to have
same problem

For example, the Flash 10 decoder has these issues - it's especially visible
on YouTube movies.

Regards.

2009/7/8 Stephan Assmus <[hidden email]>

> Hi,
>
> On 2009-07-08 at 19:42:01 [+0200], Mediasystems GmbH <[hidden email]
> >
> wrote:
> > Now I probably have a proof of fact that frame accurate seeking on H.264
> > video (transport) streams is apparently not possible with ffmpeg until
> > now: I tried out VLC media player V 1.0.0 and noticed that VLC also has
> > problems with frame accurate seeking. When I open H.264 mts files and
> > scroll the slider most frames are not decoded properly and show
> > artifacts. Under assumption that VLC uses ffmpeg to decode H.264 I can
> > assume that frame accurate seeking on H.264 doesn't work with ffmpeg. Can
> > anybody confirm that?
>
> From what I heard, H.264 can reference frames completely outside of the
> current GOP, which may be the problem here. If those are flushed with the
> rest of the buffers, or haven't even been decoded yet, then it should be a
> problem. I am using the API to give the time stamp for a keyframe backwards
> of a requested time stamp, but as far as I can tell, the code searches the
> index to find the next timestamp, it doesn't seem to take into account more
> advanced requirements of the current decoder.
>
> Best regards,
> -Stephan
> _______________________________________________
> libav-user mailing list
> [hidden email]
> https://lists.mplayerhq.hu/mailman/listinfo/libav-user
>

_______________________________________________
libav-user mailing list
[hidden email]
https://lists.mplayerhq.hu/mailman/listinfo/libav-user