[Libav-user] How to corretly setting container and stream information when using an external encoder and using libavformat to pack the movie container

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[Libav-user] How to corretly setting container and stream information when using an external encoder and using libavformat to pack the movie container

Ingmar Rieger
Hello,

I'm currently working on integrating Cineform file read and write into a
small toy project for processing image sequences. Worked with exr
sequences up until now but those are so storage heavy I decided to
include a intermediate video format and with it being the only freely
available one I decided to give cineform a go.

Therefore I implemented a small file reader first using libavformat to
open video containers and unpacking the stream while passing the frames
to the Cineform-sdk for decompressing to have the reference decoder.
Works fine so far.

Now I'm trying the other way round and pack the frames with the cineform
encoder and then create a movie container. I tried several approaches
from several samples and ours of going through the Doxygen documentation
and google searches etc.

I'm able to write a file that works correct with my simple reader that
will just extract package for package from the video stream and gets the
number of frames right and the result looks fine. But there are still a
few points I haven't managed to get to work correctly related to
metadata of the container and the stream:

1. Set correct container and stream metadata. I copied the codec
parameters I extracted from a read cineform example (exported from
Blackmagic DaVinci Resolve) to begin with as I found no better solution
for this use case yet.
  - Bit rate is just the value from one sample and depends very much on
the input resolution, I think the value is from a 4K sample file. Does
this really matter? The value reported from "ffmpeg -i testout.mov" is
completely different anyway, I assume it is calculated when calling
"av_write_trailer"?
2. Correct frame rate/time base settings. Tried several examples, tried
replicating what I extracted from them and made of the documentation
regarding libav I found about this topic but as the link below (ffmpeg
-i output) shows, the framerate is always wrong. Had combinations of
parameters where I had clips with 16k fps etc.
3. Setting start time code of the stream/clip like in the example.
4. Set stream metadata like encoder information.

Can somebody give me a hint how to set those data to have valid, well
formed files?

I have the basic structure of calls I do to libav extracted from my
framework and packed it in a gist for better readability:
https://gist.github.com/irieger/604125fba000f76095626696f34ae619

Also there is the output of an "ffmpeg -i" call for each the reference
file and the generated output in the file ffmpeg.

Looking forward to some tips how to tackle this task.

--
Kind regards,
Ingmar
_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: How to corretly setting container and stream information when using an external encoder and using libavformat to pack the movie container

Carl Eugen Hoyos-2
Am Sa., 26. Okt. 2019 um 10:26 Uhr schrieb Ingmar Rieger <[hidden email]>:

> I'm currently working on integrating Cineform file read and write into a
> small toy project for processing image sequences. Worked with exr
> sequences up until now but those are so storage heavy I decided to

> include a intermediate video format and with it being the only freely
> available one I decided to give cineform a go.

That's at least a surprising argumentation given the number of encoders
included in FFmpeg.

> Therefore I implemented a small file reader first using libavformat to
> open video containers and unpacking the stream while passing the frames
> to the Cineform-sdk for decompressing to have the reference decoder.
> Works fine so far.
>
> Now I'm trying the other way round and pack the frames with the cineform
> encoder and then create a movie container. I tried several approaches
> from several samples and ours of going through the Doxygen documentation
> and google searches etc.
>
> I'm able to write a file that works correct with my simple reader that
> will just extract package for package from the video stream and gets the
> number of frames right and the result looks fine. But there are still a
> few points I haven't managed to get to work correctly related to
> metadata of the container and the stream:
>
> 1. Set correct container and stream metadata. I copied the codec
> parameters I extracted from a read cineform example (exported from
> Blackmagic DaVinci Resolve) to begin with as I found no better solution
> for this use case yet.

>   - Bit rate is just the value from one sample and depends very much on
> the input resolution, I think the value is from a 4K sample file. Does
> this really matter? The value reported from "ffmpeg -i testout.mov" is
> completely different anyway, I assume it is calculated when calling
> "av_write_trailer"?

And is the written value correct?

> 2. Correct frame rate/time base settings. Tried several examples, tried
> replicating what I extracted from them and made of the documentation
> regarding libav I found about this topic but as the link below (ffmpeg
> -i output) shows, the framerate is always wrong. Had combinations of
> parameters where I had clips with 16k fps etc.

You have to read back the actual timebase used by the muxer (can
be different than the one you requested) and calculate the timestamps
accordingly.

Carl Eugen
_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: How to corretly setting container and stream information when using an external encoder and using libavformat to pack the movie container

Paul B Mahol
In reply to this post by Ingmar Rieger
On 10/26/19, Ingmar Rieger <[hidden email]> wrote:

> Hello,
>
> I'm currently working on integrating Cineform file read and write into a
> small toy project for processing image sequences. Worked with exr
> sequences up until now but those are so storage heavy I decided to
> include a intermediate video format and with it being the only freely
> available one I decided to give cineform a go.
>
> Therefore I implemented a small file reader first using libavformat to
> open video containers and unpacking the stream while passing the frames
> to the Cineform-sdk for decompressing to have the reference decoder.

Using native cfhd decoder should be faster and better.

> Works fine so far.
>
> Now I'm trying the other way round and pack the frames with the cineform
> encoder and then create a movie container. I tried several approaches
> from several samples and ours of going through the Doxygen documentation
> and google searches etc.
>
> I'm able to write a file that works correct with my simple reader that
> will just extract package for package from the video stream and gets the
> number of frames right and the result looks fine. But there are still a
> few points I haven't managed to get to work correctly related to
> metadata of the container and the stream:
>
> 1. Set correct container and stream metadata. I copied the codec
> parameters I extracted from a read cineform example (exported from
> Blackmagic DaVinci Resolve) to begin with as I found no better solution
> for this use case yet.
>   - Bit rate is just the value from one sample and depends very much on
> the input resolution, I think the value is from a 4K sample file. Does
> this really matter? The value reported from "ffmpeg -i testout.mov" is
> completely different anyway, I assume it is calculated when calling
> "av_write_trailer"?
> 2. Correct frame rate/time base settings. Tried several examples, tried
> replicating what I extracted from them and made of the documentation
> regarding libav I found about this topic but as the link below (ffmpeg
> -i output) shows, the framerate is always wrong. Had combinations of
> parameters where I had clips with 16k fps etc.
> 3. Setting start time code of the stream/clip like in the example.
> 4. Set stream metadata like encoder information.
>
> Can somebody give me a hint how to set those data to have valid, well
> formed files?
>
> I have the basic structure of calls I do to libav extracted from my
> framework and packed it in a gist for better readability:
> https://gist.github.com/irieger/604125fba000f76095626696f34ae619
>
> Also there is the output of an "ffmpeg -i" call for each the reference
> file and the generated output in the file ffmpeg.
>
> Looking forward to some tips how to tackle this task.
>
> --
> Kind regards,
> Ingmar
> _______________________________________________
> Libav-user mailing list
> [hidden email]
> https://ffmpeg.org/mailman/listinfo/libav-user
>
> To unsubscribe, visit link above, or email
> [hidden email] with subject "unsubscribe".
_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: How to corretly setting container and stream information when using an external encoder and using libavformat to pack the movie container

Ingmar Rieger
In reply to this post by Carl Eugen Hoyos-2
Hey,

On 26.10.19 10:59, Carl Eugen Hoyos wrote:
> That's at least a surprising argumentation given the number of encoders
> included in FFmpeg.
Did I miss something? For me the relevant codecs I know for intermediate
work are ProRes, DNxHR and Cineform with the later being seldomly used
but the only open standard and as I recently found out supported in
Resolve for some time now so as my main target is Resolve it seems worth
giving it a try. ProRes and DNxHR are both proprietary with only reverse
engineered codecs available.

>>    - Bit rate is just the value from one sample and depends very much on
>> the input resolution, I think the value is from a 4K sample file. Does
>> this really matter? The value reported from "ffmpeg -i testout.mov" is
>> completely different anyway, I assume it is calculated when calling
>> "av_write_trailer"?
>
> And is the written value correct?
Looking at my example with just below 800 KB per frame and 5 frames
written (at 120fps) and calculating it it makes sense. Also after retime
without reencoding to 5 fps it matches the file size. Was just wondering
if this is something one has to take care of or if it will be done by
libav based on actual data?

>> 2. Correct frame rate/time base settings. Tried several examples, tried
>> replicating what I extracted from them and made of the documentation
>> regarding libav I found about this topic but as the link below (ffmpeg
>> -i output) shows, the framerate is always wrong. Had combinations of
>> parameters where I had clips with 16k fps etc.
>
> You have to read back the actual timebase used by the muxer (can
> be different than the one you requested) and calculate the timestamps
> accordingly.
Is there a good introduction how to correctly set time bases and
calculate frame numbers/timecodes from it?

Whatever I change I get different frame rates but never what I intend
like in this example 24fps. (This shall be variable later on but would
be nice to at least understand it)

So basically I have an input like frame_rate (in integer for simplicity,
don't care about the crazy 23.98 and other strange rates), a frame
number from 0 to NUM_FRAMES-1 and ideally a start time input -> How to
set the correct time base/rates and calculate the frame dts/pts?


--
Kind regards,
Ingmar
_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: How to corretly setting container and stream information when using an external encoder and using libavformat to pack the movie container

Carl Eugen Hoyos-2
Am Sa., 26. Okt. 2019 um 14:14 Uhr schrieb Ingmar Rieger <[hidden email]>:

> On 26.10.19 10:59, Carl Eugen Hoyos wrote:
> > That's at least a surprising argumentation given the number of encoders
> > included in FFmpeg.
>
> Did I miss something? For me the relevant codecs I know for intermediate
> work are ProRes, DNxHR and Cineform with the later being seldomly used
> but the only open standard and as I recently found out supported in
> Resolve for some time now so as my main target is Resolve it seems worth
> giving it a try. ProRes and DNxHR are both proprietary with only reverse
> engineered codecs available.

Correct:
FFmpeg contains encoders for both ProRes and DNxHR.
(Note that Cineform being an open standard is a joke, it is exactly as
proprietary as the others.)

[...]

> So basically I have an input like frame_rate (in integer for simplicity,
> don't care about the crazy 23.98 and other strange rates), a frame
> number from 0 to NUM_FRAMES-1 and ideally a start time input -> How to
> set the correct time base/rates and calculate the frame dts/pts?

As said, you cannot set the time base (you can but you don't have to) but
you have to read the time base used by the muxer, then calculate timestamps
using this time base, should be possible with av_rescale_q().

Carl Eugen
_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: How to corretly setting container and stream information when using an external encoder and using libavformat to pack the movie container

Ingmar Rieger


On 26.10.19 14:37, Carl Eugen Hoyos wrote:

> Am Sa., 26. Okt. 2019 um 14:14 Uhr schrieb Ingmar Rieger <[hidden email]>:
>
>> On 26.10.19 10:59, Carl Eugen Hoyos wrote:
>>> That's at least a surprising argumentation given the number of encoders
>>> included in FFmpeg.
>>
>> Did I miss something? For me the relevant codecs I know for intermediate
>> work are ProRes, DNxHR and Cineform with the later being seldomly used
>> but the only open standard and as I recently found out supported in
>> Resolve for some time now so as my main target is Resolve it seems worth
>> giving it a try. ProRes and DNxHR are both proprietary with only reverse
>> engineered codecs available.
>
> Correct:
> FFmpeg contains encoders for both ProRes and DNxHR.
> (Note that Cineform being an open standard is a joke, it is exactly as
> proprietary as the others.)
>
> [...]
Ok you are right. But still Apple is evil etc. (Blackmagic accidentially
activated ProRes in a beta version showing everyone what is already
known: There is quicktime support ready...)

Still this discussion helps nothing regarding writing the correct
container metadata.

>> So basically I have an input like frame_rate (in integer for simplicity,
>> don't care about the crazy 23.98 and other strange rates), a frame
>> number from 0 to NUM_FRAMES-1 and ideally a start time input -> How to
>> set the correct time base/rates and calculate the frame dts/pts?
>
> As said, you cannot set the time base (you can but you don't have to) but
> you have to read the time base used by the muxer, then calculate timestamps
> using this time base, should be possible with av_rescale_q().

Ok, so what do I set and what will be set automatically in general?
As I said I found no documentation describing how to setup a stream
manually, that is why I asked in the first place. Just used the remux
example (which uses avcodec_parameters_copy) as the base and manually
read information from an existing Cineform encoded by Blackmagic to set
the parameters manually.

With
 > out_avstream->avg_frame_rate = (AVRational){24, 1};
 > out_avstream->r_frame_rate   = out_avstream->avg_frame_rate;
I now have to correct frame rate shown when av_dump_format is called but
after writing some frames the frame rate seems to be reset.

My current code that tries to set correct pts/dts values:
 > AVRational timebase_clip  = out_avstream->time_base;
 > AVRational timebase_count = (AVRational){1, 24}; // also tried {24, 1}
 > pkg.pts  = av_rescale_q(frame, timebase_count, timebase_clip);
 > pkg.dts  = pkg.dts;

This results in errors displayed by libav:
 > [mov @ 0x55b6572cfbc0] Application provided invalid, non
monotonically increasing dts to muxer in stream 0: 6 >= 6

So what is the right way to go from frame number to correct timebase
conversion. Can someone just give a hint how to correctly call this
conversions?
 From the outputs and everything I would say I seem to be near a solution.

--
Kind regards,
Ingmar
_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: How to corretly setting container and stream information when using an external encoder and using libavformat to pack the movie container

Carl Eugen Hoyos-2
Am Sa., 26. Okt. 2019 um 20:57 Uhr schrieb Ingmar Rieger <[hidden email]>:
>
> On 26.10.19 14:37, Carl Eugen Hoyos wrote:
> > Am Sa., 26. Okt. 2019 um 14:14 Uhr schrieb Ingmar Rieger <[hidden email]>:
> >
> >> On 26.10.19 10:59, Carl Eugen Hoyos wrote:

> >> So basically I have an input like frame_rate (in integer for simplicity,
> >> don't care about the crazy 23.98 and other strange rates), a frame
> >> number from 0 to NUM_FRAMES-1 and ideally a start time input -> How to
> >> set the correct time base/rates and calculate the frame dts/pts?
> >
> > As said, you cannot set the time base (you can but you don't have to) but
> > you have to read the time base used by the muxer, then calculate timestamps
> > using this time base, should be possible with av_rescale_q().
>
> Ok, so what do I set and what will be set automatically in general?
> As I said I found no documentation describing how to setup a stream
> manually, that is why I asked in the first place. Just used the remux
> example (which uses avcodec_parameters_copy) as the base and manually
> read information from an existing Cineform encoded by Blackmagic to set
> the parameters manually.

There is also a transcoding example in doc/examples and there is a lot
of documentation in libavformat/avformat.h.

> With
>  > out_avstream->avg_frame_rate = (AVRational){24, 1};
>  > out_avstream->r_frame_rate   = out_avstream->avg_frame_rate;
> I now have to correct frame rate shown when av_dump_format is called but
> after writing some frames the frame rate seems to be reset.
>
> My current code that tries to set correct pts/dts values:
>  > AVRational timebase_clip  = out_avstream->time_base;

This looks reasonable.

>  > AVRational timebase_count = (AVRational){1, 24}; // also tried {24, 1}

Don't know.

>  > pkg.pts  = av_rescale_q(frame, timebase_count, timebase_clip);

Probably av_packet_rescale_ts()

>  > pkg.dts  = pkg.dts;

This looks wrong.

Carl Eugen
_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".