[Libav-user] Writing Audio AVPacket without resampling

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

[Libav-user] Writing Audio AVPacket without resampling

sebastian.wichmann

Hello,
I’m trying to record the microphone on windows via dshow and ffmpeg in real time.

The problem is, my code is to slow.

 

At the moment I get a

  • AVPacket
  • Resample it
  • And write it to a file

 

The codec is unimportant for me at this stage of the program.

 

So is it possible to write the received AVPacket to a mp4 file?

 

Sebastian Wichmann


_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Writing Audio AVPacket without resampling

Carl Eugen Hoyos-2
Am Do., 19. März 2020 um 18:57 Uhr schrieb
<[hidden email]>:

>
> Hello,
> I’m trying to record the microphone on windows via dshow and ffmpeg in real time.
>
> The problem is, my code is to slow.
>
>
>
> At the moment I get a
>
> AVPacket
> Resample it
> And write it to a file
>
>
>
> The codec is unimportant for me at this stage of the program.
>
>
>
> So is it possible to write the received AVPacket to a mp4 file?

Did you try to write pcm_s16be?

Carl Eugen
_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Writing Audio AVPacket without resampling

sebastian.wichmann
Hello,

I'm very new to this subject, so I'm not sure what you mean.
At the moment I can't even tell you the codec of the read AVPacket.
I can tell you that I'm trying to resample it to AV_SAMPLE_FMT_FLTP.

The problem that I'm currently facing is that I cant get the codec from an AVPacket. Is there anyway to gain access to this?

Best regards
Sebastian Wichman

-----Ursprüngliche Nachricht-----
Von: Libav-user <[hidden email]> Im Auftrag von Carl Eugen Hoyos
Gesendet: Donnerstag, 19. März 2020 19:01
An: This list is about using libavcodec, libavformat, libavutil, libavdevice and libavfilter. <[hidden email]>
Betreff: Re: [Libav-user] Writing Audio AVPacket without resampling

Am Do., 19. März 2020 um 18:57 Uhr schrieb
<[hidden email]>:

>
> Hello,
> I’m trying to record the microphone on windows via dshow and ffmpeg in real time.
>
> The problem is, my code is to slow.
>
>
>
> At the moment I get a
>
> AVPacket
> Resample it
> And write it to a file
>
>
>
> The codec is unimportant for me at this stage of the program.
>
>
>
> So is it possible to write the received AVPacket to a mp4 file?

Did you try to write pcm_s16be?

Carl Eugen
_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email [hidden email] with subject "unsubscribe".

_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Writing Audio AVPacket without resampling

Carl Eugen Hoyos-2
In reply to this post by sebastian.wichmann
Am Do., 19. März 2020 um 18:57 Uhr schrieb
<[hidden email]>:

> I’m trying to record the microphone on windows via dshow and
> ffmpeg in real time.
>
> The problem is, my code is to slow.
>
> At the moment I get a
>
> AVPacket
> Resample it
> And write it to a file

Since your (original) issue is "too slow":
Did you already test with ffmpeg (the cli)?
If it fast enough to do what you want, you
should be able to do the same programmatically.

If not, you already know that it will be difficult.

> The codec is unimportant for me at this stage of the program.

Start with pcm_s16be which should be only I/O limited.

> So is it possible to write the received AVPacket to a mp4 file?

You didn't tell us what's in the AVPacket.

Please find out what top-posting means and avoid it here
if you want support.

Carl Eugen
_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Writing Audio AVPacket without resampling

sebastian.wichmann
Hello,
I'm sorry for the bad worded question. Its hard to write a question if you don't fully understand the matter.

> Since your (original) issue is "too slow":
> Did you already test with ffmpeg (the cli)?
> If it fast enough to do what you want, you should be able to do the same programmatically.
Yes the standalone application is able to record the audio and video in real time.

> Start with pcm_s16be which should be only I/O limited.
My hope is that there isn't any need for resampling at all. I'll try it in the next days.

> You didn't tell us what's in the AVPacket.
At the moment I'm just concentrating on the recorded audio. So the packet should contain audio snippets recorded by dshow.
I don’t think there is any way to have any influence on this packet.
So in my opinion I need to decode the packet and encode it again?

Another big problem for me are the pts and dts. I already tried the example code of the muxing example.
But the resulting file can't be played.

> Please find out what top-posting means and avoid it here if you want support.
I hope this mail will fulfil this requirement now?
Are there any other guidelines I need to follow? Especiallyto send code snippets?

Best regards
Sebastian Wichmann

_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Writing Audio AVPacket without resampling

Carl Eugen Hoyos-2
Am So., 22. März 2020 um 13:23 Uhr schrieb
<[hidden email]>:

> I'm sorry for the bad worded question. Its hard to write a question if you don't fully understand the matter.

And even more difficult to answer...

> > Since your (original) issue is "too slow":
> > Did you already test with ffmpeg (the cli)?
> > If it fast enough to do what you want, you should be able to do the same programmatically.
> Yes the standalone application is able to record the audio and video in real time.

If you show us the command line you tested together with the complete, uncut
console output, we can answer if resampling will be necessary.

> > Start with pcm_s16be which should be only I/O limited.
> My hope is that there isn't any need for resampling at all. I'll try it in the next days.

I cannot answer this but you already answered that you don't have a performance
issue anyway...

Carl Eugen
_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Writing Audio AVPacket without resampling

sebastian.wichmann
Hello,

>> I'm sorry for the bad worded question. Its hard to write a question if you don't fully understand the matter.
>And even more difficult to answer...
I'm sorry about that, I'll try to word my questions better the next time.

> > > Since your (original) issue is "too slow":
> > > Did you already test with ffmpeg (the cli)?
> > > If it fast enough to do what you want, you should be able to do the same programmatically.
> > Yes the standalone application is able to record the audio and video in real time.
> If you show us the command line you tested together with the complete, uncut console output, we can answer if resampling will be necessary.

The command looks like this:
ffmpeg -f gdigrab -framerate 30 -i desktop -f dshow -i audio="Mikrofon (Realtek High Definition Audio)" test.mp4

The console output looks like this:
ffmpeg version 4.1.3 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 8.3.1 (GCC) 20190414
  configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
[gdigrab @ 0000022a0bc6e580] Capturing whole desktop as 1366x768x32 at (0,0)
[gdigrab @ 0000022a0bc6e580] Stream #0: not enough frames to estimate rate; consider increasing probesize
Input #0, gdigrab, from 'desktop':
  Duration: N/A, start: 1584949529.887344, bitrate: 1007137 kb/s
    Stream #0:0: Video: bmp, bgra, 1366x768, 1007137 kb/s, 30 fps, 1000k tbr, 1000k tbn, 1000k tbc
Guessed Channel Layout for Input Stream #1.0 : stereo
Input #1, dshow, from 'audio=Mikrofon (Realtek High Definition Audio)':
  Duration: N/A, start: 400094.071000, bitrate: 1411 kb/s
    Stream #1:0: Audio: pcm_s16le, 44100 Hz, stereo, s16, 1411 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (bmp (native) -> h264 (libx264))
  Stream #1:0 -> #0:1 (pcm_s16le (native) -> aac (native))
Press [q] to stop, [?] for help
[libx264 @ 0000022a0bcea840] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0000022a0bcea840] profile High 4:4:4 Predictive, level 3.2, 4:4:4, 8-bit
[libx264 @ 0000022a0bcea840] 264 - core 157 r2970 5493be8 - H.264/MPEG-4 AVC codec - Copyleft 2003-2019 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=4 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'test.mp4':
  Metadata:
    encoder         : Lavf58.20.100
    Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv444p(progressive), 1366x768, q=-1--1, 30 fps, 15360 tbn, 30 tbc
    Metadata:
      encoder         : Lavc58.35.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
    Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s
    Metadata:
      encoder         : Lavc58.35.100 aac
[gdigrab @ 0000022a0bc6e580] Thread message queue blocking; consider raising the thread_queue_size option (current value: 8)
frame=  354 fps= 32 q=-1.0 Lsize=    1050kB time=00:00:11.70 bitrate= 735.5kbits/s dup=41 drop=0 speed=1.06x
video:872kB audio:165kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.262673%
[libx264 @ 0000022a0bcea840] frame I:2     Avg QP:19.05  size:232061
[libx264 @ 0000022a0bcea840] frame P:89    Avg QP:19.14  size:  4440
[libx264 @ 0000022a0bcea840] frame B:263   Avg QP:27.81  size:   126
[libx264 @ 0000022a0bcea840] consecutive B-frames:  0.8%  0.0%  0.8% 98.3%
[libx264 @ 0000022a0bcea840] mb I  I16..4: 38.5% 19.4% 42.1%
[libx264 @ 0000022a0bcea840] mb P  I16..4:  0.3%  0.5%  0.6%  P16..4:  1.6%  0.2%  0.1%  0.0%  0.0%    skip:96.7%
[libx264 @ 0000022a0bcea840] mb B  I16..4:  0.0%  0.1%  0.0%  B16..8:  2.3%  0.0%  0.0%  direct: 0.0%  skip:97.6%  L0:61.7% L1:38.2% BI: 0.1%
[libx264 @ 0000022a0bcea840] 8x8 transform intra:28.6% inter:43.9%
[libx264 @ 0000022a0bcea840] coded y,u,v intra: 31.3% 28.0% 27.6% inter: 0.1% 0.0% 0.0%
[libx264 @ 0000022a0bcea840] i16 v,h,dc,p: 31% 66%  3%  0%
[libx264 @ 0000022a0bcea840] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 34% 24% 36%  1%  0%  0%  0%  0%  4%
[libx264 @ 0000022a0bcea840] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 33% 26% 17%  2%  3%  4%  6%  3%  6%
[libx264 @ 0000022a0bcea840] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0000022a0bcea840] ref P L0: 33.5%  3.6% 44.4% 18.5%
[libx264 @ 0000022a0bcea840] ref B L0: 41.4% 55.7%  2.9%
[libx264 @ 0000022a0bcea840] ref B L1: 95.1%  4.9%
[libx264 @ 0000022a0bcea840] kb/s:605.09
[aac @ 0000022a0bcec780] Qavg: 243.525

Best regards,
Sebastian Wichmann

_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Writing Audio AVPacket without resampling

Carl Eugen Hoyos-2
Am Mo., 23. März 2020 um 08:51 Uhr schrieb
<[hidden email]>:

> The command looks like this:
> ffmpeg -f gdigrab -framerate 30 -i desktop -f dshow -i audio="Mikrofon (Realtek High Definition Audio)" test.mp4

This command is (very often!) reading a frame from memory, converting
its colour space,
encoding the frame on the cpu and writing the encoded frame to disk.
The tiny (tiny, tiny) bit of work for resampling s16 to fltp and
encoding it into aac
is ridiculous in comparison. If you want to spare the time for
resampling (and audio
encoding) you can use pcm_s16be as encoder, this makes the output file bigger
and puts a little more pressure on I/O.
It might not be easy to measure the difference though;-)

Unrelated: You are using yuv444p as colour space for your h264 encoding,
this colour space cannot be played back unless your player is FFmpeg-based.

Carl Eugen
_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Writing Audio AVPacket without resampling

Giuseppe Torelli
On Tue, 24 Mar 2020 at 22:21, Carl Eugen Hoyos <[hidden email]> wrote:

This command is (very often!) reading a frame from memory, converting
its colour space,
encoding the frame

Hi Carl, out of curiosity, are you the onle one replying in this list...?

Giuseppe 

_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Writing Audio AVPacket without resampling

Carl Eugen Hoyos-2
Am Di., 24. März 2020 um 23:27 Uhr schrieb Giuseppe Torelli
<[hidden email]>:
>
> On Tue, 24 Mar 2020 at 22:21, Carl Eugen Hoyos <[hidden email]> wrote:
>>
>>
>> This command is (very often!) reading a frame from memory, converting
>> its colour space, encoding the frame
>
> Hi Carl, out of curiosity, are you the onle one replying in this list...?

I think I don't understand your question...

Carl Eugen
_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Writing Audio AVPacket without resampling

Giuseppe Torelli



I think I don't understand your question...

You are the only one active in this mailing list. The replies are only coming from you.

Giueppe

_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: Writing Audio AVPacket without resampling

sebastian.wichmann
In reply to this post by Carl Eugen Hoyos-2
Hello,
I'm sorry for the late response.

> > The command looks like this:
> > ffmpeg -f gdigrab -framerate 30 -i desktop -f dshow -i audio="Mikrofon
> > (Realtek High Definition Audio)" test.mp4

> This command is (very often!) reading a frame from memory, converting its colour space,
> encoding the frame on the cpu and writing the encoded frame to disk.
At the moment this part is working more or less like a charm. So I'm not worrying at the moment.

> The tiny (tiny, tiny) bit of work for resampling s16 to fltp and encoding it into aac is ridiculous in comparison.
> If you want to spare the time for resampling (and audioencoding) you can use pcm_s16be as encoder,
> this makes the output file bigger and puts a little more pressure on I/O.
> It might not be easy to measure the difference though;-)
Another big problem is that this part isn't working at all. I mean that I'm not able to write any audio in an empty mp4 file. But this isn't part of this question.
So I think I will open another one for this problem.

> Unrelated: You are using yuv444p as colour space for your h264 encoding, this colour space cannot be played back
> unless your player is FFmpeg-based.
Thanks for this information!

Sebastian Wichmann

_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".