[Libav-user] I there a way to prevent format guessing on the mpegts demuxer?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[Libav-user] I there a way to prevent format guessing on the mpegts demuxer?

Jesper Taxbøl
Hi

I am working on a streaming video application with a client and a server.

My transport stream contains HEVC, AAC and a customdata track. 

In the server I am using these settings on the data stream.

codec_type = AVMEDIA_TYPE_DATA;
codec_id = AV_CODEC_ID_BIN_DATA; 

The data is a small byte chunk sent for every frame with some parameters relating to the video. Fixed size 38bytes. Video, audio and data pass through the demuxer unharmed.

But, I am experiencing that my AVFormatContext returns the first ~20 packages within rrecieving the first 18000 bytes. Then it stalls for almost 2.8MB before returning the following packages.

It must be noted that I use an AVIOContext to feed the data into the AVFormatContext.

When debugging I have tried some changes and found that;

If I dont inject the actual data packets there is much less delay. So I suspect my problem has something to do with me triggering some analysis of the stream with my data packets. 

Is there a way to tell the AVFormatContext what the contents of the stream is, so I can get data from the stream with as little delay as posible?

I have added my test setup below, where the socket is replaced with a chunkwise data feeder.

kind regards

Jesper


#include <stdio.h>
#include <queue>


using namespace std;

#define __STDC_CONSTANT_MACROS

extern "C" {
#include <libavutil/avassert.h>
#include <libavutil/channel_layout.h>
#include <libavutil/opt.h>
#include <libavutil/mathematics.h>
#include <libavutil/timestamp.h>
#include <libavformat/avformat.h>
#include <libswscale/swscale.h>
#include <libswresample/swresample.h>
#include <libavutil/file.h>
}

struct Packet{
uint8_t* data;
uint64_t len;
};

queue<Packet*> Q;
static int x = 0;
uint64_t bc = 0;
static int read_packet(void *opaque, uint8_t *buf, int buf_size)
{
int r = 0;

//printf("x:%d %llu\r\n", x++, bc);
if(x == 35)
{
int x;
x = 2;
}
if(Q.size() > 0)
{
Packet* p = Q.front();
if(p->len > buf_size)
{
//printf("Cut\r\n");
memcpy(buf, p->data, buf_size);
p->len -= buf_size;
uint8_t* new_data = (uint8_t*)malloc(p->len);
memcpy(new_data, &p->data[buf_size], p->len);
free(p->data);
p->data = new_data;
r = buf_size;
}
else
{
//printf("Full\r\n");
memcpy(buf, p->data, p->len);
free(p->data);
r = p->len;
Q.pop();
free(p);
}

}
bc += r;
return r;
}



int main(int argc, char* argv[])
{
printf("Hello world...\r\n");

static AVFormatContext *fmt_ctx = NULL;
//fmt_ctx->flags |= AVFMT_FLA

//Read file data into queue packets
FILE* f = fopen("customdata.ts", "rb");
#define BUF_SIZE 1000
uint8_t b[BUF_SIZE];
while(1)
{
int l = fread(b, 1, BUF_SIZE, f);
if(l <= 0)
{
break;
}
Packet* p = (Packet*)malloc(sizeof(Packet));
p->data = (uint8_t*)malloc(l);
memcpy((void*)p->data, (void*)b, l);
p->len = l;
Q.push(p);
}
fclose(f);
printf("File read into %d chunks\r\n", Q.size());


if (!(fmt_ctx = avformat_alloc_context())) {
fprintf(stderr, "Could not allocate fmt_ctx\n");
exit(1);
}

size_t avio_ctx_buffer_size = 4096;

uint8_t* avio_ctx_buffer = (uint8_t*)av_malloc(avio_ctx_buffer_size);
if (!avio_ctx_buffer) {
printf("error allocating avio_ctx_buffer\r\n");
exit(0);
}

AVIOContext* avio_ctx = avio_alloc_context(avio_ctx_buffer, avio_ctx_buffer_size, 0, NULL, &read_packet, NULL, NULL);

if (!avio_ctx) {
printf("error allocating avio_ctx\r\n");
exit(0);
}
fmt_ctx->pb = avio_ctx;




AVInputFormat *inputFormat = av_find_input_format("mpegts");
AVDictionary *inOptions = NULL;
av_dict_set(&inOptions, "test","test", 0);
if (avformat_open_input(&fmt_ctx, NULL, inputFormat, &inOptions) < 0)
{
fprintf(stderr, "Could not open source file \r\n");
exit(1);
}

AVPacket pkt;
av_init_packet(&pkt);
pkt.data = NULL;
pkt.size = 0;
int framecounter = 0;
while(av_read_frame(fmt_ctx, &pkt) >= 0)
{
printf("index %d %llu %d\r\n", framecounter, bc, pkt.stream_index);
av_free_packet(&pkt);
if(framecounter == 22)
{
printf("now somethin funny start\r\n");
}
if(framecounter++ > 100)
{
break;
}
}

return 0;
}

_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: I there a way to prevent format guessing on the mpegts demuxer?

Jesper Taxbøl
As I have made no progress on this problem, I have made a small repo with a testcase and two sample transport streams. 


I hope that it will help me get some feedback to progress from.

The problem is that when my stream contains customdata, that data triggers something in my (mpegts) demuxer that keeps the demuxer from returning AVPackets for more than 2.8MB of the stream. That delays my playback with almost 5 seconds which is quite annoying.

When there is no customdata is in the stream the packages are received at much more regular intervals.

I am on "avformat_version()" version 3808612

The output I get here is (streamindex 2 is the customdata):

make test

Testing with Plain
./main plain.ts
plain.ts read into 1382 chunks
We are using libavformat version: 3808612
Package returned after 34816 bytes read. It has StreamIndex: 0 and is 24681 bytes long.
Package returned after 38912 bytes read. It has StreamIndex: 0 and is 6510 bytes long.
Package returned after 45056 bytes read. It has StreamIndex: 0 and is 3350 bytes long.
Package returned after 49152 bytes read. It has StreamIndex: 0 and is 4758 bytes long.
Package returned after 59392 bytes read. It has StreamIndex: 0 and is 3394 bytes long.
Package returned after 69632 bytes read. It has StreamIndex: 1 and is 306 bytes long.
First video package returned after having read 69632 bytes.

Testing with Customdata
customdata.ts read into 1418 chunks
We are using libavformat version: 3808612
Package returned after 2824192 bytes read. It has StreamIndex: 2 and is 36 bytes long.
Package returned after 2824192 bytes read. It has StreamIndex: 2 and is 36 bytes long.
Package returned after 2824192 bytes read. It has StreamIndex: 2 and is 36 bytes long.
Package returned after 2824192 bytes read. It has StreamIndex: 1 and is 24681 bytes long.
First video package returned after having read 2824192 bytes.

The test case shows that the first video AVPacket is returned after 69632 bytes when there is no customdata in the stream. With customdata in the stream the first video AVPacket is returned after 2.8MB.

I have tried setting variables like max_analyze_duration on my AVFormatContext, but with no luck yet.

The ffprobe output of customdata.ts is 

ffprobe -hide_banner customdata.ts
Input #0, mpegts, from 'customdata.ts':
 Duration: 00:00:52.28, start: 0.062000, bitrate: 2220 kb/s
 Program 1  
   Metadata:
     service_name    : Service01
     service_provider: FFmpeg
   Stream #0:0[0x46]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 93 kb/s
   Stream #0:1[0x45]: Video: hevc (Main) (HEVC / 0x43564548), yuv420p(tv), 1280x720 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 90k tbn, 30 tbc
   Stream #0:2[0x47]: Data: bin_data ([6][0][0][0] / 0x0006)
Unsupported codec with id 100359 for input stream 2

Any hints that can help me deal with this issue will be highly appreciated.

Kind regards

Jesper

Den søn. 24. maj 2020 kl. 00.29 skrev Jesper Taxbøl <[hidden email]>:
Hi

I am working on a streaming video application with a client and a server.

My transport stream contains HEVC, AAC and a customdata track. 

In the server I am using these settings on the data stream.

codec_type = AVMEDIA_TYPE_DATA;
codec_id = AV_CODEC_ID_BIN_DATA; 

The data is a small byte chunk sent for every frame with some parameters relating to the video. Fixed size 38bytes. Video, audio and data pass through the demuxer unharmed.

But, I am experiencing that my AVFormatContext returns the first ~20 packages within rrecieving the first 18000 bytes. Then it stalls for almost 2.8MB before returning the following packages.

It must be noted that I use an AVIOContext to feed the data into the AVFormatContext.

When debugging I have tried some changes and found that;

If I dont inject the actual data packets there is much less delay. So I suspect my problem has something to do with me triggering some analysis of the stream with my data packets. 

Is there a way to tell the AVFormatContext what the contents of the stream is, so I can get data from the stream with as little delay as posible?

I have added my test setup below, where the socket is replaced with a chunkwise data feeder.

kind regards

Jesper


#include <stdio.h>
#include <queue>


using namespace std;

#define __STDC_CONSTANT_MACROS

extern "C" {
#include <libavutil/avassert.h>
#include <libavutil/channel_layout.h>
#include <libavutil/opt.h>
#include <libavutil/mathematics.h>
#include <libavutil/timestamp.h>
#include <libavformat/avformat.h>
#include <libswscale/swscale.h>
#include <libswresample/swresample.h>
#include <libavutil/file.h>
}

struct Packet{
uint8_t* data;
uint64_t len;
};

queue<Packet*> Q;
static int x = 0;
uint64_t bc = 0;
static int read_packet(void *opaque, uint8_t *buf, int buf_size)
{
int r = 0;

//printf("x:%d %llu\r\n", x++, bc);
if(x == 35)
{
int x;
x = 2;
}
if(Q.size() > 0)
{
Packet* p = Q.front();
if(p->len > buf_size)
{
//printf("Cut\r\n");
memcpy(buf, p->data, buf_size);
p->len -= buf_size;
uint8_t* new_data = (uint8_t*)malloc(p->len);
memcpy(new_data, &p->data[buf_size], p->len);
free(p->data);
p->data = new_data;
r = buf_size;
}
else
{
//printf("Full\r\n");
memcpy(buf, p->data, p->len);
free(p->data);
r = p->len;
Q.pop();
free(p);
}

}
bc += r;
return r;
}



int main(int argc, char* argv[])
{
printf("Hello world...\r\n");

static AVFormatContext *fmt_ctx = NULL;
//fmt_ctx->flags |= AVFMT_FLA

//Read file data into queue packets
FILE* f = fopen("customdata.ts", "rb");
#define BUF_SIZE 1000
uint8_t b[BUF_SIZE];
while(1)
{
int l = fread(b, 1, BUF_SIZE, f);
if(l <= 0)
{
break;
}
Packet* p = (Packet*)malloc(sizeof(Packet));
p->data = (uint8_t*)malloc(l);
memcpy((void*)p->data, (void*)b, l);
p->len = l;
Q.push(p);
}
fclose(f);
printf("File read into %d chunks\r\n", Q.size());


if (!(fmt_ctx = avformat_alloc_context())) {
fprintf(stderr, "Could not allocate fmt_ctx\n");
exit(1);
}

size_t avio_ctx_buffer_size = 4096;

uint8_t* avio_ctx_buffer = (uint8_t*)av_malloc(avio_ctx_buffer_size);
if (!avio_ctx_buffer) {
printf("error allocating avio_ctx_buffer\r\n");
exit(0);
}

AVIOContext* avio_ctx = avio_alloc_context(avio_ctx_buffer, avio_ctx_buffer_size, 0, NULL, &read_packet, NULL, NULL);

if (!avio_ctx) {
printf("error allocating avio_ctx\r\n");
exit(0);
}
fmt_ctx->pb = avio_ctx;




AVInputFormat *inputFormat = av_find_input_format("mpegts");
AVDictionary *inOptions = NULL;
av_dict_set(&inOptions, "test","test", 0);
if (avformat_open_input(&fmt_ctx, NULL, inputFormat, &inOptions) < 0)
{
fprintf(stderr, "Could not open source file \r\n");
exit(1);
}

AVPacket pkt;
av_init_packet(&pkt);
pkt.data = NULL;
pkt.size = 0;
int framecounter = 0;
while(av_read_frame(fmt_ctx, &pkt) >= 0)
{
printf("index %d %llu %d\r\n", framecounter, bc, pkt.stream_index);
av_free_packet(&pkt);
if(framecounter == 22)
{
printf("now somethin funny start\r\n");
}
if(framecounter++ > 100)
{
break;
}
}

return 0;
}


--
Jesper Taxbøl
+45 61627501


_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: I there a way to prevent format guessing on the mpegts demuxer?

KaptajnVom
Perhaps you can ask libavformat to discard your custom data?

fmt_ctx->streams[2]->discard = AVDISCARD_ALL;



--
Sent from: http://libav-users.943685.n4.nabble.com/
_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".
Reply | Threaded
Open this post in threaded view
|

Re: I there a way to prevent format guessing on the mpegts demuxer?

Jesper Taxbøl
The streams does not exist in my AVFormatContext before they are detected and I do need the data. :)

But the idea does carry an idea. Perhaps I can populate the stream before start, to avoid the problem?

I am too noob in the ffmpeg source to figure out if/how its possible and I have not found any examples that does it.

Alternatively I could perhaps hide the data as something else that will pass detection more smoothly.

Jesper


tir. 9. jun. 2020 23.37 skrev KaptajnVom <[hidden email]>:
Perhaps you can ask libavformat to discard your custom data?

fmt_ctx->streams[2]->discard = AVDISCARD_ALL;



--
Sent from: http://libav-users.943685.n4.nabble.com/
_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".

_______________________________________________
Libav-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/libav-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".