[Libav-user] Questions about using libav in a multi-threaded application

classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

[Libav-user] Questions about using libav in a multi-threaded application

Tr3wory
Hi all!

I started writing a multi-threaded program which can read multiple
videos simultaneously.
Basically every thread read it's own video, and use it's own
AVFormatContext and AVCodecContext so no libav related shared data
exist.
What is not clear to me: is this scenario supported by default, or I
need to use av_lockmgr_register etc.?

My other questions is logging related: I know I can register a new
callback with av_log_set_callback, but because I have more than one
video in processing, how can I know where the logging information came
from?

Thanks for your help!

Best,
Tr3wory
_______________________________________________
Libav-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/libav-user
Reply | Threaded
Open this post in threaded view
|

Re: Questions about using libav in a multi-threaded application

Andrey Utkin
2012/1/15 Tr3wory <[hidden email]>:
> Hi all!
>
> I started writing a multi-threaded program which can read multiple
> videos simultaneously.
> Basically every thread read it's own video, and use it's own
> AVFormatContext and AVCodecContext so no libav related shared data
> exist.
> What is not clear to me: is this scenario supported by default, or I
> need to use av_lockmgr_register etc.?

You should use av_lockmgr_register(), otherwise you'll have warnings
on avcodec_open.
Although i quite don't get the reason why avcodec_open should require
this from library user.

> My other questions is logging related: I know I can register a new
> callback with av_log_set_callback, but because I have more than one
> video in processing, how can I know where the logging information came
> from?

The first callback arg, void *ptr, is the 'context' of what log
message is related to.
Although you can get many different values in this field:
AVFormatContext*, AVCodecContext, even URLContext* or so. So you get
the idea, it'll not be easy to check what is about what in this case.
But still possible, as you can traverse all AVCodecContexts of each
AVFormatContext, and so on.

--
Andrey Utkin
_______________________________________________
Libav-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/libav-user
Reply | Threaded
Open this post in threaded view
|

Re: [Libav-user] Questions about using libav in a multi-threaded application

wl2776
Administrator
In reply to this post by Tr3wory
Tr3wory wrote
My other questions is logging related: I know I can register a new
callback with av_log_set_callback, but because I have more than one
video in processing, how can I know where the logging information came
from?
I suggest to specify some kind of a thread identifier in a log message.
Reply | Threaded
Open this post in threaded view
|

Re: Questions about using libav in a multi-threaded application

Tr3wory
In the callback will the current thread be the same as the thread
where I called the libav functions?

tr3w

On Mon, Jan 16, 2012 at 10:43 AM, wl2776 <[hidden email]> wrote:

>
> Tr3wory wrote
>>
>> My other questions is logging related: I know I can register a new
>> callback with av_log_set_callback, but because I have more than one
>> video in processing, how can I know where the logging information came
>> from?
>>
>
> I suggest to specify some kind of a thread identifier in a log message.
>
>
> --
> View this message in context: http://libav-users.943685.n4.nabble.com/Libav-user-Questions-about-using-libav-in-a-multi-threaded-application-tp4297394p4298956.html
> Sent from the libav-users mailing list archive at Nabble.com.
> _______________________________________________
> Libav-user mailing list
> [hidden email]
> http://ffmpeg.org/mailman/listinfo/libav-user
_______________________________________________
Libav-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/libav-user
Reply | Threaded
Open this post in threaded view
|

Re: Questions about using libav in a multi-threaded application

Andrey Utkin
2012/1/16 Tr3wory <[hidden email]>:
> In the callback will the current thread be the same as the thread
> where I called the libav functions?

Yes, except if multithreaded decoder/encoder will log something from
its thread pool. Their multithreading is disabled by default,
currently.

--
Andrey Utkin
_______________________________________________
Libav-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/libav-user
Reply | Threaded
Open this post in threaded view
|

Re: Questions about using libav in a multi-threaded application

Tr3wory
Can't I just set some pointer in a context or somewhere else what I
can use in the callback like in most other callback driven library?

tr3w

On Mon, Jan 16, 2012 at 10:56 AM, Andrey Utkin
<[hidden email]> wrote:

> 2012/1/16 Tr3wory <[hidden email]>:
>> In the callback will the current thread be the same as the thread
>> where I called the libav functions?
>
> Yes, except if multithreaded decoder/encoder will log something from
> its thread pool. Their multithreading is disabled by default,
> currently.
>
> --
> Andrey Utkin
> _______________________________________________
> Libav-user mailing list
> [hidden email]
> http://ffmpeg.org/mailman/listinfo/libav-user
_______________________________________________
Libav-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/libav-user
Reply | Threaded
Open this post in threaded view
|

Re: Questions about using libav in a multi-threaded application

wl2776
Administrator
Yes, y
Tr3wory wrote
Can't I just set some pointer in a context or somewhere else what I
can use in the callback like in most other callback driven library?
Yes, you can.
Generally, you call av_log(/*log level*/, /* context pointer */, "message", /*other arguments*/ );
Default log callback casts context pointer from void * to AVClass * and extracts all necessary information from it.
You can create your own variable of type AVClass and pass its pointer to the log.
Reply | Threaded
Open this post in threaded view
|

Re: Questions about using libav in a multi-threaded application

Tr3wory
But I don't call the av_log myself. I just call some libav function to
decode a video, and during the processing it calls the av_log.
I can register my own log callback, but because I run more than one
decoding, I don't know where the log information came from.

In other libraries, I can set a pointer called userData or something
when I register a callback, and I can use this pointer in the callback
because I get it as a parameter.

Is this possible somehow in libav?

tr3w

On Mon, Jan 16, 2012 at 12:13 PM, wl2776 <[hidden email]> wrote:

> Yes, y
> Tr3wory wrote
>>
>> Can't I just set some pointer in a context or somewhere else what I
>> can use in the callback like in most other callback driven library?
>>
>
> Yes, you can.
> Generally, you call av_log(/*log level*/, /* context pointer */, "message",
> /*other arguments*/ );
> Default log callback casts context pointer from void * to AVClass * and
> extracts all necessary information from it.
> You can create your own variable of type AVClass and pass its pointer to the
> log.
>
>
> --
> View this message in context: http://libav-users.943685.n4.nabble.com/Libav-user-Questions-about-using-libav-in-a-multi-threaded-application-tp4297394p4299173.html
> Sent from the libav-users mailing list archive at Nabble.com.
> _______________________________________________
> Libav-user mailing list
> [hidden email]
> http://ffmpeg.org/mailman/listinfo/libav-user
_______________________________________________
Libav-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/libav-user
Reply | Threaded
Open this post in threaded view
|

Re: Questions about using libav in a multi-threaded application

Hendrik Leppkes
Hi,

On 1/16/12, Tr3wory <[hidden email]> wrote:
> Is this possible somehow in libav?

Not for the logging callback, no.

- Hendrik
_______________________________________________
Libav-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/libav-user
Reply | Threaded
Open this post in threaded view
|

Re: Questions about using libav in a multi-threaded application

Andrey Utkin
In reply to this post by wl2776
2012/1/16 wl2776 <[hidden email]>:

> Yes, y
> Tr3wory wrote
>>
>> Can't I just set some pointer in a context or somewhere else what I
>> can use in the callback like in most other callback driven library?
>>
>
> Yes, you can.
> Generally, you call av_log(/*log level*/, /* context pointer */, "message",
> /*other arguments*/ );
> Default log callback casts context pointer from void * to AVClass * and
> extracts all necessary information from it.
> You can create your own variable of type AVClass and pass its pointer to the
> log.

I suggest topicstarter needs to identify log calls that come from
inside of libs, not his own loggings.
But logging facility is not meant to provide such identification with
corresponding task, unfortunately.

--
Andrey Utkin
_______________________________________________
Libav-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/libav-user
Reply | Threaded
Open this post in threaded view
|

Re: Questions about using libav in a multi-threaded application

Andrey Utkin
In reply to this post by Tr3wory
2012/1/16 Tr3wory <[hidden email]>:

> But I don't call the av_log myself. I just call some libav function to
> decode a video, and during the processing it calls the av_log.
> I can register my own log callback, but because I run more than one
> decoding, I don't know where the log information came from.
>
> In other libraries, I can set a pointer called userData or something
> when I register a callback, and I can use this pointer in the callback
> because I get it as a parameter.
>
> Is this possible somehow in libav?

No.

--
Andrey Utkin
_______________________________________________
Libav-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/libav-user
Reply | Threaded
Open this post in threaded view
|

Re: Questions about using libav in a multi-threaded application

Tr3wory
In reply to this post by Andrey Utkin
So basically the only possible way of doing this, is to use the
parent_log_context_offset from AVClass, and go upwards from child to
parent until I found the topmost context which is an AVFormatContext
or an AVCodecContext what I used when I called the libav functions. Is
that right? Will it work?

tr3w

On Sun, Jan 15, 2012 at 8:26 PM, Andrey Utkin
<[hidden email]> wrote:

> 2012/1/15 Tr3wory <[hidden email]>:
>> Hi all!
>>
>> I started writing a multi-threaded program which can read multiple
>> videos simultaneously.
>> Basically every thread read it's own video, and use it's own
>> AVFormatContext and AVCodecContext so no libav related shared data
>> exist.
>> What is not clear to me: is this scenario supported by default, or I
>> need to use av_lockmgr_register etc.?
>
> You should use av_lockmgr_register(), otherwise you'll have warnings
> on avcodec_open.
> Although i quite don't get the reason why avcodec_open should require
> this from library user.
>
>> My other questions is logging related: I know I can register a new
>> callback with av_log_set_callback, but because I have more than one
>> video in processing, how can I know where the logging information came
>> from?
>
> The first callback arg, void *ptr, is the 'context' of what log
> message is related to.
> Although you can get many different values in this field:
> AVFormatContext*, AVCodecContext, even URLContext* or so. So you get
> the idea, it'll not be easy to check what is about what in this case.
> But still possible, as you can traverse all AVCodecContexts of each
> AVFormatContext, and so on.
>
> --
> Andrey Utkin
> _______________________________________________
> Libav-user mailing list
> [hidden email]
> http://ffmpeg.org/mailman/listinfo/libav-user
_______________________________________________
Libav-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/libav-user
Reply | Threaded
Open this post in threaded view
|

Re: Questions about using libav in a multi-threaded application

Andrey Utkin
2012/1/16 Tr3wory <[hidden email]>:
> So basically the only possible way of doing this, is to use the
> parent_log_context_offset from AVClass, and go upwards from child to
> parent until I found the topmost context which is an AVFormatContext
> or an AVCodecContext what I used when I called the libav functions. Is
> that right? Will it work?

Too difficult to me.
I'd rather
1. postpone this issue
or
2. use separate processes for each task. Thus, seemingly, log messages
mustn't mix together.

--
Andrey Utkin
_______________________________________________
Libav-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/libav-user
Reply | Threaded
Open this post in threaded view
|

Re: Questions about using libav in a multi-threaded application

wl2776
Administrator
In reply to this post by Tr3wory
Tr3wory wrote
Is this possible somehow in libav?
The only way I see is modifying the part of AVCodecContext and AVFormatContext, related to AVClass *.
I have the following code in one of my projects:

static const char* task_to_name(void* ptr) {
    AVCodecContext *avc = (AVCodecContext *) ptr;

    if (avc && avc->codec_name)
        return avc->codec_name;
    else
        return "NULL";
}

static const AVClass av_codec_context_class_for_task = {"AVCodecContextTask", task_to_name, NULL};


... 

AVCodecContext *codec;
...

codec->av_class = &av_codec_context_class_for_task;

Reply | Threaded
Open this post in threaded view
|

Re: Questions about using libav in a multi-threaded application

wl2776
Administrator
wl2776 wrote
Tr3wory wrote
Is this possible somehow in libav?
The only way I see is modifying the part of AVCodecContext and AVFormatContext, related to AVClass *.
In other words, you can adopt some code from libav:
http://cekirdek.pardus.org.tr/~ismail/ffmpeg-docs/libavcodec_2options_8c-source.html#l00449

Define your own avclass structure and set its pointer to AVFormatContext or AVCodecContext right after their creation.
Reply | Threaded
Open this post in threaded view
|

Re: Questions about using libav in a multi-threaded application

wl2776
Administrator
wl2776 wrote
Define your own avclass structure and set its pointer to AVFormatContext or AVCodecContext right after their creation.
Sorry, not structure, variable of type AVClass.
Reply | Threaded
Open this post in threaded view
|

Re: Questions about using libav in a multi-threaded application

Carl Eugen Hoyos
In reply to this post by Andrey Utkin
Andrey Utkin <andrey.krieger.utkin@...> writes:

> Yes, except if multithreaded decoder/encoder will log something from
> its thread pool.

> Their multithreading is disabled by default,

I don't think this is correct anymore.

Carl Eugen

_______________________________________________
Libav-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/libav-user