[ltt-dev] Create a new lttd library

[Michael Sills Lavoie]

2010/2/11 Mathieu Desnoyers <compudj at krystal.dyndns.org>

> * Michael Sills Lavoie (michael.sills-lavoie at polymtl.ca) wrote:
> > Hey Mathieu,
> >
> > We would like to create a library with lttd so we could share lttd
> > functionality with other program like the lttng tcf agent to send data
> over
> > the network. This way, every program would be maintain at the same time.
> If
> > you remember, we talked about how we could do that this summer at the end
> of
> > my internship.
>
> Hi Michael,
>
> Yep.
>
> >
> > This is what we would do based on our conversations :
> >
> > 1) Create a new library (liblttd) that would contain the basic
> functionality
> > of lttd.
> > 2) Add a set of callbacks used to modified the behaviour of the program
> > while it gathers the data.
>
> Not sure I see what you mean by "modify the behavior of the program".
> Please describe. I especially wonder what you plan to modify while
> gathering data.
>

I did not express myself correctly. I only meant that we can do what we want
within the callbacks. For example, one program can send the data to the
network but another could write it to the disk.

>
> > 3) Create a new implementation of lttd that use the lib.
> > 4) Modify the lttng-agent tcf implementation to use the new lib.
> >
> > A callback stuct would be passed to the lib to start recolting the data :
> We
> > tough about this struct :
> >
> > struct liblttd_callbacks {
> > int(*on_open_channel)(struct liblttd_callbacks *data, struct fd_pair
> *pair,
> > char *relative_channel_path);
> > int(*on_close_channel)(struct liblttd_callbacks *data, struct fd_pair
> > *pair);
> > int(*on_new_channels_folder)(struct liblttd_callbacks *data, char
> > *relative_folder_path);
> > /*on_read_subbuffer has to be reentrant, it'll be called by many
> threads*/
>
> reentrant may mean signal-safe or thread-safe (or both). Saying it's
> threas-safe is more precise.
>

Ok we'll do that!

>
> > int(*on_read_subbuffer)(struct liblttd_callbacks *data, struct fd_pair
> > *pair, unsigned int len);
> > int(*on_trace_end)(struct liblttd_callbacks *data);
> > /*on_new_thread has to be reentrant, it'll be called by many threads*/
>
> Same here.
>
> > int(*on_new_thread)(struct liblttd_callbacks *data, unsigned long
> > thread_num);
> > /*on_close_thread has to be reentrant, it'll be called by many threads*/
>
> And here.
>
> > int(*on_close_thread)(struct liblttd_callbacks *data, unsigned long
> > thread_num);
> > void *user_data;
> > };
>
> You should describe more clearly, in the final implementation, exactly
> when the callbacks are called (before or after the action?).
>

This will be done.

>
> >
> > int liblttd_start(char *channel_path, unsigned long n_threads,
> > int flight_only, int normal_only, int verbose,
> > struct liblttd_callbacks *user_data);
> > int liblttd_stop();
>
> Weird. Stop does not take any parameters ? So it's a all-or-nothing ?
>

In this case, liblttd_stop would be the equivalent of setting quit_program
to 1 in the current lttd implementation. So it'll wait for every thread to
finish their work and stop the recording gracefully. Do you have something
else in mind?

>
> >
> > - on_[open,close,new_channel_folder] would be used by lttd to create the
> > trace structure (folders and files)
> > - on_read_subbuffer would be called by the lib when new data is available
> in
> > a subbuffer
> > - on_[new,close]_thread would be used to initialize some data related to
> the
> > different recolting thread (ex. thread_pipe)
> > etc.
> >
> > In order to use the lib, a program would have to fill a callback struct
> with
> > its functions and data and pass it to the lib by calling liblttd_start.
> >
> > That is it!
> >
> > What do you think about it?
>
> You'd also probably have to specify what guarantees you give in terms of
> threading, e.g.: no "on_read_subbuffer" callbacks can be called after
> the on_close_channel has been called sort-of-thing. This will help the
> lib user to figure out when it's quiescent.
>

We will do that.

>
> >
> > P.S. I think that it would be necessary for a program to have more than
> one
> > instance on a "deamon" running at the same time ??? In this case, what do
> > you think of an interface that give the possibility to create and to
> destroy
> > instance of the daemon? For example :
> >
> > struct liblttd_instance * liblttd_get_new_instance()
> > int liblttd_delete_instance(struct liblttd_instance *instance)
> > (we could add different function to modify the instance parameter (ex.
> > thread number, flight recorder only,...))
> > int liblttd_set_callbacks(struct liblttd_instance *instance, struct
> > liblttd_callbacks *callbacks)
> > int liblttd_start_instance(struct liblttd_instance *instance)
>
> With lttd, we simply (currently) spawn multiple lttd processes if we
> need that. Is it a limitation with TCF ? E.g. the use case is to start
> one lttd for a flight recorder trace while another lttd is already
> running gathering data for another trace session.
>

Yes, it is a problem in tcf because liblttd run in the same process as the
"main" program because we want to send the data directly to the tcf socket (
in the same process but in different threads). In this case, we can't simply
spawn multiple processes. I think that the best way to counter this problem
would be to create different instance of the "daemon" like I explain above.
What do you think?

>
> You doing good progress, I look forward to your updates.
>
> Thanks,
>
> Mathieu
>
>
>
We already have a working implementation of the new liblttd and lttd( that
use the lib) without the possibility to make multiple instances. After we
made the changes that you suggest, would you like us to send it to you
before we do anything else? This way you could tell if there is something
that you would like to change.


> >
> > Michael
>
> > _______________________________________________
> > ltt-dev mailing list
> > ltt-dev at lists.casi.polymtl.ca
> > http://lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev
>
>
> --
> Mathieu Desnoyers
> OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20100212/af4f434b/attachment-0003.htm>