[lttng-dev] [RFC babeltrace 1/2] fs-src: add argument for metadata src dir

[Alexander Aring]

Hi,

On Mon, Apr 30, 2018 at 06:36:08PM -0400, Philippe Proulx wrote:
> On Mon, Apr 30, 2018 at 5:31 PM, Alexander Aring <aring at mojatatu.com> wrote:
> >
> > This patch adds an argument for the fs-src plugin to declare the
> > directory to find the metadata file instead of placing it in every
> > subdir of traces.
> >
> > If parameter assign every subdirectory which does not have a
> > subdirectory and at least some regular files will be assumed as a trace
> > directory. The regular files will be assumed as ctf trace files.
> 
> Although I really appreciate the contribution effort, before even
> reading this patch, let me indicate that a CTF trace located on the file
> system is a directory containing the data stream and metadata stream
> files. Knowing this, this patch implements a hack to circumvent this,
> but I'm personally not willing to have this upstream as, in my opinion,
> it is the user's responsibility to have valid CTF traces to open.
> 

Yea, I already thought that will happen... but I want to talk about my
use-case and how to handle it with babeltrace.

My use-case is the following:

 - I use barectf with a fs platform based platform. I want to contribute
   them back to barectf if this is welcome. I see I already have the
   barectf expert here. I use it for userspace application trace.

 - I have a distributed application which stores the stream files in a
   directory with the scheme:

   $TRACES/
           $HOSTNAME/
	             ....
	             $APPNAME_$PID_stream

Currently I mostly run my application on one host with one clock source,
which makes everything about clock handling very simple.

So far I know the multiple streams and merge has the use case to have a
trace per cpu, well we do it per application... I hope this is okay.

> Your custom scenario asks for a custom solution on your side: copying
> the metadata file to each trace's directory seems appropriate here.
> 

Really?

I thought about that I cannot use traces on a read-only filesystem,
because I am not allowed to copy there. One example where my solution
hits it's limitation...

I also thought about: using barectf and generate a object file linked
to my platform who has the metadata file inside and will be created to
my stream file when barectf init's (if it's not already exists).
So far I see it could be done because metadata file is compile time
related and binded to barectf generated code.

That will bloat my binary as one disadvantage, otherwise I can be sure
there is no mixed metadata handling everywhere.

> Also, do your CTF traces have UUIDs? If they do, they should all be
> different, but having the same metadata file makes them all the same.
> This is not valid either.
> 

I have UUID but my applications use all the same metadata file by
barectf. :-) I hope this is a valid point?

I admit, currently I have mostly functionality to make backwards
compatibility with an old trace handling. It's just put some ascii info
inside the stream. Later I want to move ascii into metadata description.
It works like tracelog [0], but just temporary.

- Alex

[0] http://man7.org/linux/man-pages/man3/tracelog.3.html