[lttng-dev] [RFC babeltrace 1/2] fs-src: add argument for metadata src dir

[Philippe Proulx]

On Tue, May 1, 2018 at 11:12 AM, Alexander Aring <aring at mojatatu.com> wrote:
> Hi,
>
> On Mon, Apr 30, 2018 at 06:36:08PM -0400, Philippe Proulx wrote:
>> On Mon, Apr 30, 2018 at 5:31 PM, Alexander Aring <aring at mojatatu.com> wrote:
>> >
>> > This patch adds an argument for the fs-src plugin to declare the
>> > directory to find the metadata file instead of placing it in every
>> > subdir of traces.
>> >
>> > If parameter assign every subdirectory which does not have a
>> > subdirectory and at least some regular files will be assumed as a trace
>> > directory. The regular files will be assumed as ctf trace files.
>>
>> Although I really appreciate the contribution effort, before even
>> reading this patch, let me indicate that a CTF trace located on the file
>> system is a directory containing the data stream and metadata stream
>> files. Knowing this, this patch implements a hack to circumvent this,
>> but I'm personally not willing to have this upstream as, in my opinion,
>> it is the user's responsibility to have valid CTF traces to open.
>>
>
> Yea, I already thought that will happen... but I want to talk about my
> use-case and how to handle it with babeltrace.
>
> My use-case is the following:
>
>  - I use barectf with a fs platform based platform. I want to contribute
>    them back to barectf if this is welcome. I see I already have the
>    barectf expert here. I use it for userspace application trace.
>
>  - I have a distributed application which stores the stream files in a
>    directory with the scheme:
>
>    $TRACES/
>            $HOSTNAME/
>                      ....
>                      $APPNAME_$PID_stream
>
> Currently I mostly run my application on one host with one clock source,
> which makes everything about clock handling very simple.
>
> So far I know the multiple streams and merge has the use case to have a
> trace per cpu, well we do it per application... I hope this is okay.
>
>> Your custom scenario asks for a custom solution on your side: copying
>> the metadata file to each trace's directory seems appropriate here.
>>
>
> Really?
>
> I thought about that I cannot use traces on a read-only filesystem,
> because I am not allowed to copy there. One example where my solution
> hits it's limitation...
>
> I also thought about: using barectf and generate a object file linked
> to my platform who has the metadata file inside and will be created to
> my stream file when barectf init's (if it's not already exists).
> So far I see it could be done because metadata file is compile time
> related and binded to barectf generated code.

Yes this is a better idea. If you cannot postprocess the trace directories,
than it's better that you create valid traces in the beginning.

>
> That will bloat my binary as one disadvantage, otherwise I can be sure
> there is no mixed metadata handling everywhere.

Can you create symbolic links on this filesystem (when you have write
access)? This would avoid bloating.

>
>> Also, do your CTF traces have UUIDs? If they do, they should all be
>> different, but having the same metadata file makes them all the same.
>> This is not valid either.
>>
>
> I have UUID but my applications use all the same metadata file by
> barectf. :-) I hope this is a valid point?

Normally, a trace has a unique UUID. If you cannot ensure this, I
suggest that you remove the UUID field from the packet header.

Phil

>
> I admit, currently I have mostly functionality to make backwards
> compatibility with an old trace handling. It's just put some ascii info
> inside the stream. Later I want to move ascii into metadata description.
> It works like tracelog [0], but just temporary.
>
> - Alex
>
> [0] http://man7.org/linux/man-pages/man3/tracelog.3.html