[lttng-dev] relayd with many consumer daemons

[Jonathan Rajotte-Julien]

Hi Kuba,

> > 
> > Note that Babeltrace can read and merge data from multiple sources:
> >     babeltrace $LTTNG_HOME/lttng-traces/DATA1/MySession-TIMESTAMP
> > $LTTNG_HOME/lttng-traces/DATA2/MySession-
> > TIMESTAMP/DOMAIN<Paste>
> > 
> This looks very promising. Can it merge sessions in live mode as well?

Unfortunately Babeltrace 1.x does not support multiple input for lttng-live.

Babeltrace 2 does support reading from multiple sources for live session.
Babeltrace 2 is currently in its pre-release phase and is considered
experimental.

> 
> > Not sure what you mean by "consumer daemons collecting data from *single
> > session*".
> > 
> > Do you mean having multiple consumer "writing" to the same session on
> > lttng-relayd side?
> > What would motivate this design choice, is a limitation you encountered?
> > 
> > Is it not sufficient that multiple traces can be merged using babeltrace?
> 
> Yes. I meant multiple consumers (acting as single host but generated elsewhere (traffic NATed from different machine)) writing to single session in relayd.

lttng-relayd will always append a timestamp to the session name on session
creation on its side. Hence, even if all hosts have the same hostname and the same
session name, the actual trace data will be separated in two different folder on the
lttng-relayd host.

As for the lttng-live merge reading with babeltrace 1.x,
I did a simple experiment locally and this can "work" if and only if all host have
the same hostname and session name. Given that babeltrace 1.x does not support
multiple input for lttng-live it could be a workaround.
**** I do not recommend it and is not supported officially.****

If your use case is streaming traces to relayd and reading it offline you are better
of with using babeltrace for doing the merge without any "hack" regarding
hostname etc. for lttng-live reading.

Note that babeltrace by default will read all traces under the directory you are
passing as argument. This coupled with the --output option of lttng-relayd can
be an easy way to bundle all related traces together and read them in one go.

Cheers

> 
> Babeltrace merge could be enough, but hard to say without testing.
> 
> Regards
> Kuba
> 

-- 
Jonathan Rajotte-Julien
EfficiOS