[lttng-dev] relayd with many consumer daemons

[Jakub Witczak]

Hi,
 
> > I would like to consult on setup where relayd is serving multiple consumer
> daemons at the same time.
> 
> On the same host? On multiple hosts?
> In both cases, it is supported.

It's a setup where traces are transferred over the network to remote relayd.

> It is recommended to have unique session id across multiples
> sessiond/consumer on the same host but not required.

> > I assume above setup is nothing new and is not a problem as long as each
> of consumer daemons is having unique tracing session (please confirm).
> 
> As for multiple hosts, relayd also differentiate based on the hostname.
> 
> From lttng-relayd man page:
> 
>     By default, the relay daemon writes the traces to:
> 
>     $LTTNG_HOME/lttng-traces/HOSTNAME/SESSION/DOMAIN
> 
>     with:
> 
>     HOSTNAME
>         Remote hostname.
> 
>     SESSION
>         Full session name.
> 
>     DOMAIN
>         Tracing domain.
> 
>     You can override the default output directory prefix
>     ($LTTNG_HOME/lttng-traces) with the --output option. The other parts
>     depend on the remote configuration.
> 
> For example, user on host DATA1 create and start a tracing a session
> (live/streaming) with the name MySession.
> Relayd will create its internal state with the hostname, session name, time.
> Hence, trace data will be written somewhere like this:
>     $LTTNG_HOME/lttng-traces/DATA1/MySession-TIMESTAMP/DOMAIN
> 
> At the same time on host DATA2, a user create and start a tracing session
> (live/streaming) with the name MySession.
> Trace data will be written somewhere like this:
>     $LTTNG_HOME/lttng-traces/DATA2/MySession-TIMESTAMP/DOMAIN
> 
> >
> > But what if I would like to have more consumer daemons collecting data
> from *single session* - is it possible to aggregate data in relayd?
> 
> Note that Babeltrace can read and merge data from multiple sources:
>     babeltrace $LTTNG_HOME/lttng-traces/DATA1/MySession-TIMESTAMP
> $LTTNG_HOME/lttng-traces/DATA2/MySession-
> TIMESTAMP/DOMAIN<Paste>
> 
This looks very promising. Can it merge sessions in live mode as well?

> Not sure what you mean by "consumer daemons collecting data from *single
> session*".
> 
> Do you mean having multiple consumer "writing" to the same session on
> lttng-relayd side?
> What would motivate this design choice, is a limitation you encountered?
> 
> Is it not sufficient that multiple traces can be merged using babeltrace?

Yes. I meant multiple consumers (acting as single host but generated elsewhere (traffic NATed from different machine)) writing to single session in relayd.

Babeltrace merge could be enough, but hard to say without testing.

Regards
Kuba