[lttng-dev] A question on protocols

[mrx]

On Mon, Nov 19, 2018 at 7:46 PM Jonathan Rajotte-Julien <
jonathan.rajotte-julien at efficios.com> wrote:

> Hi Patrick,
>
> > The security between consumerd and relayd isn't an issue as we'd be
> forced
> > to keep both on the device. This because we have no way of transporting
> the
> > consumerd <-> relayd communication over the HTTP proxy, which is our only
> > choice. We have already talked to the team in charge of the network
> paths.
>
> In that case using lttng-relayd yield no advantage unless you are able to
> have
> a device running only lttng-relayd and gathering data from other devices
> and
> then sending the data via http.
>

Unfortunately, we're don't have any external device that can run the
lttng-relayd component.


>
>
> >
> > The relayd side have to be on the device. The device has very limited
> > amounts of free flash. Even if there were flash available it would wear
> > down the flash faster than what we'd like.
>
> Not sure I follow here. The whole point of lttng-relayd is to be off the
> device
> being traced and to relay trace data using network only (skipping disk).
>

Sure, so given this I cannot use relayd for our purposes. The only other
option would be to set LTTng to local tracing, which we don't have flash
for. My hope was that we could somehow stream the CTF records somewhere
else, over that HTTP proxy, ot be analyzed. As you explained earlier, the
relayd can be viewed as a FTP server of sorts. So that will most likely not
work.


>
> > A RAM-based disc might have been a solution if we would have had any
> > mentionable amounts of RAM to spare for this. Unfortunately we don't have
> > that.
>
> In all cases you will need some RAM somewhere if using LTTng (and I think
> any
> other monitoring/tracing tools) since we need to allocate tracing buffer.
>

We have RAM for the tracing buffer, no issues there. The tracing buffer is
just a temporary medium until you get the trace on disc, possibly via
relayd. Correct?

I interpretted your RAM disc suggestion as a place where I could store the
trace instead of using flash.


>
> > >
> > > Keep in mind that trace production is normally much more quicker than
> trace
> > > reading/analysis. A buffering scheme is mostly always necessary.
> > >
> >
> > Very valid point, I'll for sure keep that in mind.
> >
> > This speaks strongly against replacing relayd with something homebrewed,
> as
> > was the original plan. We'd probably not be able to shuffle the data fast
> > enough over the network to keep up with the pace.
>
> If you are unable to "reserve" RAM/network bandwidth and/or use FLASH this
> is
> slowly becoming a "you can't have your cake and eat it too" situation.
> This must
> be frustrating...
>
> > >
> > > You can also use the --trace-file-size and --trace-file-count to limit
> the
> > > disk
> > > footprint of each live session. Make sure to have enough buffer for
> live
> > > reading if still using live.
> > >
> >
> > This isn't an option as our disc is flash based and we'd like to limit
> the
> > wear due to collecting metrics.
>
> I was under the impression that the lttng-relayd process was on another
> device.
> I misunderstood the situation.
>

Perhaps i could have been more clear on the situation as well.

We have LTTng tracing capabilites on our devices. During development we can
use LTTng on the device and relayd on our dev machines and everything works
great.

In production, the situation is different.

The only means we have of transporting anything from the device is a HTTP
proxy. We're trying to find a way where we can keep a conitnuous LTTng
trace on the device, tunnel it through the HTTP proxy and analyze the data
remotely.

The devices are in the embedded realm, limited amounts of RAM, CPU, and
flash. This limits what we can do on the device when it comes to storage
for example.

Hopefully I've made our situation a bit more clear.

Based on our discussion so far, I have to say that LTTng is starting to
look less and less as a viable way forward for us.

Regards,
Patrik
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.lttng.org/pipermail/lttng-dev/attachments/20181120/c8c80338/attachment.html>