[lttng-dev] Duplicate traces in multiple sessions

[Julien Desfossez]

Hi,

> We are using LTTng to collect traces from our non-stopping service (our
> service is running 24/7, highly available). We collect traces by rotate
> the trace in an interval. We do this by creating new session, stop the
> old session, and process the old traces, every 10 minutes.
> There are overlaps on the session since we create new session before
> stopping the old session. We are seeing some traces exist in both the
> old session, and new session. This is expected. We also noticed that
> these duplicated traces could have slightly different timestamp in the
> two sessions, so it is very difficult to do post-processing to remove
> the duplicates.
Yes, this is normal since you are enabling 2 probes during the overlap
and each probe reads its own timestamp.

> Assuming we could read the raw binary trace files, we would like to know
> whether this is any way to identify the duplicating traces. Is there any
> kind of unique id or hash that could be used to identity the duplicate
> traces in two sessions?
There is currently no clean way to match those duplicate events.

We have already discussed about the kind of use-case you are talking
about and we have an idea to introduce a kind of "rotate" command for a
session. This would allow to make a clean transition to a new set of
trace files with no overlap, but it is not on our short term priority list.

Thanks,

Julien