[lttng-dev] [RFC] on-disk multiple tracefiles circular buffer

[Julien Desfossez]

This RFC discusses at high level the modification to be made to limit
the maximum size of tracefiles and the number of tracefiles to keep on disk.
The intent is to create an "on-disk circular buffer" with multiple
tracefiles, thus limiting the maximum disk usage of traces.

The semantic is similar to the -C and -W flags of tcpdump. In tcpdump,
-C limits the maximum file size and -W the amount of such files to keep
on disk. The -W flag is not mandatory with the -C, so it is possible to
have an "infinite" number of data files of a specific size which is
useful for analysis afterwards.

In LTTng, we have one stream (tracefile) per-channel per-CPU, so the
restrictions will apply for all the streams of a channel. For example,
if in a channel we want 5 tracefiles of 100MB each and we have 4 CPUs,
we will have a maximum of 20 tracefiles of 100MB (and a single metadata
file).

Each tracefile will be readable independently or with the others (and
correctly sorted) as long as the metadata file is available.

This functionnality will be enabled by channel in LTTng, so the proposed
approach is to add parameters to the channel creation calls (UI and API)
and make the streams inherit these parameters so that the consumer and
the relay will be able to handle the tracefiles accordingly.

The proposed options names are :
--tracefile_size
--tracefile_count

Or maybe add "max_" in there : max_tracefile_size, max_tracefile_count ?

The tracefile_count is ignored if the tracefile_size is not specified.
The tracefile will be splitted in a "infinite" number of files if
tracefile_size is specified and tracefile_count is omitted.

At the end of the tracefile name, we will add the number of tracefile,
for example the 5th tracefile on CPU 2 of channel "channel0" will be
named "channel0_2.5".

When a tracefile must be reused (after we reach the tracefile_count),
its content is truncated on open so the old data is no longer available
even if we don't overwrite it completely.

So this is the high-level breakdown of this new feature, if you have any
comments, please send them asap.

Thanks,

Julien