[lttng-dev] LTTng buffer mode

[Mathieu Desnoyers]

----- On Feb 26, 2016, at 4:12 AM, syed zaidi <zaidi1039 at gmail.com> wrote: 

> Hi Mathieu,
> I have few questions related to the questions above and please forgive me for my
> ignorance in advance.
> 1. When LTTng creates the buffers lets say per uid, the files that show up in
> LTTng folder, are those files that LTTng has stopped writing in? What I mean
> is, are those files still being written to or they are safe to be moved.

They are still being written to. 

> 2. If I want to collect traces lets say every 5 mins for the last 5 mins while
> LTTng keeps running. What is the best approach? Do I just run BabelTrace on the
> folder and remove all files from the folder to create space for next files and
> keep repeating?

I would not use Babeltrace on the target, but rather just move the CTF files. 

You could swap between two sessions to do that: lttng supports multiple 
concurrent sessions. Create and start a new session before you stop, 
destroy, and move away the old session, every 5 minutes... 

> 3. I understand that I should delete the corresponding .idx file for each log
> file that I remove, what about metadata file? Do I remove it or leave it as is?

You need to leave it as is. 

> 4. The trace file produced by Babeltrace, does it guarantee that the file has
> traces from all the users and the core till the time stamp of the last line?'
> For example if the last line of the trace file looks like this
> 23:00:00.912089772,...,...
> Does that mean that the trace file has all the traces until that time?

It depends on your tracing mode (discard vs snapshot buffers). In discard 
mode (default if you don't use the --snapshot option on lttng create), 
Babeltrace will print on stderr whenever it encounters discarded events. 

Best regards, 

Mathieu 

> Thanks
> Ammar

> On Sat, Feb 20, 2016 at 10:17 AM, Mathieu Desnoyers <
> mathieu.desnoyers at efficios.com > wrote:

>> Hi,

>> LTTng implements per-cpu buffers to eliminate false-sharing between
>> CPUs. Babeltrace and Trace Compass implement iterators that allow
>> doing the merge of those per-cpu buffers into a single stream on
>> the fly. Users of Babeltrace and Trace Compass should not have to
>> worry about doing this merge.

>> Since LTTng is inherently multi-buffers, we also use this buffer
>> merging scheme to deal with tracefile rotation, multi-channel, and
>> combining traces gathered from various processes/user IDs/kernel vs
>> userspace.

>> So my question here is: why you say it is much more difficult for
>> you to deal with those multiple buffers ? Since there are already
>> libraries in Babeltrace that do that for you, I would expect this
>> to be a non-issue.

>> Although it would be technically possible to implement a ring
>> buffer shared across CPUs, it would be inefficient, and we would
>> need a good justification for adding this complexity to the
>> tracers.

>> Thanks,

>> Mathieu

>> ----- On Feb 19, 2016, at 4:00 AM, Jeffrey Chen cpthk at hotmail.com wrote:

>> > Thanks.
>> > If we make changes to LTTng to add this feature, would it violate the LTTng
>> > design principle in any way? Thanks.


>> > ________________________________________
>> > From: Philippe Proulx < eeppeliteloop at gmail.com >
>> > Sent: Tuesday, February 16, 2016 5:02 PM
>> > To: Jeffrey Chen
>> > Cc: lttng-dev at lists.lttng.org
>> > Subject: Re: [lttng-dev] LTTng buffer mode

>> > No you cannot.

>> > There's no hidden global buffering scheme option for user space tracing.

>> > From the LTTng documentation:

>> >> In the user space tracing domain, two buffering schemes are available
>> >> when creating a channel:

>> >> Per-PID buffering: keep one ring buffer per process.
>> >> Per-UID buffering: keep one ring buffer for all processes of a single user.

>> > and:

>> >> The Linux kernel tracing domain only has one available buffering scheme
>> >> which is to use a single ring buffer for the whole system.

>> > Phil

>> > On Tue, Feb 16, 2016 at 7:55 PM, Jeffrey Chen < cpthk at hotmail.com > wrote:
>> >> Hi LTTng community:


>> >> I have a question about LTTng buffer mode. According to documentation, LTTng
>> >> supports 2 buffer modes, per-pid, and per-uid. Is there any mode to have a
>> >> global buffer that does not have separate buffers? The reason I asked is
>> >> because, LTTng writes different buffer traces to different directory/file.
>> >> It is much more difficult for us to post-process separate directories. We
>> >> would like to have all traces write into one single buffer and
>> >> directory/file, so it is easier to do post-processing. Thanks.



>> >> _______________________________________________
>> >> lttng-dev mailing list
>> >> lttng-dev at lists.lttng.org
>> >> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev


>> > _______________________________________________
>> > lttng-dev mailing list
>> > lttng-dev at lists.lttng.org
>> > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

>> --
>> Mathieu Desnoyers
>> EfficiOS Inc.
>> http://www.efficios.com

>> _______________________________________________
>> lttng-dev mailing list
>> lttng-dev at lists.lttng.org
>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

-- 
Mathieu Desnoyers 
EfficiOS Inc. 
http://www.efficios.com 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lttng.org/pipermail/lttng-dev/attachments/20160226/e93d0fe0/attachment-0001.html>