[lttng-dev] Processing large trace files

[Philippe Proulx]

On Tue, Jul 10, 2018 at 11:33 AM Shehab Elsayed <shehabyomn at gmail.com> wrote:
>
> I haven't tried the C API yet.
>
> A smaller trace of about 2.7GB took +5 hrs before I had to kill the process.

Wow. The Python bindings are slow by nature, but that's a lot of time!

>
> As for trace cutting, that sounds promising. How broken is it in babeltrace 2? I mean, can I use it to some extent or is it not functioning at all?

It works in fact, but the master branch of Babeltrace 2 does not contain
some important optimizations yet so it can be slow, which is already
your issue, therefore I can't say if it will help.

Once built, the command line would look like:

    babeltrace /path/to/trace/dir --begin=20:32:17.182738475 \
               --end=20:32:24.177283746 -octf -w/output/dir

See babeltrace-convert(1) for more details.

Try it and see if you can save time. Moreover I'm working on various
optimizations, one of which targets faster trimming. This will be part
of 2.0-rc1.

Phil

>
> Thanks!
>
> Shehab Y. Elsayed, MSc.
> PhD Student
> The Edwards S. Rogers Sr. Dept. of Electrical and Computer Engineering
> University of Toronto
> E-mail: shehabyomn at gmail.com
>
> On Tue, Jul 10, 2018 at 9:37 AM, Jonathan Rajotte-Julien <jonathan.rajotte-julien at efficios.com> wrote:
>>
>> Hi Shehab,
>>
>> On Mon, Jul 09, 2018 at 06:48:05PM -0400, Shehab Elsayed wrote:
>> > Hello All,
>> >
>> > I was wondering if anyone had any suggestions on how to speedup trace
>> > processing. Right now I am using babeltrace in python to read the traces
>> > and process them. Some of the traces I am dealing with reach up to 10GB.
>>
>> Well Babeltrace also provide a C API. You might get some performance gains there.
>>
>> Depending of the data you extract you might be able to split the 10 GB trace
>> into smaller chunk and parallelize your analysis. Babeltrace 2 will support
>> trace cutting, it seems that it is broken for now. Babeltrace 1 does not support
>> trace cutting.
>>
>> You might want to look at ways to reduce the size of your traces using tracing
>> mechanism such as the snapshot feature and enabling/recording only pertinent
>> event if it is not already the case.
>>
>> Could you share with us the time it take for analyzing your bigger trace?
>>
>> Cheers
>>
>> >
>> > I read the trace, extract the info I need and write them to a csv gzipped
>> > file (all in python). I fill up a list first with all extracted info for
>> > each tracked event and then write once after the script has gone through
>> > all events in the trace.
>> >
>> > Thanks,
>> > Shehab
>>
>> > _______________________________________________
>> > lttng-dev mailing list
>> > lttng-dev at lists.lttng.org
>> > https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>
>>
>> --
>> Jonathan Rajotte-Julien
>> EfficiOS
>
>
> _______________________________________________
> lttng-dev mailing list
> lttng-dev at lists.lttng.org
> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev