[lttng-dev] [tracecompass-dev] Scalability

[Philippe Proulx]

On Wed, Mar 13, 2019 at 8:13 AM Matthew Khouzam
<matthew.khouzam at ericsson.com> wrote:
>
> Hi Doug & Efficios,
>
>
> I suspect the issue is with libbabeltrace-ctf. When opening the trace in Trace Compass, it failed saying the sequence was 2gb big. When opening in babeltrace, it OOMed with 2gb allocations. When opening with a hex editor, I saw 2gb allocations.

Yes, byte order issues. Here's the fix:
<https://github.com/efficios/babeltrace/pull/103>.

Phil

>
>
> On another subject, fixing the trace made it scale well in both babeltrace and Trace Compass.
>
>
> I don't know why, data_length writes the wrong data. Hope this helps all parties.
>
>
> BR,
>
> Matthew
>
> ________________________________
> From: tracecompass-dev-bounces at eclipse.org <tracecompass-dev-bounces at eclipse.org> on behalf of Doug Schaefer <dschaefer at blackberry.com>
> Sent: Tuesday, March 12, 2019 2:50:29 PM
> To: jonathan.rajotte-julien at efficios.com
> Cc: lttng-dev at lists.lttng.org; tracecompass-dev at eclipse.org
> Subject: Re: [tracecompass-dev] [lttng-dev] Scalability
>
> I have attached the C file for a generator that creates a trace that reproduces the memory issue with babeltrace. The same issue happens with a custom program in the call to add a trace to a context.
>
> I am using babeltrace that I get from my Ubuntu 18.04 distro. It has the version number 1.5.5.
>
> Thanks for the help!
>
> On Tue, 2019-03-12 at 15:47 +0000, Doug Schaefer wrote:
>
> In the thread on the tracecompass list, I also mentioned that the babeltrace utility itself ran out of memory when reading the file. That's actually my biggest worry. I got around the generator issue by flushing every 100K events.
>
> I'll create a quick generator that shows the same issue and post it in a bit.
>
> Doug.
>
> On Tue, 2019-03-12 at 11:34 -0400, Jonathan Rajotte-Julien wrote:
>
> On Tue, Mar 12, 2019 at 03:23:23PM +0000, Doug Schaefer wrote:
>
> It should be easy to generate a random one with the class I mentioned below
>
> and fill it with 16 million events.
>
>
> Not if the problem is in the generator code ;).
>
>
> BTW, two of the int fields are 5 and 10
>
> bits respectively, but I'm not sure that matters (or at least it shouldn't).
>
>
> Make sure to check the endian-ness of those fields.
>
>
> What class? Not sure I see any reference to any.
>
>
> Since you are generating CTF the error can be in the generator code. Please
>
> provide either a trace reproducing the issue or a base reproducer. It will be
>
> much easier overall than us trying to reproduce it blindly.
>
>
> Cheers.
>
>
> _______________________________________________
>
> lttng-dev mailing list
>
> lttng-dev at lists.lttng.org
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.lttng.org_cgi-2Dbin_mailman_listinfo_lttng-2Ddev&d=DwICAg&c=yzoHOc_ZK-sxl-kfGNSEvlJYanssXN3q-lhj0sp26wE&r=NrrbvTHWa2Nbp_kAN0Hl1o3lM1WAwSes64uBjxjNhMc&m=DWwlFfl5FD5R8-HPY3GfbG6l6m7y_o2gXPER8IkQstk&s=bPK6X3vufUtEAemGpslNIlIL4UslBsgK-SWnraL-e9g&e=
>
>
> ---------------------------------------------------------------------
> This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful.
> _______________________________________________
> lttng-dev mailing list
> lttng-dev at lists.lttng.org
> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev