<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;" dir="ltr">
<p style="margin-top:0;margin-bottom:0">Welcome Jonathan,</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<p style="margin-top:0;margin-bottom:0">Also, for people seeking great babeltrace support, I feel I should plug the lttng-dev mailinglist.
<br>
</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<p style="margin-top:0;margin-bottom:0">I hope the URL is not censored. <a href="https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev" class="OWAAutoLink" id="LPlnk853878" previewremoved="true">
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev</a> if it is, just search for "<b><font size="+1" color="#000000">lttng-dev -- LTTng development list</font></b>"<br>
</p>
<div id="LPBorder_GT_15524033734260.13838571933289845" style="margin-bottom: 20px; overflow: auto; width: 100%; text-indent: 0px;">
<table id="LPContainer_15524033734220.025667838474824234" style="width: 90%; background-color: rgb(255, 255, 255); position: relative; overflow: auto; padding-top: 20px; padding-bottom: 20px; margin-top: 20px; border-top: 1px dotted rgb(200, 200, 200); border-bottom: 1px dotted rgb(200, 200, 200);" role="presentation" cellspacing="0">
<tbody>
<tr style="border-spacing: 0px;" valign="top">
<td id="TextCell_15524033734240.05101294807418255" style="vertical-align: top; position: relative; padding: 0px; display: table-cell;" colspan="2">
<div id="LPRemovePreviewContainer_15524033734240.45704162409583526"></div>
<div id="LPTitle_15524033734240.768426324418359" style="top: 0px; color: rgb(0, 75, 139); font-weight: 400; font-size: 21px; font-family: "wf_segoe-ui_light", "Segoe UI Light", "Segoe WP Light", "Segoe UI", "Segoe WP", Tahoma, Arial, sans-serif; line-height: 21px;">
<a id="LPUrlAnchor_15524033734240.08400261288183442" style="text-decoration: none;" href="https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev" target="_blank">lttng-dev Info Page</a></div>
<div id="LPMetadata_15524033734250.5254476983267053" style="margin: 10px 0px 16px; color: rgb(102, 102, 102); font-weight: 400; font-family: "wf_segoe-ui_normal", "Segoe UI", "Segoe WP", Tahoma, Arial, sans-serif; font-size: 14px; line-height: 14px;">
lists.lttng.org</div>
<div id="LPDescription_15524033734250.43499263846109015" style="display: block; color: rgb(102, 102, 102); font-weight: 400; font-family: "wf_segoe-ui_normal", "Segoe UI", "Segoe WP", Tahoma, Arial, sans-serif; font-size: 14px; line-height: 20px; max-height: 100px; overflow: hidden;">
To see the collection of prior postings to the list, visit the lttng-dev Archives.. Using lttng-dev: To post a message to all the list members, send email to lttng-dev@lists.lttng.org. You can subscribe to the list, or change your existing subscription, in
the sections below.</div>
</td>
</tr>
</tbody>
</table>
</div>
<br>
<p></p>
</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Jonathan Rajotte-Julien <jonathan.rajotte-julien@efficios.com><br>
<b>Sent:</b> Tuesday, March 12, 2019 11:03:14 AM<br>
<b>To:</b> Doug Schaefer; tracecompass developer discussions<br>
<b>Cc:</b> Matthew Khouzam; lttng-dev<br>
<b>Subject:</b> Re: [tracecompass-dev] Scalability</font>
<div> </div>
</div>
<div class="BodyFragment"><font size="2"><span style="font-size:11pt;">
<div class="PlainText">Hi Doug,<br>
<br>
On Tue, Mar 12, 2019 at 02:56:26PM +0000, Doug Schaefer wrote:<br>
> Thanks, Matthew, I ended up flushing every 100K events and that seemed OK.<br>
> <br>
> My biggest worry is on the read side. Babeltrace blew up on that file. It's a pretty simple trace (for now) with a single event class with 4 ints and a sequence of 32-bit unsigned ints which is usually only 2 elements long.<br>
<br>
This is not I expect. Still eithet jgalar or eep might have a more insight on<br>
this. CCing the lttng-dev mailing list.<br>
<br>
Is there any way to share us a similar trace? Either with a generator or we can<br>
provide a link for you to upload it. The current limit on bugs.lttng.org is a<br>
bit too small for such trace.<br>
<br>
> <br>
> Aside from that, am very pleased with the how easy CTF is to work with. Looking forward to doing more.<br>
> <br>
> Doug.<br>
> <br>
> On Tue, 2019-03-12 at 14:41 +0000, Matthew Khouzam wrote:<br>
> <br>
> Hi Doug,<br>
> <br>
> <br>
> Great to hear you're coming to a standard! I don't know if trace compass will scale properly as I don't know what the trace configuration is.<br>
> <br>
> <br>
> > I have one of our trace which has 16 million events, the size of each event is around 32 bytes giving me a 540MB file. My first attempt at simply writing out the CTF events without a flush ran out of virtual memory. I then flushed after every event which
made each event take 32K. So I found a middle ground and the resulting stream file is close to the same size.<br>
> <br>
> <br>
> <br>
> My suggestion is to have 1 MB packets. This makes seeking very efficient. If each event is 32 bytes, basically flush every 25k events or so.<br>
> <br>
> <br>
> Please keep us posted!<br>
> <br>
> <br>
> Matthew.<br>
> <br>
> ________________________________<br>
> From: tracecompass-dev-bounces@eclipse.org <tracecompass-dev-bounces@eclipse.org> on behalf of Doug Schaefer <dschaefer@blackberry.com><br>
> Sent: Tuesday, March 12, 2019 10:30:15 AM<br>
> To: tracecompass-dev@eclipse.org<br>
> Subject: [tracecompass-dev] Scalability<br>
> <br>
> Hey gang,<br>
> <br>
> We're finally starting to look at converting our custom traces into CTF so we can leverage tools like TraceCompass and, of course, contribute to it. One thing I quickly ran into is a scalability issue I'm seeing with libbabeltrace.<br>
> <br>
> I have one of our trace which has 16 million events, the size of each event is around 32 bytes giving me a 540MB file. My first attempt at simply writing out the CTF events without a flush ran out of virtual memory. I then flushed after every event which
made each event take 32K. So I found a middle ground and the resulting stream file is close to the same size.<br>
> <br>
> But when I use babeltrace to print it out, I ran out of virtual memory. I then hand coded a reader and simply adding the trace to the context caused the memory issue. It really looks like libbabeltrace (version 1.5 from the Ubuntu 18.04 distro), tries to
inflate the events into it's internal representation for the entire trace. I need to do more investigation to confirm that.<br>
> <br>
> So my question for this list, would TraceCompass do better? Does it have it's own parsing libraries?<br>
> <br>
> Thanks,<br>
> Doug<br>
<br>
> _______________________________________________<br>
> tracecompass-dev mailing list<br>
> tracecompass-dev@eclipse.org<br>
> To change your delivery options, retrieve your password, or unsubscribe from this list, visit<br>
> <a href="https://www.eclipse.org/mailman/listinfo/tracecompass-dev">https://www.eclipse.org/mailman/listinfo/tracecompass-dev</a><br>
<br>
<br>
-- <br>
Jonathan Rajotte-Julien<br>
EfficiOS<br>
</div>
</span></font></div>
</body>
</html>