[lttng-dev] Human read-writeable format for CTF traces

Tue Feb 4 12:46:05 EST 2014

On Tue, Feb 4, 2014 at 12:35 PM, Geneviève Bastien
<gbastien+lttng at versatic.net> wrote:
> Hi Jeremie,
>
> Thanks for your reply.
>
>
> On 02/04/2014 11:19 AM, Jérémie Galarneau wrote:
>>
>> Hi Geneviève,
>>
>> I wonder if we are not overly complicating things here... The Python
>> bindings seem to address your immediate testing concerns and will let
>> you produce test traces with minimal effort.
>
> Do you have any information, links, examples how to use what exists in
> python to generate traces with minimal effort? Our very immediate need is
> actually to generate traces manually, because we don't need to know how to
> script in python to do this ;-) and we can manually modify the events. But
> since I have no idea (yet) what those python bindings do and how they do it,
> then maybe I miss something here.
>

They are part of the current Babeltrace master branch and will be
included in the next release. An example script is available under
babeltrace/bindings/python/examples/ctf_writer.py.
Let me know if you have any questions.

Regards,
Jérémie

> Thanks,
> Geneviève
>
>> While I agree that it doesn't solve the problem of testing analysis on
>> Windows, I must ask if this really is a primary concern at this point.
>>
>> I would personally start by writing tests in Python using the
>> CTF-Writer bindings and then, as the analysis feature gains traction,
>> work with users to determine the best testing strategy. I somehow
>> doubt that having an external dependancy on Babeltrace would really be
>> a problem to these power users...
>>
>> Sorry for the late reply,
>> Jérémie
>>
>> On Mon, Feb 3, 2014 at 12:20 PM, Geneviève Bastien
>> <gbastien+lttng at versatic.net> wrote:
>>>
>>> On 02/03/2014 12:16 PM, Mathieu Desnoyers wrote:
>>>>
>>>> ----- Original Message -----
>>>>>
>>>>> From: "Geneviève Bastien" <gbastien+lttng at versatic.net>
>>>>> To: "Michel Dagenais" <michel.dagenais at polymtl.ca>, "Mathieu Desnoyers"
>>>>> <mathieu.desnoyers at efficios.com>
>>>>> Cc: lttng-dev at lists.lttng.org
>>>>> Sent: Monday, February 3, 2014 12:05:38 PM
>>>>> Subject: Re: [lttng-dev] Human read-writeable format for CTF traces
>>>>>
>>>>> Ok, I'll wait for Jérémie's answer for more details. As I said, my
>>>>> concern is to have something fully standalone in TMF. But if one has
>>>>> access to babeltrace and eventual plugins to read-write a CTF trace to
>>>>> XML, then all the better. We could then import an XML generated by a
>>>>> python script into TMF, edit it there and then use it to test analyses.
>>>>>
>>>>> All we have to settle on is the intermediate format that should be
>>>>> used.
>>>>> I'd go for XML because of the possibility to validate it and have
>>>>> visual
>>>>> editors.
>>>>
>>>> Michel's idea of going for Python seems even better to generate test
>>>> suites.
>>>> It would allow importing and combining test "patterns" very easily, thus
>>>> allowing to create tests by construction without having to copy-paste
>>>> huge
>>>> XML files.
>>>>
>>>> I don't clearly see why having external dependencies on other tools
>>>> for a TMF CI test suite would be an issue. What would be the main
>>>> arguments
>>>> for having all those tests stand-alone in TMF for the test-suite ?
>>>
>>> It is not just for test suite. XML-defined analysis will need test traces
>>> as
>>> well, and that is in main TMF, not in unit tests (one idea of the XML
>>> analysis is to allow end-user to develop their own analysis without
>>> writing
>>> a single line of code or requiring the TMF development environment). And
>>> the
>>> user of these analysis and test traces may not have access to babeltrace
>>> or
>>> even to a Linux command line.
>>>
>>> Thanks,
>>> Geneviève
>>>
>>>> Thanks,
>>>>
>>>> Mathieu
>>>>
>>>>> On 02/03/2014 11:00 AM, Michel Dagenais wrote:
>>>>>>>
>>>>>>> I would expect that the ctf writer API recently added to babeltrace
>>>>>>> (currently in master branch), along with the Python bindings that
>>>>>>> cover
>>>>>>> trace read and write APIs, should allow you to implement things like:
>>>>>>>
>>>>>>> - A plugin to read a CTF trace, and output it in an intermediate
>>>>>>> format
>>>>>>>      to facilitate edits (e.g. XML as you propose),
>>>>>>> - A plugin to read this XML format and output a CTF trace.
>>>>>>
>>>>>> Yes, this would indeed be extremely helpful, in XML and/or JSON.
>>>>>>
>>>>>>> You could also generate the XML trace completely by hand if you like,
>>>>>>> and
>>>>>>> then convert it to CTF with the second plugin I'm relating to above.
>>>>>>
>>>>>> The likely scenario is to add a few events by hand.
>>>>>>
>>>>>>> Another possibility is that the XML description also allows
>>>>>>> describing what the trace contains at a slightly higher level. For
>>>>>>> instance, if you
>>>>>>> have a periodic event happening for a certain amount of time, it
>>>>>>> would
>>>>>>> be described in XML, and then "generated" by the XML-to-CTF
>>>>>>> converter.
>>>>>>
>>>>>> Do we want to describe this in XML or in Python? We could have "CTF"
>>>>>> to
>>>>>> "Python statements" generating XML. Then we could add loops by hand.
>>>>>> We
>>>>>> could also have CTF to XML, with hooks to merge Python generated
>>>>>> events.
>>>>>
>>>>> Indeed being able to script a trace would be extremely helpful and
>>>>> convert it either directly to CTF or to the intermediate format. Some
>>>>> scenarios for unit test would be to script a custom trace then change a
>>>>> few events for the test purpose, then either import it in TMF or
>>>>> convert
>>>>> it to CTF.
>>>>>
>>>>> Thanks,
>>>>> Geneviève
>>>>>>
>>>>>> In addition, TMF may also want to offer similar functionality, an XML
>>>>>> dump
>>>>>> of events and an XML events reader. Indeed, TMF supports a few formats
>>>>>> other than CTF.
>>>>>
>>>>>
>>>
>>> _______________________________________________
>>> lttng-dev mailing list
>>> lttng-dev at lists.lttng.org
>>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>
>>
>>
>

-- 
Jérémie Galarneau
EfficiOS Inc.
http://www.efficios.com