[lttng-dev] Human read-writeable format for CTF traces

[Geneviève Bastien]

On 02/03/2014 12:16 PM, Mathieu Desnoyers wrote:
> ----- Original Message -----
>> From: "Geneviève Bastien" <gbastien+lttng at versatic.net>
>> To: "Michel Dagenais" <michel.dagenais at polymtl.ca>, "Mathieu Desnoyers" <mathieu.desnoyers at efficios.com>
>> Cc: lttng-dev at lists.lttng.org
>> Sent: Monday, February 3, 2014 12:05:38 PM
>> Subject: Re: [lttng-dev] Human read-writeable format for CTF traces
>>
>> Ok, I'll wait for Jérémie's answer for more details. As I said, my
>> concern is to have something fully standalone in TMF. But if one has
>> access to babeltrace and eventual plugins to read-write a CTF trace to
>> XML, then all the better. We could then import an XML generated by a
>> python script into TMF, edit it there and then use it to test analyses.
>>
>> All we have to settle on is the intermediate format that should be used.
>> I'd go for XML because of the possibility to validate it and have visual
>> editors.
> Michel's idea of going for Python seems even better to generate test suites.
> It would allow importing and combining test "patterns" very easily, thus
> allowing to create tests by construction without having to copy-paste huge
> XML files.
>
> I don't clearly see why having external dependencies on other tools
> for a TMF CI test suite would be an issue. What would be the main arguments
> for having all those tests stand-alone in TMF for the test-suite ?
It is not just for test suite. XML-defined analysis will need test 
traces as well, and that is in main TMF, not in unit tests (one idea of 
the XML analysis is to allow end-user to develop their own analysis 
without writing a single line of code or requiring the TMF development 
environment). And the user of these analysis and test traces may not 
have access to babeltrace or even to a Linux command line.

Thanks,
Geneviève
>
> Thanks,
>
> Mathieu
>
>>
>> On 02/03/2014 11:00 AM, Michel Dagenais wrote:
>>>> I would expect that the ctf writer API recently added to babeltrace
>>>> (currently in master branch), along with the Python bindings that cover
>>>> trace read and write APIs, should allow you to implement things like:
>>>>
>>>> - A plugin to read a CTF trace, and output it in an intermediate format
>>>>     to facilitate edits (e.g. XML as you propose),
>>>> - A plugin to read this XML format and output a CTF trace.
>>> Yes, this would indeed be extremely helpful, in XML and/or JSON.
>>>
>>>> You could also generate the XML trace completely by hand if you like, and
>>>> then convert it to CTF with the second plugin I'm relating to above.
>>> The likely scenario is to add a few events by hand.
>>>
>>>> Another possibility is that the XML description also allows
>>>> describing what the trace contains at a slightly higher level. For
>>>> instance, if you
>>>> have a periodic event happening for a certain amount of time, it would
>>>> be described in XML, and then "generated" by the XML-to-CTF
>>>> converter.
>>> Do we want to describe this in XML or in Python? We could have "CTF" to
>>> "Python statements" generating XML. Then we could add loops by hand. We
>>> could also have CTF to XML, with hooks to merge Python generated events.
>> Indeed being able to script a trace would be extremely helpful and
>> convert it either directly to CTF or to the intermediate format. Some
>> scenarios for unit test would be to script a custom trace then change a
>> few events for the test purpose, then either import it in TMF or convert
>> it to CTF.
>>
>> Thanks,
>> Geneviève
>>> In addition, TMF may also want to offer similar functionality, an XML dump
>>> of events and an XML events reader. Indeed, TMF supports a few formats
>>> other than CTF.
>>