[lttng-dev] Human read-writeable format for CTF traces

[Geneviève Bastien]

Hi Jeremie,

Thanks for your reply.

On 02/04/2014 11:19 AM, Jérémie Galarneau wrote:
> Hi Geneviève,
>
> I wonder if we are not overly complicating things here... The Python
> bindings seem to address your immediate testing concerns and will let
> you produce test traces with minimal effort.
Do you have any information, links, examples how to use what exists in 
python to generate traces with minimal effort? Our very immediate need 
is actually to generate traces manually, because we don't need to know 
how to script in python to do this ;-) and we can manually modify the 
events. But since I have no idea (yet) what those python bindings do and 
how they do it, then maybe I miss something here.

Thanks,
Geneviève
> While I agree that it doesn't solve the problem of testing analysis on
> Windows, I must ask if this really is a primary concern at this point.
>
> I would personally start by writing tests in Python using the
> CTF-Writer bindings and then, as the analysis feature gains traction,
> work with users to determine the best testing strategy. I somehow
> doubt that having an external dependancy on Babeltrace would really be
> a problem to these power users...
>
> Sorry for the late reply,
> Jérémie
>
> On Mon, Feb 3, 2014 at 12:20 PM, Geneviève Bastien
> <gbastien+lttng at versatic.net> wrote:
>> On 02/03/2014 12:16 PM, Mathieu Desnoyers wrote:
>>> ----- Original Message -----
>>>> From: "Geneviève Bastien" <gbastien+lttng at versatic.net>
>>>> To: "Michel Dagenais" <michel.dagenais at polymtl.ca>, "Mathieu Desnoyers"
>>>> <mathieu.desnoyers at efficios.com>
>>>> Cc: lttng-dev at lists.lttng.org
>>>> Sent: Monday, February 3, 2014 12:05:38 PM
>>>> Subject: Re: [lttng-dev] Human read-writeable format for CTF traces
>>>>
>>>> Ok, I'll wait for Jérémie's answer for more details. As I said, my
>>>> concern is to have something fully standalone in TMF. But if one has
>>>> access to babeltrace and eventual plugins to read-write a CTF trace to
>>>> XML, then all the better. We could then import an XML generated by a
>>>> python script into TMF, edit it there and then use it to test analyses.
>>>>
>>>> All we have to settle on is the intermediate format that should be used.
>>>> I'd go for XML because of the possibility to validate it and have visual
>>>> editors.
>>> Michel's idea of going for Python seems even better to generate test
>>> suites.
>>> It would allow importing and combining test "patterns" very easily, thus
>>> allowing to create tests by construction without having to copy-paste huge
>>> XML files.
>>>
>>> I don't clearly see why having external dependencies on other tools
>>> for a TMF CI test suite would be an issue. What would be the main
>>> arguments
>>> for having all those tests stand-alone in TMF for the test-suite ?
>> It is not just for test suite. XML-defined analysis will need test traces as
>> well, and that is in main TMF, not in unit tests (one idea of the XML
>> analysis is to allow end-user to develop their own analysis without writing
>> a single line of code or requiring the TMF development environment). And the
>> user of these analysis and test traces may not have access to babeltrace or
>> even to a Linux command line.
>>
>> Thanks,
>> Geneviève
>>
>>> Thanks,
>>>
>>> Mathieu
>>>
>>>> On 02/03/2014 11:00 AM, Michel Dagenais wrote:
>>>>>> I would expect that the ctf writer API recently added to babeltrace
>>>>>> (currently in master branch), along with the Python bindings that cover
>>>>>> trace read and write APIs, should allow you to implement things like:
>>>>>>
>>>>>> - A plugin to read a CTF trace, and output it in an intermediate format
>>>>>>      to facilitate edits (e.g. XML as you propose),
>>>>>> - A plugin to read this XML format and output a CTF trace.
>>>>> Yes, this would indeed be extremely helpful, in XML and/or JSON.
>>>>>
>>>>>> You could also generate the XML trace completely by hand if you like,
>>>>>> and
>>>>>> then convert it to CTF with the second plugin I'm relating to above.
>>>>> The likely scenario is to add a few events by hand.
>>>>>
>>>>>> Another possibility is that the XML description also allows
>>>>>> describing what the trace contains at a slightly higher level. For
>>>>>> instance, if you
>>>>>> have a periodic event happening for a certain amount of time, it would
>>>>>> be described in XML, and then "generated" by the XML-to-CTF
>>>>>> converter.
>>>>> Do we want to describe this in XML or in Python? We could have "CTF" to
>>>>> "Python statements" generating XML. Then we could add loops by hand. We
>>>>> could also have CTF to XML, with hooks to merge Python generated events.
>>>> Indeed being able to script a trace would be extremely helpful and
>>>> convert it either directly to CTF or to the intermediate format. Some
>>>> scenarios for unit test would be to script a custom trace then change a
>>>> few events for the test purpose, then either import it in TMF or convert
>>>> it to CTF.
>>>>
>>>> Thanks,
>>>> Geneviève
>>>>> In addition, TMF may also want to offer similar functionality, an XML
>>>>> dump
>>>>> of events and an XML events reader. Indeed, TMF supports a few formats
>>>>> other than CTF.
>>>>
>>
>> _______________________________________________
>> lttng-dev mailing list
>> lttng-dev at lists.lttng.org
>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>
>