[lttng-dev] Human read-writeable format for CTF traces

[Jérémie Galarneau]

Hi Geneviève,

I wonder if we are not overly complicating things here... The Python
bindings seem to address your immediate testing concerns and will let
you produce test traces with minimal effort.
While I agree that it doesn't solve the problem of testing analysis on
Windows, I must ask if this really is a primary concern at this point.

I would personally start by writing tests in Python using the
CTF-Writer bindings and then, as the analysis feature gains traction,
work with users to determine the best testing strategy. I somehow
doubt that having an external dependancy on Babeltrace would really be
a problem to these power users...

Sorry for the late reply,
Jérémie

On Mon, Feb 3, 2014 at 12:20 PM, Geneviève Bastien
<gbastien+lttng at versatic.net> wrote:
> On 02/03/2014 12:16 PM, Mathieu Desnoyers wrote:
>>
>> ----- Original Message -----
>>>
>>> From: "Geneviève Bastien" <gbastien+lttng at versatic.net>
>>> To: "Michel Dagenais" <michel.dagenais at polymtl.ca>, "Mathieu Desnoyers"
>>> <mathieu.desnoyers at efficios.com>
>>> Cc: lttng-dev at lists.lttng.org
>>> Sent: Monday, February 3, 2014 12:05:38 PM
>>> Subject: Re: [lttng-dev] Human read-writeable format for CTF traces
>>>
>>> Ok, I'll wait for Jérémie's answer for more details. As I said, my
>>> concern is to have something fully standalone in TMF. But if one has
>>> access to babeltrace and eventual plugins to read-write a CTF trace to
>>> XML, then all the better. We could then import an XML generated by a
>>> python script into TMF, edit it there and then use it to test analyses.
>>>
>>> All we have to settle on is the intermediate format that should be used.
>>> I'd go for XML because of the possibility to validate it and have visual
>>> editors.
>>
>> Michel's idea of going for Python seems even better to generate test
>> suites.
>> It would allow importing and combining test "patterns" very easily, thus
>> allowing to create tests by construction without having to copy-paste huge
>> XML files.
>>
>> I don't clearly see why having external dependencies on other tools
>> for a TMF CI test suite would be an issue. What would be the main
>> arguments
>> for having all those tests stand-alone in TMF for the test-suite ?
>
> It is not just for test suite. XML-defined analysis will need test traces as
> well, and that is in main TMF, not in unit tests (one idea of the XML
> analysis is to allow end-user to develop their own analysis without writing
> a single line of code or requiring the TMF development environment). And the
> user of these analysis and test traces may not have access to babeltrace or
> even to a Linux command line.
>
> Thanks,
> Geneviève
>
>>
>> Thanks,
>>
>> Mathieu
>>
>>>
>>> On 02/03/2014 11:00 AM, Michel Dagenais wrote:
>>>>>
>>>>> I would expect that the ctf writer API recently added to babeltrace
>>>>> (currently in master branch), along with the Python bindings that cover
>>>>> trace read and write APIs, should allow you to implement things like:
>>>>>
>>>>> - A plugin to read a CTF trace, and output it in an intermediate format
>>>>>     to facilitate edits (e.g. XML as you propose),
>>>>> - A plugin to read this XML format and output a CTF trace.
>>>>
>>>> Yes, this would indeed be extremely helpful, in XML and/or JSON.
>>>>
>>>>> You could also generate the XML trace completely by hand if you like,
>>>>> and
>>>>> then convert it to CTF with the second plugin I'm relating to above.
>>>>
>>>> The likely scenario is to add a few events by hand.
>>>>
>>>>> Another possibility is that the XML description also allows
>>>>> describing what the trace contains at a slightly higher level. For
>>>>> instance, if you
>>>>> have a periodic event happening for a certain amount of time, it would
>>>>> be described in XML, and then "generated" by the XML-to-CTF
>>>>> converter.
>>>>
>>>> Do we want to describe this in XML or in Python? We could have "CTF" to
>>>> "Python statements" generating XML. Then we could add loops by hand. We
>>>> could also have CTF to XML, with hooks to merge Python generated events.
>>>
>>> Indeed being able to script a trace would be extremely helpful and
>>> convert it either directly to CTF or to the intermediate format. Some
>>> scenarios for unit test would be to script a custom trace then change a
>>> few events for the test purpose, then either import it in TMF or convert
>>> it to CTF.
>>>
>>> Thanks,
>>> Geneviève
>>>>
>>>> In addition, TMF may also want to offer similar functionality, an XML
>>>> dump
>>>> of events and an XML events reader. Indeed, TMF supports a few formats
>>>> other than CTF.
>>>
>>>
>
>
> _______________________________________________
> lttng-dev mailing list
> lttng-dev at lists.lttng.org
> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev



-- 
Jérémie Galarneau
EfficiOS Inc.
http://www.efficios.com