[lttng-dev] Proposal: Make lttng-analyses part of lttng-tools

[Jérémie Galarneau]

On 18 July 2018 at 12:48, Geneviève Bastien <gbastien at versatic.net> wrote:
> On 2018-07-18 12:11 PM, Philippe Proulx wrote:
>> On Wed, Jul 18, 2018 at 11:37 AM Geneviève Bastien
>> <gbastien at versatic.net> wrote:
>>>> I'm eager to get some feedback from Mathieu and Jérémie on all this.
>>> So am I!
>> Adding mine even if you're not eager to get it ;-).
> I'm eager to hear anyone's opinion! And I think from the 2 opinions I
> got so far, analyses will not be part of tools. I'll take that as
> granted and consider all future comments of mine for lttng-analyses repo
> alone...
>>
>> As someone who knows a few parts of the LTTng analyses project, it is
>> still considered experimental, prototypal, in other words not ready at
>> all to be integrated within any stable project. It is with good reason
>> that its version is still 0.x.
>>
>> LTTng analyses helps us prove that such a tool can be useful (we
>> received good feedback in general), and it's a great project to quickly
>> develop a tracing analysis, but it's lacking thorough documentation
>> (outside its README file), tests, and by the nature of Python is very
>> slow and will probably remain like this forever. LTTng analyses also
>> does not handle exceptional scenarios very well.
>>
>> Also, do not forget that this project has hard dependencies on Python
>> and Babeltrace. What would this mean, on the packaging level, when
>> making it part of LTTng-tools, which has a very limited set of
>> dependencies on purpose?
> Interesting remark: You highlight that lttng-"analyses" package has 2
> purposes: trace collection and trace analysis.
>
> Trace collection can only be bash, does not require any other
> dependencies than lttng and that needs to be available on all machines
> to trace (with this mechanism)
>
> Trace analyses use babeltrace python bindings and needs to be run on the
> "analysis" machine, which may not even be one that is traced.
>
> 2 very different use cases... Should they be split in 2 packages? Could
> the trace collection part of analyses be moved to tools, leaving only
> the analyses in analyses? (oh I lied up there, I did mention merging
> with lttng-tools again ;-)

There is always the possibility of shipping session configurations along
with lttng-tools and/or scripts if the configuration needs to be "smart".

That's something that can be improved right now, we just need to
identify presets that make sense. Is that a good first step?

>>
>> I believe our mid/long-term goal is to:
>>
>> 1. Release Babeltrace 2 ASAP.
>> 2. Optimize Babeltrace 2 for the typical CTF input with stream muxing
>>    scenario.
>> 3. Create/transfer LTTng analyses as Babeltrace 2 filter or sink
>>    components.
>> 4. Encourage people to use Babeltrace 2 to analyse LTTng traces.
> Indeed our goals at Poly are different. But lttng being an open source
> project, I hope our goals can be reconciled for the good of the project :)
>
> Here are my short/mid-term goals:
>
> 1- Make trace collection with lttng for newcomers a painless process,
> easy to document and explain, to get fast results (whether with
> lttng-analyses or Trace Compass)
> 2- Develop and document methods to collect traces in an OpenStack/ceph
> cloud, in conjunction with monitoring tools.
>

I share Philippe's goals, but I also understand you want to improve the
situation in the short term.

Without integrating lttng-analyses in lttng-tools, I think Jonathan has
a good point in terms of marking lttng-analyses as a recommended
package or making it part of the 'lttng' metapackages (if applicable).

Packagers (Michael?) may want to comment here as I guess this
is a distro-specific issue.

> In the short term, I'm more interested in the trace collection part of
> lttng-analyses (or tools?), and that, in the [very!] short term, I'm
> ready to bring to 1.0.

I'm a lot more comfortable shipping this than the analysis part.

I don't mind shipping analysis tools as part of lttng-tools at some
point, but as Philippe mentioned, they would need tests,
documentation and maintenance on par with the rest of the project.

>>
>> That's not to say I disagree with the highlighted problem: our efforts
>> should be directed towards offering a streamlined experience to the
>> user, from instrumenting to tracing to analyzing.
>>
>> By the way, there are still a few professional, rigorous individuals who
>> read the documentation, thank God ;-).
> By the number of times I've been told to read the documentation, I'm
> definitely not one of them :p But yeah! Every time, the answer I was
> looking for _was_ in the doc! The person who wrote it really knew his
> stuff ;-)
>

There's a need for comprehensive documentation which, as a project,
I feel we have addressed. However, there's also a need for tutorials and
guides, which are lacking right now.

I'm glad you are choosing to invest time there.

Thanks,
Jérémie

> Geneviève
>
> _______________________________________________
> lttng-dev mailing list
> lttng-dev at lists.lttng.org
> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev



-- 
Jérémie Galarneau
EfficiOS Inc.
http://www.efficios.com