[lttng-dev] porting LTTng to Go(lang)?

Sat Jun 3 02:24:37 UTC 2017

hi all,

the suggestion to write a pre-processor for LTTng tracepoints is
interesting and may be a good idea.  however, i'm not sure that it will
provide a general purpose solution.

the problem i for see is that tracepoint invocations, at least, should be
written in the syntax of the language they're being used in (a programmer
would would expect the tracepoint() function call to look like any other
function call in the language).

so the tracepoint parser needs to understand the function invocation syntax
of each language.  it also needs to understand at least some of the rest of
the syntax of the language -- in the case of Go it at least needs to be
able recognize "import" statements and import those files and it needs to
understand when something that looks like a tracepoint() is actually
embedded in a comment or string.  it also needs to emit code that matches
the syntax of the language.

so a general purpose lttng tracepoint parsing tool would need at least a
limited understanding of the syntax and grammar of each language for
reading the source file in and then writing the source file out.  there
could be code in the middle that's common across languages, but probably
not a lot.  perhaps an lttng tracepoint parsing tool could be better
structured as a common framework for language specific tracepoint parsers
to be plugged in to.

but writing a parser that actually understands the grammar of a language is
not easy.  if i was really going to do this, i'd be tempted to see if i
could just take this from gcc (which has front-ends that supports C, C++,
Go, and other languages (i think)).

and then there's the tracepoint definitions.  they could be written using
the barectf() tool that Philippe mentions, but i'm not sure if that would
be satisfactory for most programmers, who would prefer to write tracepoint
definitions in the same language that they are programming in.  they would
also need to support the data types of the language (i'm thinking Python
here as an example, where tracepoints would be expected to support most of
the Python basic data types).

because Go seems to be similar to C i'm strongly tempted to just co-opt the
C pre-processor and existing LTTng tracepoint machinery and modify the
LTTng header files to generate valid Go code for a tracepoint definition.
this does add a LTTng tracepoint tool to the steps required to compile Go,
except that the tracepoint tool is called "cpp" (plus header files).

i don't really know Go yet and i hae not been assigned to get LTTng to work
in Go, so this is all quite preliminary.  but i think the project i'm about
to start on really needs LTTng or LTTng-like tracepoint functionality so
i'm motivated to see if this can work.

i'm going to investigate some more.

--craig

On Fri, Jun 2, 2017 at 10:05 AM, Philippe Proulx <eeppeliteloop at gmail.com>
wrote:

> On Fri, Jun 2, 2017 at 11:58 AM, Mathieu Desnoyers <
> mathieu.desnoyers at efficios.com> wrote:
>
>>
>> ----- On Jun 2, 2017, at 2:27 AM, craig harmer <charmster at gmail.com>
>> wrote:
>>
>> hi all,
>>
>> has anyone looked at porting LTTng to Go
>> <https://en.wikipedia.org/wiki/Go_%28programming_language%29> (AKA Golang
>> <https://golang.org/>)?
>>
>>
>> Not yet ! I'm glad someone is looking into it. :)
>>
>>
>>
>> Go is a language very similar to C, but it has intrinsic support for
>> concurrency and better memory protection (at the expense of including
>> garbage collection).  its from Google and is pretty hot these days for big
>> data and "web scale" distributed systems.  the similarity to C is not
>> surprising since two of the three authors were also involved in the
>> development of UNIX in at AT&T in the 1970's: Rob Pike and Ken Thompson
>> (Robert Griesemer is the third author).
>>
>> Go is able to link with C object files, so implementing support for
>> tracef(3) should not be very difficult.
>>
>> but i want to have full support for user-defined tracepoint() events.
>> that looks to be much trickier since the LTTng tracepoint() functionality
>> relies heavily on (some might say abuses
>>
>> "abuses" is appropriate ;-)
>
>> ) the C pre-processor, and Go does not have an equivalent of the C
>> pre-processor.  however, i don't see an obvious reason why Go source files
>> that contain tracepoints couldn't be run through the C pre-processor --
>> except that Go specific LTTng header files would be required.
>>
>> here is an example of "hello, world" in Go
>> <https://tour.golang.org/welcome/1>.  (note the use of "import" rather
>> then "#include").
>>
>> i've spent more than a few hours looking at the C code generated by the C
>> pre-processor trying to track down exactly why a tracepoint() definition is
>> causing a compiler error, so i think i have an idea of just how hard it
>> would be to development equivalent header files for Go + C pre-processor.
>> it may be the case that the Go "header files", once developed, would be
>> compiler specific, i.e only work with gccgo
>> <https://golang.org/doc/install/gccgo> (the gcc front-end for Go).
>>
>> anyway, i'm wondering if anybody has attempted this and/or what your
>> thoughts would be.
>>
>> In the case of C/C++ code, using the C preprocessor got the job done
>> without requiring any additional
>> dependency. Given the context you describe, perhaps it would be wise to
>> consider introducing a dedicated
>> "lttng probe description" parser. The idea here would be to parse the
>> tracepoint probe definitions and
>> translate those into native code for various languages (e.g. Golang).
>>
>> This would create a new lttng utility that would be required to to
>> produce the language-specific files from
>> those descriptions. I would allow us to port the "tracepoint" concept to
>> many more languages easily, and
>> not be so much tied to the C preprocessor anymore.
>>
>
>> So perhaps adding this new tool as a dependency that needs to be invoked
>> prior to compilation might
>> not be too much of an issue ? I see it as being slightly similar to the
>> role accomplished by Flex and
>> Bison: they are required to translate from source files to an
>> intermediary language, and then the
>> resulting files can be included into the distribution source packages, so
>> only those who aim at
>> changing the source descriptions need to have the translation tool
>> installed.
>>
>
>> Thoughts ?
>>
>
> Good point and we thought about this before. In fact this is just what
> barectf <http://barectf.org/> does, for example. What I see in the future
> is that a subset of (eventual) CTF 2's metadata language (JSON) could be
> used here (or an equivalent, human-friendly YAML, like barectf) to describe
> LTTng event and stream classes and translate them to what's needed to
> record such events for a given programming language. CTF 2's user
> attributes can be used to insert additional, language-specific properties,
> like parameter types and names, and other options.
>
> Phil
>
>>
>> Thanks,
>>
>> Mathieu
>>
>>
>>
>>
>>
>> --craig
>>
>> _______________________________________________
>> lttng-dev mailing list
>> lttng-dev at lists.lttng.org
>> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>
>>
>> --
>> Mathieu Desnoyers
>> EfficiOS Inc.
>> http://www.efficios.com
>>
>> _______________________________________________
>> lttng-dev mailing list
>> lttng-dev at lists.lttng.org
>> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.lttng.org/pipermail/lttng-dev/attachments/20170602/b1f5f3fe/attachment-0001.html>