[lttng-dev] Golang agent for LTTng-ust

[Jérémie Galarneau]

On 4 May 2018 at 06:03, Loïc Gelle <loic.gelle at polymtl.ca> wrote:

> Hi,
>
> There has been a previous discussion on the mailing list about porting
> LTTng to Golang, about a year ago: https://lists.lttng.org/
> pipermail/lttng-dev/2017-June/027203.html . This new topic is to discuss
> more precisely about implementation possibilities.
>
> Currently, one has to use the the C UST agent from LTTng in order to
> instrument Golang programs, and to compile the whole thing using custom
> Makefiles and cgo. Here is a recent example that I wrote:
> https://github.com/loicgelle/jaeger-go-lttng-instr
>
> As you can guess, there are a low of drawbacks in that approach. It is
> actually a hack and cannot be integrated into more complex Golang program
> that use a more complex build process (e.g. the Golang runtime itself),
> because of the compiler instructions that you have to include at the top of
> the Golang files. There is also a big concern about the performance of this
> solution, as calling a C function from Go requires to do a full context
> switch on the stack, because the calling conventions in C and Golang are
> different.
>

> I think a more integrated and performant solution is needed. We can’t
> really ignore a language such as Golang that is now widely adopted for
> cloud applications. LTTng is really the best solution out there in terms of
> overhead per tracepoint, and could benefit from being made available to
> such a large community. My question to the experts on this mailing list:
> how much would it take to write a Golang agent for LTTng?
>


Hi Loïc,

Without having performed any measurements myself, it does seem like calling
C from Go is very expensive. In that context, I can see that LTTng would
probably lose its performance advantage over any native Go solution.
However, it wouldn't hurt to measure the impact and see if it really is a
deal breaker.

We faced the same dilemma when implementing the Java and Python support in
lttng-ust. In those cases, we ended up calling C code, with the performance
penalties it implies. The correlation with other applications' and the
kernel's events, along with the rest of LTTng's features, provided enough
value to make that solution worthwhile.

There aren't a ton of solutions if we can't call existing C code. We
basically have to reimplement a ring-buffer and the setup/communication
infrastructure to interact with the lttng-sessiond. The communication with
the session daemon is not a big concern as the protocol is fairly
straightforward.

The "hairy" part is that lttng-ust and lttng-consumerd use a shared memory
map to produce and consume the tracing buffers. This means that all changes
to that memory layout would need to be replicated in the Go tracer, making
future evolution more difficult. Also, I don't know how easy it would be to
synchronize C and Go applications interacting in a shared memory map given
those languages have different memory models. My knowledge of Go doesn't go
that far.

A more viable solution could be to introduce a Go-native consumer daemon
implementing its own synchronization with Go applications. This way, that
implementation could evolve on its own and could also start with a simpler
ring buffer than lttng-ust's.

Still, it is not a small undertaking and it basically means maintaining a
third tracer implementation.


What do you think?

Thanks!
Jérémie


> Cheers,
> Loïc.
>
> _______________________________________________
> lttng-dev mailing list
> lttng-dev at lists.lttng.org
> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>
>


-- 
Jérémie Galarneau
EfficiOS Inc.
http://www.efficios.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.lttng.org/pipermail/lttng-dev/attachments/20180528/e0ab70f4/attachment.html>