[lttng-dev] [RFC] Dynamic instrumentation support in UST

[Zifei Tong]

Thank you for your comments.

>   When using process IDs, one also has to be careful of PID namespaces: for a member of the 'tracing' group, a PID issued from a local lttng command (e.g. '$ lttng enable-event --pid ...') may need to be translated by the time it reaches the root lttng-sessiond.  Similar concerns exist with user namespaces.  The implications of namespaces are not peculiar to this extension of lttng, however: I would expect lttng to provide namespace-handling utility functions or to otherwise handle namespaces consistently throughout its services.

If lttng could implement a set of general namespace-handling utility
functions, we could use the facility to address the PID namespaces
problem in dynamic instrumentation naturally.

>    The -c command option by itself may be insufficient in some cases.  For instance, if the lttng command is issued from an su shell (e.g. '# lttng trace -c ...'), from a 'tracing' group member (e.g. '$ lttng trace -c ...'), or from some ordinary account (e.g. '$ sudo -H lttng trace -c ...'), the intent may be to specify which user account to run the COMMAND as.

The `lttng trace` command is a long-standing not-implemented-yet
feature #15 [1]. We may need a separated RFC for that.

> An 'lttng trace' command would be problematic anyway since it may need to specify several events to enable, each assigned to a different channel and possibly with a filter as well.  The overall command line could become very unwieldy.

I am thinking we could have a script to specify all these stuff
(context, channel, filter), like Systemtap [2], gdb [3], or extension
based on lttng-gen-tp.

>   I would allow the above command line to specify a process name instead of a process ID, in order to capture multi-threaded and multi-instance processes.
> If enable-event is able to set up latent events (as described in my previous comment block), then 'lttng trace' is unnecessary: the user need only set up his events, channels, etc., using --processname and then issue a proper 'sudo' command to launch the process (sudo has all the requisite options to assign the process to the
proper user account with the proper environment settings, etc.).

If we use the command line interface you described, like

      lttng enable-event --processname a.out

Then we will need a mechanism to distinguish processes with the same
name. For example, I want to instrument my program for debugging but
not touch any processes in production.

>   If it follows the lttng kernel implementation, --function sets up a kretprobe and --probe sets up a kprobe, so in user-space this should work similarly: --probe sets up a function entry event while --function sets up a function entry AND a function return event.

I've updated the RFC to following lttng kernel function probe behavior.

[1]: http://bugs.lttng.org/issues/15
[2]: http://sourceware.org/systemtap/SystemTap_Beginners_Guide/scripts.html
[3]: http://sourceware.org/gdb/onlinedocs/gdb/Tracepoint-Actions.html#Tracepoint-Actions

Thanks,
Zifei Tong