[ltt-dev] LTTng kernel integration roadmap, update

[Mathieu Desnoyers]

(sorry for repost, I messed up lkml address in the original post, please
reply to this one)

Hi everyone,

Let's see what the current status is and what we still have to focus on
before submission. First I'd like to say a big thanks to Zhaolei from
Fujitsu who did a great job for the debugfs-based LTTng control we now
have.

- trace_clock()
  - Port to the -tip tree. Submit. Should be ready for inclusion after
    the few rounds of review on LKML it had.
  Assigned-to: Mathieu Desnoyers <mathieu.desnoyers at polymtl.ca>

- LTTng instrumentation
  - Port to the -tip tree, submit incrementally.
  Assigned-to: Mathieu Desnoyers <mathieu.desnoyers at polymtl.ca>

- Add periodical subbuffer flush for streaming
  - Will be required by ascii text output.
  - Also requires modification to lttd so it can only transmit the
    buffer data, without the padding. Some modification to LTTV might be
    needed to cope with variable-sized subbuffers (we would need an
    index to be able to seek with O(log(num subbuffers)) as we currently
    do, or we would have to use sparse files for trace files).
  Assigned-to: François Prenoveau <francois.prenoveau at polymtl.ca>

- Marker ID management
  Background :
  The markers are currently used to identify the numerical event IDs
  associated with a named event as well as the event types (format
  string) associated with that event. Those two informations are kept in
  a table written in the metadata channel. A "channel" could be thought
  as being a set of per-cpu buffers, where information transits. This
  information would typically be related to a specific tracer. Note,
  however, that the information generated by a tracer should always be
  parseable generically and be transformable into text output by a
  simple parser, because we want to permit analysis across information
  logged by the various tracer.

  Action item :
  The current event IDs are global to the whole kernel. It would be
  beneficial to change their scope so they are per-channel instead,
  because we can then encode most events in the low-order bits of the
  event ID bits, therefore keeping events as small as possible.

- Channel management
  Background :
  LTTng currently has the following statically declared channels :

  cpu (default channel)
  metadata (marker name, marker ID, marker format string table)
  interrupts (interrupt handlers table, interrupt handler connexion
  events)
  modules (loaded module table, module load/free events)
  network (active interfaces table, interface up/down events)
  processes (processes existing at trace start table, scheduling events)

  Action item :
  But if we want to integrate ftrace easily within this framework, we
  would like to offer an API so tracers can dynamically register a new
  channel whenever they are loaded.

  Note that the marker ID management and channel management are closely
  tied together; IDs would be per-channel, and channels would be
  registered by tracers.

- trimmed-down lttv for the kernel tree
  Need to look at
  git://git.kernel.org/pub/scm/linux/kernel/git/sam/test.git#master
  tests/ directory, which permits building userspace tools with the
  Linux kernel. I recently got the idea of populating debugfs with
  userspace tools that would sits in kernel module data. How (in)sane
  does this idea look ? That would seems like a rather good solution to
  ship the userspace tool with the kernel.

- Port ftrace to LTTng
  ftrace could ask LTTng to create channels for its data, and would use
  this handle to record into the buffers. Given that LTTng allows
  multiple traces to be taken at once, and given LTTng exports all its
  trace control primitives as in-kernel API, ftrace could use those to
  automatically control tracing behavior from within the kernel wrt the
  buffers for which it produces _and_ consumes information.

  I think we might need to add the notion of traces for which buffers
  are meant to be consumed within the kernel (private) and not exported
  to userspace. Or maybe the current reference counting which insure
  that at most one reader is connected on a trace buffer, is enough.

- ASCII text output.
  - Merge sort done within the kernel. Binary-to-text translation also
    done within the kernel.
  Assigned-to: Zhaolei <zhaolei at cn.fujitsu.com>
  Assigned-to: Lai Jiangshan <laijs at cn.fujitsu.com>

Mathieu

-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68