[lttng-dev] LTTng 2.13.0 - Nordicité - Linux kernel and user-space tracer

Tue Aug 3 14:40:21 EDT 2021

Hi everyone,

Today is the official release of LTTng 2.13 - Nordicité! It is the result of
one year of development from most of the EfficiOS team.

The most notable features of this new release are:

  - Event-rule matches condition triggers and new actions, allowing internal
    actions or external monitoring applications to quickly react when kernel
    or user-space instrumentation is hit,
  - Notification payload capture, allowing external monitoring applications
    to read elements of the instrumentation payload when instrumentation is
    hit.
  - Instrumentation API: vtracef and vtracelog (LTTng-UST),
  - User space time namespace context (LTTng-UST and LTTng-modules).

This release is named after "Nordicité", the product of a collaboration between
Champ Libre and Boréale. This farmhouse IPA is brewed with Kveik yeast and
Québec-grown barley, oats and juniper branches. The result is a remarkable
fruity hazy golden IPA that offers a balanced touch of resinous and woodsy
bitterness.

Based on the LTTng project's documented stable releases lifetime, this 2.13
release coincides with the end-of-life (EOL) of the LTTng 2.11 release series.

Read on for a short description of each of the new features and the
links to this release.

A prettified version of this announcement will be available soon on GitHub:
https://github.com/lttng/lttng-tools/releases/tag/v2.13.0

Note on LTTng-UST backward compatibility
---

- soname major version change
  This release changes the LTTng-UST soname major from 0 to 1.

  The event notifier (triggers using an event-rule-matches condition)
  functionality required a significant rework of public data structures which
  should never have been made public in the first place.

  Bumping the soname major to 1, will require applications and tracepoint
  providers to be rebuilt against an updated LTTng-UST to use it.

  Old applications and tracepoint providers linked against libraries with
  major soname 0 should be able to co-exist on the same system.

- Building probe providers using a C++ compiler requires C++11

- API namespaceing
  The LTTng-UST API is now systematically namespaced under `lttng_ust_*` (e.g
  `tracepoint()` becomes `lttng_ust_tracepoint()`).

  However, the non-namespaced names are still exposed to maintain API
  compatibility.

Event-rule matches condition and new actions
---

Expanding the trigger infrastructure and making it usable through the `lttng`
client was the core focus of this release.

A trigger is an association between a condition and one or more actions. When
the condition associated to a trigger is met, the actions associated to that
trigger are executed. The tracing does not have to be active for the conditions
to be met, and triggers are independent from tracing sessions.

Since their introduction as part of LTTng 2.10, new conditions and actions were
added to make this little-known mechanism more flexible.

For instance, before this release, triggers supported the following condition
types:
  - Buffer usage exceeded a given threshold,
  - Buffer usage went under a configurable threshold,
  - A session rotation occurred,
  - A session rotation completed.

A _notify_ action could be used to send a notification to a third party
applications whenever those conditions were met.

This made it possible, for instance, to disable certain event rules if the
tracing buffers were almost full. It could also be used to wait for session
rotations to be completed to start processing the resulting trace chunk
archives as part of various post-processing trace analyses.

This release introduces a new powerful condition type: event-rule matches.

This type of condition is met when the tracer encounters an event matching the
given even rule. The arguments describing the event rule are the same as those
describing the event rules of the `enable-event` command.

While this is not intended as a general replacement for the existing
high-throughput tracing facilities, this makes it possible for an application
to wait for a very-specific event to occur and take action whenever it occurs.
The purpose of event-rule matches triggers is to react quickly to an event
without the delay introduced by buffering.

For example, the following command will create a trigger that emits a
notification whenever the 'openat' system call is invoked with the
'/etc/passwd' filename argument.

$ lttng add-trigger
    --condition event-rule-matches
      --type=kernel:syscall
      --name='openat'
    --action notify

New actions were also introduced as part of this release:
  - Start session
    This action causes the LTTng session daemon to start tracing for the session
    with the given name. If no session with the given name exist at the time the
    condition is met, nothing is done.

  - Stop session
    This action causes the LTTng session daemon to stop tracing for the session
    with the given name. If no session with the given name exist at the time the
    condition is met, nothing is done.

  - Rotate session
    This action causes the LTTng session daemon to rotate the session with the
    given name. See lttng-rotate(1) for more information about the session
    rotation concept. If no session with the given name exist at the time the
    condition is met, nothing is done.

  - Snapshot session
    This action causes the LTTng session daemon to take a snapshot of the
    session with the given name. See lttng-snapshot(1) for more information
    about the session snapshot concept. If no session with the given name exist
    at the time the condition is met, nothing is done.

These new actions can also be combined together. For instance, the following
trigger will stop `my_session`, record a snapshot of `my_session`, and notify
any listening application when '/etc/passwd' is opened:

$ lttng add-trigger
    --condition event-rule-matches
      --type kernel:syscall
      --name 'openat'
      --filter 'filename == "/etc/passwd"'
    --action stop-session my_session
    --action snapshot-session my_session
    --action notify

For more information, see the following manual pages:
  - lttng-add-trigger(1),
  - lttng-remove-trigger(1),
  - lttng-list-triggers(1).

Notification payload capture
---

The new event-rule matches condition type also allows 'captures'.
This allow event record and context fields to be captured when an event-rule
matches condition is satisfied.

The captured field values are made available in the evaluation object of the
notifications transmitted to listening applications.

Capture descriptors can be specified using a syntax reminiscent of the one used
by the filter expressions.

The following example will capture a process's name and the 'filename' argument
of all `openat()` system calls:

$ lttng add-trigger
    --condition event-rule-matches
      --type kernel:syscall
      --name 'openat'
      --capture 'filename'
      --capture '$ctx.procname'
    --action notify

See the lttng-add-trigger(1) manual page for more information.

vtracef and vtracelog (LTTng-UST)
---

New versions of the `tracef()` and `tracelog()` tracing helpers accepting
variable argument lists are introduced as `vtracef()` and `vtracelog()`.

See the tracef(3) and tracelog(3) manual pages for more information.

Add time namespace context (LTTng-UST and LTTng-modules)
---

It is now possible to add the time namespace of a process as a context to
channels (`time_ns`) using the `add-context` command.

See the time_namespaces(7) manual page for more information.

Links
---

Project website: https://lttng.org

Download links:
https://lttng.org/files/lttng-tools/lttng-tools-2.13.0.tar.bz2
https://lttng.org/files/lttng-ust/lttng-ust-2.13.0.tar.bz2
https://lttng.org/files/lttng-modules/lttng-modules-2.13.0.tar.bz2

GPG signatures:
https://lttng.org/files/lttng-tools/lttng-tools-2.13.0.tar.bz2.asc
https://lttng.org/files/lttng-ust/lttng-ust-2.13.0.tar.bz2.asc
https://lttng.org/files/lttng-modules/lttng-modules-2.13.0.tar.bz2.asc

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com