[lttng-dev] RFC LTTng session and daemon configuration save and restore

Wed Dec 4 16:33:09 EST 2013

RFC - LTTng session and daemon configuration save and restore

Author: Jérémie Galarneau <jeremie.galarneau at efficios.com>

Version:
- v0.2: 04/12/2013
* Reorganized the text following David Goulet's recommendations
- v0.1: 03/12/2013
* Initial proposal

Introduction
-------------

This proposal details the file formats, LTTng APIs and command line interfaces
used to save and restore session and daemon configurations.

Daemon Configuration File Format
--------------------------------

The LTTng session and relay daemons shall be configurable using INI
configuration files. The set of configuration options accepted by LTTng daemons
corresponds to the command line options available for each daemon.

The sections in the files must match the daemons' names: [sessiond] and
[relayd]. The entries under each section must match the long option names
supported by the daemons. Options specified on the command line shall always
have precedence over options set by the configuration files.

An example demonstrating the use of such configuration
options follows.

[sessiond]
consumerd32-path=/lib/lttng/libexec/lttng-consumerd
consumerd64-path=/lib64/lttng/libexec/lttng-consumerd
no-kernel=1

[relayd]
data-port=tcp://0.0.0.0:5343
output=/tmp/MyTraces

Daemon configuration files are, by default, either stored system-wide under
/etc/lttng/*.conf or in the user's home directory under $HOME/.lttng/*.conf.

New options may be added to the daemon configuration file as features are
added to the relay and session daemons. The daemons shall ignore unrecognized
options to ensure forward compatibility.

LTTng Command Line - Daemon Configuration
-----------------------------------------

On launch, the session, relay and consumer daemons will search for files with
the .conf extension in the following folders, in order of precedence:
1) Any path specified using a new --load-config option
2) $HOME/.lttng/*.conf
3) /etc/lttng/*.conf

Session Configuration File Format
---------------------------------

The LTTng session configuration file format must enable the serialization of the
session, channel and event structures of the LTTng API. Since these concepts
are eminently hierarchical, it only makes sense to use a file format that can
express such relationships.

While an INI based file format was the first considered, for its ease of use
and human readability, it appears that INI provides no standard way of
expressing hierarchies. It is, of course, possible to flatten session/channel
hierarchies using prefixed section names, but this is both error prone and
unsupported by most INI file reading libraries. This means that we would have
to code our own INI handling code and produce a specification to document our
additions to the INI format. INI also provides no standard versioning mechanism.

In keeping with the desire to use a human-readable format, we have considered
rolling our own format. This format could express hierarchy using tabulations
to delimit scopes, not unlike the Python language. Such a format would be both
relatively easy to parse, but also readable. However, this both requires
documenting the format in a specification, rolling our own parsing and
validation code, thus losing the benefit of reusing external libraries. This
option has the highest maintenance cost considering the amount of code and
documentation to be produced while presenting marginal benefits over the INI
approach.

Finally, it seems that using a standard hierarchical format such as JSON or
XML would be the most appropriate choice. Both of these formats have intrinsic
support for representation of hierarchical relationships. They also benefit from
having a lot of hardened external libraries that will provide parsing,
validation and write support. This is a huge advantage from both
maintainability and interoperability standpoints. However, both formats
present the disadvantage of being harder, although minimally, to edit manually.
It remains to be seen if manual editing of session configurations really is an
important use-case. Using lttng's save feature would probably prove more
convenient than editing the files manually. Furthermore, the addition of a
"dry-run" option to the lttng client would significantly alleviate this pain
point.

XML seems like a better option than JSON since it makes it possible to have
robust file validation using a pre-defined schema. The use of version-specific
schemas could also be beneficial for backward compatibility as the format
moves forward. Finally, character encoding handling is already a solved
problem for most XML libraries.

LTTng ABI/API
-------------

Two new functions are added to the lttng.h API.

int lttng_load_sessions(const char *url, const char *session_name,
int flags/struct lttng_session_load_attr *attr);

Load sessions. If url the is a directory, all .lttng files it contains will be
loaded. If session_name is NULL, all sessions found in url are restored.
If a session_name is provided, only this session will be restored.

A supplementary argument, either a "flags" argument or an attribute structure,
is used to indicate whether or not the sessions currently known to the
session daemon should be replaced by the ones found in the configuration
file(s), provided that their names are the same.
Even though this is the only current use-case for this argument, a structure
with a reasonable amount of padding may be more suitable than a primary
type to accommodate new features. Thoughts?

int lttng_save_sessions(const char *url, const char *session_name
int flags/struct lttng_session_save_attr *attr);

Save sessions. If url is a directory, the session configuration will be saved
as session_name.lttng. If a complete file name is provided, the session(s) to be
saved will be written to this file. If session_name is NULL, all sessions
currently known to the session daemon will be saved. If a name is provided, only
that session will be saved.

The reasoning for the flags/attr argument is the same as for the
lttng_load_sessions() function, but for a configuration file overwrite
option.

LTTng Command Line - Session Configuration
------------------------------------------

Tracing session configurations can be saved and restored using the lttng command
line client. This capability introduces two new command line options.

-- Load session configurations --

$ lttng load -s SESSION_NAME [OPTIONS]

Loads tracing session configurations.

.lttng files will be searched in the following default paths, in order of
precedence:
1) $(HOME)/.lttng/sessions
2) /etc/lttng/sessions

A session name or the -a option must be passed as the SESSION_NAME argument.
Like some other lttng commands, such as enable-event, the -a option stands for
"all". If this option is used, every session found in the paths will be loaded.

If a session was saved as active, the tracing for that session will be activated
automatically on load. The current session configuration of the session daemon
is also preserved when a configuration is loaded. The new session configurations
are added to the current session set.

Tracing sessions saved in an active state will be resumed on load.

--input-path, -i
Path in which to search for SESSION_NAME's description. If the path is a
directory, all .lttng files it contains will be opened and searched for
SESSION_NAME. If an input path is provided, the -a option will load all
session configurations found in this path. The default paths are ignored when
this option is used.

--force, -f
If a restored session's name conflicts with a currently existing session, the
original session configuration is preserved. However, if the --force option is
used, the restored session takes precedence. The original session will be
destroyed.

-- Save session configurations --

$ lttng save -s SESSION_NAME [OPTIONS]

Saves tracing configurations.

The selected tracing session's configuration will be saved in the
$HOME/.lttng/sessions folder. If the -a option is used, every session known
to the session daemon will be saved.

--output-path, -o
If the provided output path is a directory, the session will be saved in a
file named "SESSION_NAME.lttng". However, if the path is a file, the session(s)
will all be saved in the same file.

-- 
Jérémie Galarneau
EfficiOS Inc.
http://www.efficios.com