[lttng-dev] RFC LTTng session and daemon configuration save and restore

[David Goulet]

On 05 Dec (12:12:33), Julien Desfossez wrote:
> Hi,
> 
> [...]
> 
> > 
> > LTTng Command Line - Daemon Configuration
> > -----------------------------------------
> > 
> > On launch, the session, relay and consumer daemons will search for files with
> > the .conf extension in the following folders, in order of precedence:
> > 1) Any path specified using a new --load-config option
> Why "load" ?
> Why not --config (like openvpn for example) ?
> -c or -f would be also good for short options.

Agree, --config is better and more common.

> 
> > 2) $HOME/.lttng/*.conf
> > 3) /etc/lttng/*.conf
> > 
> > 
> > Session Configuration File Format
> > ---------------------------------
> > 
> > The LTTng session configuration file format must enable the serialization of the
> > session, channel and event structures of the LTTng API. Since these concepts
> > are eminently hierarchical, it only makes sense to use a file format that can
> > express such relationships.
> > 
> > While an INI based file format was the first considered, for its ease of use
> > and human readability, it appears that INI provides no standard way of
> > expressing hierarchies. It is, of course, possible to flatten session/channel
> > hierarchies using prefixed section names, but this is both error prone and
> > unsupported by most INI file reading libraries. This means that we would have
> > to code our own INI handling code and produce a specification to document our
> > additions to the INI format. INI also provides no standard versioning mechanism.
> > 
> > In keeping with the desire to use a human-readable format, we have considered
> > rolling our own format. This format could express hierarchy using tabulations
> > to delimit scopes, not unlike the Python language. Such a format would be both
> > relatively easy to parse, but also readable. However, this both requires
> > documenting the format in a specification, rolling our own parsing and
> > validation code, thus losing the benefit of reusing external libraries. This
> > option has the highest maintenance cost considering the amount of code and
> > documentation to be produced while presenting marginal benefits over the INI
> > approach.
> Also, we will have fun with debugging the tab vs space issues.
> 	
> > 
> > Finally, it seems that using a standard hierarchical format such as JSON or
> > XML would be the most appropriate choice. Both of these formats have intrinsic
> > support for representation of hierarchical relationships. They also benefit from
> > having a lot of hardened external libraries that will provide parsing,
> > validation and write support. This is a huge advantage from both
> > maintainability and interoperability standpoints. However, both formats
> > present the disadvantage of being harder, although minimally, to edit manually.
> > It remains to be seen if manual editing of session configurations really is an
> > important use-case. Using lttng's save feature would probably prove more
> > convenient than editing the files manually. Furthermore, the addition of a
> > "dry-run" option to the lttng client would significantly alleviate this pain
> > point.
> > 
> > XML seems like a better option than JSON since it makes it possible to have
> > robust file validation using a pre-defined schema. The use of version-specific
> > schemas could also be beneficial for backward compatibility as the format
> > moves forward. Finally, character encoding handling is already a solved
> > problem for most XML libraries.
> > 
> > 
> > LTTng ABI/API
> > -------------
> > 
> > Two new functions are added to the lttng.h API.
> > 
> > int lttng_load_sessions(const char *url, const char *session_name,
> > int flags/struct lttng_session_load_attr *attr);
> > 
> > Load sessions. If url the is a directory, all .lttng files it contains will be
> > loaded. If session_name is NULL, all sessions found in url are restored.
> > If a session_name is provided, only this session will be restored.
> > 
> > A supplementary argument, either a "flags" argument or an attribute structure,
> > is used to indicate whether or not the sessions currently known to the
> > session daemon should be replaced by the ones found in the configuration
> > file(s), provided that their names are the same.
> > Even though this is the only current use-case for this argument, a structure
> > with a reasonable amount of padding may be more suitable than a primary
> > type to accommodate new features. Thoughts?
> Would it be possible instead, to have an enum and a len field, to avoid
> padding issues ?

Depending if that call needs a payload or not at some point in time for
a specific option, I would go for either using a "int optname + void
*data" or an opaque data structure with getter/setter.

There is one use case that I have in mind where a user could want to
receive the configuration of a specific session inline meaning in a
given string for instance.

> 
> > 
> > 
> > int lttng_save_sessions(const char *url, const char *session_name
> > int flags/struct lttng_session_save_attr *attr);
> > 
> > Save sessions. If url is a directory, the session configuration will be saved
> > as session_name.lttng. If a complete file name is provided, the session(s) to be
> > saved will be written to this file. If session_name is NULL, all sessions
> > currently known to the session daemon will be saved. If a name is provided, only
> > that session will be saved.
> > 
> > The reasoning for the flags/attr argument is the same as for the
> > lttng_load_sessions() function, but for a configuration file overwrite
> > option.
> > 
> In save and load, can the URL be a network URL (like the net:// we have) ?
> I can easily see a use-case for that in the future, where we would have
> a repository of session configurations. Especially in cloud/large-scale
> deployments.
> I don't think we should rush into implementing that, but having the
> interface easy to extend would be interesting.

This is exactly why there is a "url" there. :)

> 
> > 
> > LTTng Command Line - Session Configuration
> > ------------------------------------------
> > 
> > Tracing session configurations can be saved and restored using the lttng command
> > line client. This capability introduces two new command line options.
> > 
> > 
> > -- Load session configurations --
> > 
> > $ lttng load -s SESSION_NAME [OPTIONS]
> > 
> > Loads tracing session configurations.
> > 
> > .lttng files will be searched in the following default paths, in order of
> > precedence:
> > 1) $(HOME)/.lttng/sessions
> > 2) /etc/lttng/sessions
> > 
> > A session name or the -a option must be passed as the SESSION_NAME argument.
> > Like some other lttng commands, such as enable-event, the -a option stands for
> > "all". If this option is used, every session found in the paths will be loaded.
> > 
> > If a session was saved as active, the tracing for that session will be activated
> > automatically on load. The current session configuration of the session daemon
> > is also preserved when a configuration is loaded. The new session configurations
> > are added to the current session set.
> > 
> > Tracing sessions saved in an active state will be resumed on load.
> > 
> > --input-path, -i
> > Path in which to search for SESSION_NAME's description. If the path is a
> > directory, all .lttng files it contains will be opened and searched for
> > SESSION_NAME. If an input path is provided, the -a option will load all
> > session configurations found in this path. The default paths are ignored when
> > this option is used.
> Can we have multiple paths separated by ":" (like $PATH) ?
> Also, if we have a configuration path already specified in the sessiond
> configuration file, will we override or merge the search paths with -i ?

Hrm... seems a bit intense, maybe we can allow multiple -i ?

> 
> > 
> > --force, -f
> > If a restored session's name conflicts with a currently existing session, the
> > original session configuration is preserved. However, if the --force option is
> > used, the restored session takes precedence. The original session will be
> > destroyed.
> > 
> > 
> > -- Save session configurations --
> > 
> > $ lttng save -s SESSION_NAME [OPTIONS]
> > 
> > Saves tracing configurations.
> > 
> > The selected tracing session's configuration will be saved in the
> > $HOME/.lttng/sessions folder. If the -a option is used, every session known
> > to the session daemon will be saved.
> I wonder if we will have trouble with saving only the sessions of our
> user (if a root sessiond is running). I think -a will save everything
> regardless of the user. Not sure if it is a problem, but we could maybe
> specify a uid as well if we want to save all the sessions of a
> particular user ?

The credentials of the API call are passed to the session daemon so
those only will be saved. Thus if you are in the tracing group, you can
talk to the root session daemon but will only be able to save YOUR
session.

This is a good question, do we want to allow anyone in the tracing group
to save *every* session on the root one. I'm not to sure about that.

Cheers!
David

> 
> > 
> > --output-path, -o
> > If the provided output path is a directory, the session will be saved in a
> > file named "SESSION_NAME.lttng". However, if the path is a file, the session(s)
> > will all be saved in the same file.
> > 
> > 
> 
> It looks good !
> 
> I can't wait to have that for lttngtop ;-) !
> 
> Thanks,
> 
> Julien
> 
> _______________________________________________
> lttng-dev mailing list
> lttng-dev at lists.lttng.org
> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 620 bytes
Desc: Digital signature
URL: <http://lists.lttng.org/pipermail/lttng-dev/attachments/20131205/46f5897a/attachment-0001.pgp>