[lttng-dev] CTF2-FS-1.0: Layout of a CTF 2 trace stored on a file system

[Philippe Proulx]

HTML version:

    http://diamon.org/ctf/files/CTF2-FS-1.0.html

Philippe Proulx
EfficiOS Inc.
http://www.efficios.com/

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

= CTF2-FS-1.0: Layout of a CTF{nbsp}2 trace stored on a file system
Philippe Proulx <pproulx at efficios.com>
v1.0, 21 October 2016
:toc:
:toclevels: 5
:docid: did:CTF2-FS-1.0

This document defines a standard layout of files for supporting a
CTF{nbsp}2 trace (see did:CTF2-SPEC-2.0) on a
https://en.wikipedia.org/wiki/File_system[file system] with file name
and directory support.

The purpose of this standard is to formalize the interoperability of
producers writing CTF{nbsp}2 traces on a file system with consumers
reading those traces (see <<compliance,Compliance>>).

The layout defined in this document is compatible with the layout of
http://lttng.org/[LTTng]{nbsp}2 traces on a file system, and with the layout
supported by the http://diamon.org/babeltrace/[Babeltrace] and
http://tracecompass.org/[Trace Compass] consumers.

.RFC 2119
NOTE: The key words _must_, _must not_, _required_, _shall_, _shall
not_, _should_, _should not_, _recommended_, _may_, and _optional_ in
this document, when emphasized, are to be interpreted as described in
https://www.ietf.org/rfc/rfc2119.txt[RFC 2119].


[[compliance]]
== Compliance

A CTF{nbsp}2 producer is said to _apply {docid}_ if it can produce a
CTF{nbsp}2 trace and store it on a file system as defined by this
document.

A CTF{nbsp}2 consumer is said to _support {docid}_ if it can read the
streams of a CTF{nbsp}2 trace stored on a file system as defined by
this document.


== Trace

The producer _must_ write the metadata stream and all the data stream
packets of a trace to regular files in a dedicated **directory**. In
the present text, this dedicated directory is called the _trace
directory_.

The producer _must not_ create regular files in the trace directory that
do not contain the metadata stream or data stream packets, unless the
names of those files start with a full stop character (`.`, U+002e).

The producer _may_ create subdirectories in the trace directory.

The consumer _must_ ignore subdirectories in the trace directory.

There are no constraints on the name of the trace directory.


=== Metadata stream

The metadata stream _must_ be serialized to a single regular file named
`metadata`. The file's content can be either raw, in which case it
contains the complete UTF-8 JSON metadata array, or it can be a
packetized metadata stream as described in did:CTF2-PMETA-1.0.


=== Data streams

If the data stream packet context field type of a given data stream
class does not contain a field tagged with the `data-stream-id` tag,
then each data stream instantiated by this data stream class _must_ be
written to exactly one regular file. There is no constraints on the name
of this file, except that it _must not_ be `metadata`.

If the data stream packet context field type of a given data stream
class does not contain a field tagged with the `data-stream-id` tag,
then the individual packets of the data streams referring to this data
stream class can be written to one or more files within the trace
directory, except to the files described in the previous paragraph.
There is no constraints on the name of those files, except that they
_must not_ be `metadata`.

Within any data stream file, the packets of a given data stream must
be written in the order they are produced.


== Examples

Here are a few examples of trace directories following the standard
layout defined by this document.

.Four data streams (no field tagged with `data-stream-id`)
========================================================================
    my-trace/
    ┣━ metadata       (contains the raw metadata array)
    ┣━ streamA        (contains all the packets of data stream 0)
    ┣━ streamB        (contains all the packets of data stream 1)
    ┣━ streamC        (contains all the packets of data stream 2)
    ┗━ streamD        (contains all the packets of data stream 3)
========================================================================

.One data stream (field tagged with `data-stream-id`)
========================================================================
    my-trace/
    ┣━ metadata       (contains the packetized metadata array)
    ┣━ streamA        (contains some packets of data stream 0)
    ┣━ streamB        (contains some packets of data stream 0)
    ┣━ streamC        (contains some packets of data stream 0)
    ┗━ streamD        (contains some packets of data stream 0)
========================================================================

.Four data streams (field tagged with `data-stream-id`)
========================================================================
    my-trace/
    ┣━ metadata       (contains the raw metadata array)
    ┣━ streamA0       (contains some packets of data stream 0)
    ┣━ streamA1       (contains some packets of data stream 0)
    ┣━ streamB0       (contains some packets of data stream 1)
    ┣━ streamB1       (contains some packets of data stream 1)
    ┣━ streamB2       (contains some packets of data stream 1)
    ┣━ streamC0       (contains some packets of data stream 2)
    ┣━ streamC1       (contains some packets of data stream 2)
    ┣━ streamD0       (contains some packets of data stream 3)
    ┣━ streamD1       (contains some packets of data stream 3)
    ┣━ streamD2       (contains some packets of data stream 3)
    ┗━ streamD3       (contains some packets of data stream 3)
========================================================================

.Two data streams with a field tagged with `data-stream-id`, two data streams without a field tagged with `data-stream-id`
========================================================================
    my-trace/
    ┣━ metadata       (contains the packetized metadata array)
    ┣━ streamAa       (contains some packets of data stream 0)
    ┣━ streamAb       (contains some packets of data stream 0)
    ┣━ streamBa       (contains some packets of data stream 1)
    ┣━ streamBb       (contains some packets of data stream 1)
    ┣━ streamC        (contains all the packets of data stream 2)
    ┗━ streamD        (contains all the packets of data stream 3)
========================================================================

.Two data streams with a field tagged with `data-stream-id`
========================================================================
    my-trace/
    ┣━ metadata       (contains the raw metadata array)
    ┣━ streamW        (contains some packets of data stream 0 and of data stream 1)
    ┣━ streamX        (contains some packets of data stream 0)
    ┣━ streamY        (contains some packets of data stream 1)
    ┗━ streamZ        (contains some packets of data stream 0 and of data stream 1)
========================================================================