[ltt-dev] [RFC] Common Trace Format Requirements (v1.3)

[Mathieu Desnoyers]

* Frank Ch. Eigler (fche at redhat.com) wrote:
> Hi -
> 
> On Tue, Aug 31, 2010 at 12:37:22PM -0400, Mathieu Desnoyers wrote:
> 
> > [...]  Yep. But I'm trying to state the requirements as clearly as
> > possible so I can leave the justification out of the trace format
> > proposal per se.
> 
> Doing it this way is somewhat risky, since people who agree in general
> with the abstract requirements may change their minds once a specific
> design is presented.  So, expect the rationale to be a part of the
> debate once the proposed format is shown.

OK

> 
> 
> > I'm also only planning to do a proposal for a Linux-specific trace
> > format, which can be seen as a single instance of the trace format
> > model.
> 
> (How different do you expect a non-linux-specific "standard" trace
> format to be?)

E.g. something specialized for bare-metal, or for processor instruction tracing,
which might not share the same kind of constraints as a kernel-level tracing
facility.

In these cases, having timestamps might not be required, having the ordering
between events might not be required... basically, all the optional capabilities
might not be required by non-linux-specific trace formats. However, the
Linux-specific trace format will very likely impose some choices, e.g. having a
timestamps-based event ordering in the traces.

> 
> 
> > > If the metadata is not given in a standard form, then how do envision
> > > general trace analysis tools (those not hard-coded for some particular
> > > trace source) working?
> 
> > We can have a metadata format selector at the beginning of the
> > metadata section, with reserved IDs for metadata formats. We can
> > think of a format generated natively by TRACE_EVENT(), a format
> > generated in some sort of XML. The trace analyzer would need the
> > metadata format parser in order to be able to read the trace.
> 
> If one hopes that such tools should be able to consume data with
> drifting versions of TRACE_EVENT() or XML or whatnot, they themselves
> had better be fixed/standardized.  Otherwise, the data will not be
> self-describing, and all the tools will have to chase
> kernel/etc. versions.

The tool can bail out if it detects a type it does not know. So a tool upgrade
would be required.

Also the standard plans to have a major/minor trace version in there, so we can
explicitly break compatibility if it is required. It might end up being much
better than trying to support backward compatibility forever. Then it would be
up to the tool implementors to choose if they want to provide backward
compatibility for older trace versions or not.

> 
> 
> > > > * Requirements on the Tracers
> > > 
> > > > Higher-level tracer requirements that seem appropriate to support
> > > > some of the trace format requirements stated above.  [...]
> > > 
> > > The list that follows here appear to be wish-list performance
> > > characteristics of the tracing infrastructure ("make it go fast" and
> > > "use good transports").  How does it benefit the tracing *format*
> > > specification to enumerate such particulars here?
> 
> > Requirements like tracer speed influences the format in many ways, e.g.:
> 
> > - requirement for compactness leads to schemes where all information
> > repetition should be eliminated. Thus the need for optional
> > per-section context information.
> 
> > - requirement for speed and streaming leads to zero-copy
> > implementations, which imply that the trace format should be written
> > natively by the tracer.
> 
> Not exactly: the requirements of high speed influence the trace format
> to the extent that it must be *possible* to write it at a high speed.
> The format cannot mandate that it *necessarily* be written in a
> zero-copy way, or without intermediate transformations.

True. It's the same intent as for trace viewer requirements: the format should
be prepared to handle these requirements, but it does not make these
requirements mandatory on the tracers.

Thanks,

Mathieu




-- 
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20100831/7a7d60e9/attachment-0003.pgp>