[ltt-dev] [RFC] Common Trace Format Requirements (v1.3)

[Mathieu Desnoyers]

* Frank Ch. Eigler (fche at redhat.com) wrote:
> Hi -
> 
> On Tue, Aug 31, 2010 at 10:50:30AM -0400, Mathieu Desnoyers wrote:
> 
> > [...]  The goal of the present document is to propose a trace format
> > that will suit the needs [...]. It starts by doing an overview of
> > the trace format, tracer and trace analyzer requirements to consider
> > for a Common Trace Format proposal.
> 
> Was it your intent to limit this document to a listing of abstract
> requirements, as opposed to proposing an actual file format?

Yep. But I'm trying to state the requirements as clearly as possible so I can
leave the justification out of the trace format proposal per se.

I'm also only planning to do a proposal for a Linux-specific trace format, which
can be seen as a single instance of the trace format model.

> 
> 
> > 1) Architecture
> 
> > This high-level model is meant to be an industry-wide, common model,
> > fulfilling the tracing requirements. It is meant to be application-,
> > architecture-, and language-agnostic.
> 
> > [...]
> > - Metadata [...]
> >   - Metadata description language not imposed by standard
> 
> If the metadata is not given in a standard form, then how do envision
> general trace analysis tools (those not hard-coded for some particular
> trace source) working?

We can have a metadata format selector at the beginning of the metadata
section, with reserved IDs for metadata formats. We can think of a format
generated natively by TRACE_EVENT(), a format generated in some sort of XML. The
trace analyzer would need the metadata format parser in order to be able to read
the trace.

> 
> 
> > * Requirements on the Tracers
> 
> > Higher-level tracer requirements that seem appropriate to support
> > some of the trace format requirements stated above.  [...]
> 
> The list that follows here appear to be wish-list performance
> characteristics of the tracing infrastructure ("make it go fast" and
> "use good transports").  How does it benefit the tracing *format*
> specification to enumerate such particulars here?

Requirements like tracer speed influences the format in many ways, e.g.:

- requirement for compactness leads to schemes where all information repetition
  should be eliminated. Thus the need for optional per-section context
  information.
- requirement for speed and streaming leads to zero-copy implementations, which
  imply that the trace format should be written natively by the tracer.

> 
> 
> > * Trace Analyzer Requirements
> > [...]
> 
> Such requirements are specific to some particular tracing task.  It
> would not make sense to identify these items as normative.  For
> example, a trace analyzer tool that can consume the standard format
> should not be deemed to violate the standard, if it merely can't grok
> 10GB files.

I don't think this is the spirit in which I stated this. The idea here is to
ensure that the trace format structure makes it *possible* for a trace analyzer
to cope with huge traces, not to require that all trace analyzers do so.

I'll update the RFC document according to your comments,

Thanks!

Mathieu

> 
> 
> - FChE



-- 
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20100831/4adf69d9/attachment-0003.pgp>