[lttng-dev] [LTTNG-TOOLS PATCH] On-disk multiple tracefiles circular buffer
Mathieu Desnoyers
mathieu.desnoyers at efficios.com
Tue May 7 11:42:26 EDT 2013
* Thibault, Daniel (Daniel.Thibault at drdc-rddc.gc.ca) wrote:
> -----Message d'origine-----
> From: Mathieu Desnoyers <mathieu.desnoyers at efficios.com>
>
> > I having similar questions as Daniel about tracefile-size and tracefile-count.
> >
> > For each stream, do we have max size used on disk = size * count or size used = size ?
>
> My understanding is that the maximum size on disk of a stream (channel) would be size*count*CPUs*processes, where processes is the number of user-space processes emitting events on that channel (this number can theoretically be quite large); for the kernel this number is 1.
Yes. For per-UID UST buffers the number is also close to 1 (1 per UID
per ABI).
>
> > If there is a doubt about the semantic, maybe using clearer terms that don't need to be explained might help. I guess the confusion here is:
> > what is a tracefile exactly ? The notion of "stream" is much clearer than "tracefile" in the CTF spec, so we might want to consider:
> >
> >--stream-chunks-num NUM: each stream, on disk, is divided in at most
> > NUM chunks (one file per chunk). When the maximum number of chunks is
> > reached, oldest chunks are deleted for this stream, thus creating
> > a flight recorder on disk.
> >
> >--stream-chunks-size SIZE: chunk size. Must be a multiple of
> > subbuf-size. Default value: subbuffer size.
>
> According to the CTF spec ("Event Stream: An ordered sequence of
> events, containing a subset of the trace event types."), an event
> stream is an LTTng channel.
Wrong. LTTng channel contains several streams (think of it like a class
of streams).
> This is clearly not the same thing as
> an on-disk file (and its corresponding file stream). This would
> mean:
>
> --stream-chunks-num NUM: each file stream (a channel, CPU ID, originating process ID triplet),
> on disk, is divided in at most NUM chunks (one file per chunk). When the maximum
> number of chunks is reached, the oldest chunks are overwritten by this stream, thus
> creating a circular flight recorder on disk.
>
> > Actually, you might want to consider that the stream-chunk-size need to be a multiple of the subbuffer size. The check needs to be more strict.
>
> Good idea. Not necessarily a power of two, though.
The UI has been fixed already, skipping the part above,
>
> ##########
>
> On a somewhat related topic, I find the CTF (1.8.2) spec confusing in sections 5 and 6:
>
> > 5. Event Packet Header
> >
> > The event packet header consists of two parts: the "event packet header"
> > is the same for all streams of a trace. The second part, the "event
> > packet context", is described on a per-stream basis. Both are described
> > in the TSDL meta-data. The packets are aligned on architecture-page-sized
> > addresses.
> [...]
> > 5.1 Event Packet Header Description
> [...]
> > 5.2 Event Packet Context Description
> [...]
> > 6. Event Structure
> >
> > The overall structure of an event is:
> >
> > 1 - Stream Packet Context (as specified by the stream meta-data)
> > 2 - Event Header (as specified by the stream meta-data)
> > 3 - Stream Event Context (as specified by the stream meta-data)
> > 4 - Event Context (as specified by the event meta-data)
> > 5 - Event Payload (as specified by the event meta-data)
> [...]
> > 6.1 Event Header
> [...]
> > 6.1.1 Type 1 - Few event IDs
> [...]
> > 6.1.2 Type 2 - Many event IDs
> [...]
> > 6.2 Event Context
> >
> > The event context contains information relative to the current event.
> > The choice and meaning of this information is specified by the TSDL
> > stream and event meta-data descriptions. The stream context is applied
> > to all events within the stream. The stream context structure follows
> > the event header. The event context is applied to specific events. Its
> > structure follows the stream context structure.
> [...]
> > 6.3 Event Payload
> >
> > An event payload contains fields specific to a given event type. The fields
> > belonging to an event type are described in the event-specific meta-data
> > within a structure type.
>
> The babeltrace output labels the following structured fields in
> each event record:
>
> stream.packet.context
> stream.event.header
> stream.event.context
> event.context
> event.fields
>
> They match section 6:
>
> > 1 - Stream Packet Context (as specified by the stream meta-data)
> > 2 - Event Header (as specified by the stream meta-data)
> > 3 - Stream Event Context (as specified by the stream meta-data)
> > 4 - Event Context (as specified by the event meta-data)
> > 5 - Event Payload (as specified by the event meta-data)
>
> But where does section 5 fit in? There is no "Event Packet Header"
> or "Event Packet Context" in the above. Section 3 mentions that
> "The event stream header will therefore be referred to as the
> 'event packet header' throughout the rest of this document."
> Judging by the C structs described, the lack of a Stream Packet
> Context struct in section 6, and the babeltrace output, I'm pretty
> sure that the "stream packet context" is the "event packet context"
> and that it also subsumes the "event packet header" (a.k.a. "event
> stream header"). But it would be nice if this were actually stated
> in the CTF spec.
Will reply to this part in a separate email.
>
> I would also like to know if LTTng can populate the "Event Context"
> (6.4) of a trace, or if this is something only other CTF producers
> may do. I haven't yet been able to produce any traces that have
> that field in them, and don't see how to do so.
Currently lttng does not produce event context.
Thanks,
Mathieu
>
> Daniel U. Thibault
> R & D pour la défense Canada - Valcartier (RDDC Valcartier) / Defence R&D Canada - Valcartier (DRDC Valcartier)
> Cyber sécurité pour les missions essentielles (CME) / Mission Critical Cyber Security (MCCS)
> Protection des systèmes et contremesures (PSC) / Systems Protection & Countermeasures (SPC)
> 2459 route de la Bravoure
> Québec, QC G3J 1X5
> CANADA
> Vox : (418) 844-4000 x4245
> Fax : (418) 844-4538
> NAC : 918V QSDJ <http://www.travelgis.com/map.asp?addr=918V%20QSDJ>
> Gouvernement du Canada / Government of Canada
> <http://www.valcartier.drdc-rddc.gc.ca/>
>
> _______________________________________________
> lttng-dev mailing list
> lttng-dev at lists.lttng.org
> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
More information about the lttng-dev
mailing list