[lttng-dev] Overwrite mode, lost events, and reader behaviour

[Mathieu Desnoyers]

----- Original Message -----
> From: "Daniel Thibault" <Daniel.Thibault at drdc-rddc.gc.ca>
> To: lttng-dev at lists.lttng.org
> Sent: Monday, November 18, 2013 9:43:06 AM
> Subject: [lttng-dev] Overwrite mode, lost events, and reader behaviour
> 
> I'd like a clarification.  In overwrite mode, and assuming the writers are
> overwhelming the consumers, what is the behaviour?
> 
>    Assume we have this situation within a given channel (numbering the
>    sub-buffers in chronological order), and the writers are about to catch
>    up with the reader (the extra sub-buffer is double-bracketed and the
>    sub-buffer it is "shadowing" is marked with a trailing asterisk):
> 
> [8] [9] [2] [3] [4] [5] [6] [7] [[-]]   reader at [2], writers at [9], spare
> sub-buffer unused/unallocated
> 
> [8] [9] [10]* [3] [4] [5] [6] [7] [[2]]   reader at [[2]], writers at [10],
> spare sub-buffer shadowing [10]
> 
> [8] [9] [10]* [11] [4] [5] [6] [7] [[2]]   reader at [[2]], writers at [11],
> spare sub-buffer shadowing [10]
> 
>    Now, when the reader completes evacuating [[2]], it moves on to [11] and
>    realises there is a time gap.  I suspect it does not just jump its clock
>    ahead, because that would give up on [4] through [10].  Does it skip to
>    the next sub-buffer until it sees the clock go backward (the circularity
>    of the buffer guarantees there is always one such discontinuity),
>    resuming its work with [4]?

We don't use the timestamps for this. Each packet has a sequence number given by
the producer, and the reader is querying for packets by requesting sequence
numbers (increasing by one each time). In the case you present here, the reader
will try to query the sequence number of an overwritten packet, and will therefore
not be able to get it. It will move on to the next packet until it finally finds
the first packet which sequence number matches its current sequence number (or the
writer position).

Mathieu


>  If yes, in the worst case the reader will go
>    once around the sub-buffers, checking timestamps, before resuming its
>    work.  This could be avoided if the buffer metadata kept the spare
>    sub-buffer chained to the oldest one (i.e. the third step is [8] [9] [10]
>    [11]* [4] [5] [6] [7] [[2]]), so the reader never needs to call "next()"
>    more than once.  Is this what lttng already does?
> 
> Daniel U. Thibault
> Protection des systèmes et contremesures (PSC) | Systems Protection &
> Countermeasures (SPC)
> Cyber sécurité pour les missions essentielles (CME) | Mission Critical Cyber
> Security (MCCS)
> R & D pour la défense Canada - Valcartier (RDDC Valcartier) | Defence R&D
> Canada - Valcartier (DRDC Valcartier)
> 2459 route de la Bravoure
> Québec QC  G3J 1X5
> CANADA
> Vox : (418) 844-4000 x4245
> Fax : (418) 844-4538
> NAC : 918V QSDJ <http://www.travelgis.com/map.asp?addr=918V%20QSDJ>
> Gouvernement du Canada | Government of Canada
> <http://www.valcartier.drdc-rddc.gc.ca/>
> 
> _______________________________________________
> lttng-dev mailing list
> lttng-dev at lists.lttng.org
> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> 

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com