[lttng-dev] CTF Compact Header Timestamps

[Mathieu Desnoyers]

Hi Brian, 

You should think about those 27 bits as the "new lowest significant bits" 
of the 64-bit clock. 

Quoting the CTF spec to understand how to deal with 27-bit overflow 
between two consecutive time-stamps in the same stream (here N = 27): 

"For a N-bit integer type referring to a clock, if the integer overflows compared to the N low order bits of the clock prior value found in the same stream, then it is assumed that one, and only one, overflow occurred. It is therefore important that events encoding time on a small number of bits happen frequently enough to detect when more than one N-bit overflow occurs." 

Those 27 bits are _not_ an offset from the previous timestamp. They update the 
current notion of time (based on all prior timestamps read for this stream) by 
setting the low 27 bits to the new value. If that value happens to be lower than the 
previous 27 low bits of the current time, this means we had one 27-bit overflow, so 
we need to increment the 28th bit by one. 

In each packet header, you will find the full 64-bit timestamps, so we always 
have a full 64-bit base from which to start. 

It is up to the tracer (lttng) to ensure that there is never more than a single 27-bit 
overflow preceding a 27-bit compact timestamp. If lttng detects that there would 
be more than a single 27-bit overflow prior to a timestamp, it will use an extended 
header which stores the full 64-bit of the new timestamp. 

We rely on this time-compaction scheme rather than offsetting from the prior 
64-bit timestamp because it allows us to weaken the sequential inter-dependency 
between consecutive events, which is useful for lock-free ring buffers. 

Let me know if you still have questions, 

Thanks, 

Mathieu 

----- On Mar 16, 2016, at 6:55 PM, Brian Robbins <brianrob at microsoft.com> wrote: 

> Hello,

> I’m forwarding this on-behalf of one of my colleagues who is trying to parse
> CTF. Any help you can provide is greatly appreciated.

> Thanks.

> -Brian

> From: Lee Culver
> Sent: Wednesday, March 16, 2016 11:06 AM
> To: Brian Robbins <brianrob at microsoft.com>
> Subject: CTF Compact Header Timestamps

> Event headers in LTTng can have a 27-bit clock or a 64-bit clock. I have been
> able to properly map the 64-bit clock back to a real timestamp, but I’m not
> sure how to interpret the 27-bit clock value. I’ve dug through the CTF
> specification but can’t find the relevant text. I have pasted the relevant
> metadata at the bottom of this email.

> For 64-bit clocks, this calculation gets us to the proper DateTime in C#:

> var clock = _metadata.Clocks.Single();

> var offset = new DateTime(1970, 1, 1, 0, 0, 0, 0,
> DateTimeKind.Utc).AddSeconds(clock.Offset / clock.Frequency);

> long ticks = (long)(header.Timestamp * 10000000.0 / clock.Frequency);

> var time = new DateTime(offset.Ticks + ticks, DateTimeKind.Utc);

> However, for the ‘compact’ variant (see below), we end up with a timestamp
> that’s far lower than the frequency, such as ‘0x55363b9’ and I’m simply not
> sure how to interpret this. It does NOT seem to be an offset from the previous
> event since if I treat it as such we get events that are out of order. One such
> example is the following:

> Event #11 - Extended header timestamp (64-bit): 0x00098a754555ea28

> Event #12 - Compact header timestamp (27-bit): 0x04ecc38c

> Event #13 - Extended header timestamp (64-bit): 0x00098a75369cc5e4

> As you can see, Event[11].Timestamp + Event[12].Timestamp > Event[13].Timestamp.
> Which means it cannot be a simple “offset from previous event”, since we would
> be getting events out of order.

> Here is the relevant metadata:

> clock {

> name = "monotonic";

> uuid = "9fb57333-ada2-4c73-a7ce-334f46437c74";

> description = "Monotonic Clock";

> freq = 1000000000; /* Frequency, in Hz */

> /* clock value offset from Epoch is: offset * (1/freq) */

> offset = 1451946230228988436;

> };

> typealias integer {

> size = 27; align = 1; signed = false;

> map = clock.monotonic.value;

> } := uint27_clock_monotonic_t;

> typealias integer {

> size = 64; align = 8; signed = false;

> map = clock.monotonic.value;

> } := uint64_clock_monotonic_t;

> struct event_header_compact {

> enum : uint5_t { compact = 0 ... 30, extended = 31 } id;

> variant <id> {

> struct {

> uint27_clock_monotonic_t timestamp;

> } compact;

> struct {

> uint32_t id;

> uint64_clock_monotonic_t timestamp;

> } extended;

> } v;

> } align(8);

> How should we be treating these uint27_clock_monotonic_t values?

> Thanks!

> -Lee Culver

> CLR Reliability and Performance

> _______________________________________________
> lttng-dev mailing list
> lttng-dev at lists.lttng.org
> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

-- 
Mathieu Desnoyers 
EfficiOS Inc. 
http://www.efficios.com 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.lttng.org/pipermail/lttng-dev/attachments/20160317/c47eac41/attachment.html>