[lttng-dev] User-space tracing by the root lttng-sessiond

[Thibault, Daniel]

   I ran a little experiment whose results surprised me.  Using a non-tracing group user account, I first made sure I had a root lttng-sessiond running and a user-space lttng-sessiond.  In the first I set up a trace for some kernel events plus all user-space events.  In the second I set up a trace for all user-space events only.  Both traces were started, then the lttng-ust/doc/examples/easy-ust/sample demo was run to generate the user-space events.  Then both traces were stopped and destroyed.

   I noted there were three lttng-consumerd daemons: one in user-space, the other two owned by root.  Presumably one is servicing the kernel events while the other is servicing the user-space events.  (Question: If there were several users active at once, would there be several root user-space lttng-consumerd daemons or still just one?)

   As expected, both traces show the sample user-space events (with no losses, it turns out).  What surprised me, however, is that the pairs of events were not in synch. Since they're all coming out of the same trace provider, I would expect the time-stamp to be issued once (by the tracepoint provider), and then two copies of the event to go into the two trace buffers for eventual storage.  Could it be that the tracepoint provider instead issues two separate events, one for each of its clients?  (For the record, the pairs of events were out of synch by at least 4187 ns (670 times out of 100 000), and as much as 3 326 405 ns (once); in 93% of the time root event lags after the user-space event)

   This seems problematic to me, since multiple traces capturing the same event won't be able to reconcile their copies with each other.

Daniel U. Thibault
Protection des systèmes et contremesures (PSC) | Systems Protection & Countermeasures (SPC)
Cyber sécurité pour les missions essentielles (CME) | Mission Critical Cyber Security (MCCS)
R & D pour la défense Canada - Valcartier (RDDC Valcartier) | Defence R&D Canada - Valcartier (DRDC Valcartier)
2459 route de la Bravoure
Québec QC  G3J 1X5
CANADA
Vox : (418) 844-4000 x4245
Fax : (418) 844-4538
NAC : 918V QSDJ <http://www.travelgis.com/map.asp?addr=918V%20QSDJ>
Gouvernement du Canada | Government of Canada
<http://www.valcartier.drdc-rddc.gc.ca/>