[lttng-dev] [RFC Patch Ust 2/5] Serialize the CTF global structures for ust-comm

[Thibault, Daniel]

----------------------------------------------------------------------
> Date: Fri, 2 May 2014 09:29:09 +0000 (UTC)
> From: Mathieu Desnoyers <mathieu.desnoyers at efficios.com>
> To: Geneviève Bastien <gbastien+lttng at versatic.net>

> Yes, all this is about not repeating global type descriptions over the Unix socket between applications and session daemon.
>
> The issue here is that with the current approach, we have to repeat possibly very long descriptions of structures, variants, and enums because they need to be bundled after every event description.
>
> Here is what I would envision instead:
>
> - We add a "global type" notifier to the UST notifiers (there is currently only "event" and "channel"). Those notifiers are sending the descriptions
>   of channels and events as seen from within the application to lttng-sessiond, so it can put it in its ust registry to later generate
>   the metadata.
> - Those notifiers can be used to send descriptions of enums, structs, and variants.
> - We'll probably want to have some way of verifying if the global type is exactly the same as global types with the same provider and
>   global type names registered previously, thus dealing with different provider .so versions, each containing different versions of the global
>   type. We'll have to find out what's the behaviour when this check does not succeed (failure or versioning of the global type within sessiond ?).
>   This verification will be needed once per global type, per application process. This verification needs to be done by sending the provider and
>   global type name, along with the global type details (to see if it matches).
> - Per global type, per application, we can keep a flag that knows whether the global type description has been successfully sent to the session
>   daemon for each session.

   I'd strongly advise to go the versioning way rather than the failure way (although failure is easier to implement initially).  The verification needs to be done more often than "once per global type, per application process": one use case is a process that dynamically loads (dlopen) an instrumented .so, generates events with it for a while, then unloads it (dlclose) and loads another .so that happens to include a provider with the same signature but different event payloads.  The process may or may not wholly unregister from and re-register with the session manager, depending on whether or not it has other providers loaded.

   See the closely related bugs #561, #698, #720.

   If the bandwidth used in sending lengthy type descriptions for mere verification is a concern, maybe the conversation between the app tracepoint provider and session manager could start by sending an MD5 or SHA hash of the type description.  Only when those mismatch would the full type description need to be sent.

   Also note that event type verification can be "lazy" (or "delayed") and occur only when the event type in question is actually enabled or emitted.  There isn't much point to updating the trace metadata with changing event types if the events in question are not enabled (at the channel or event level).  On the other hand, sending a filter specification to a provider pretty much requires event type verification, if only to warn the user that the filter he specified for the event won't be understood by the provider.

Daniel U. Thibault
Protection des systèmes et contremesures (PSC) | Systems Protection & Countermeasures (SPC)
Cyber sécurité pour les missions essentielles (CME) | Mission Critical Cyber Security (MCCS)
R & D pour la défense Canada - Valcartier (RDDC Valcartier) | Defence R&D Canada - Valcartier (DRDC Valcartier)
2459 route de la Bravoure
Québec QC  G3J 1X5
CANADA
Vox : (418) 844-4000 x4245
Fax : (418) 844-4538
NAC : 918V QSDJ <http://www.travelgis.com/map.asp?addr=918V%20QSDJ>
Gouvernement du Canada | Government of Canada
<http://www.valcartier.drdc-rddc.gc.ca/>