[lttng-dev] reading context fields causes syscalls

[Mathieu Desnoyers]

----- On May 20, 2021, at 8:46 AM, Norbert Lange nolange79 at gmail.com wrote:

> Am Mi., 19. Mai 2021 um 20:52 Uhr schrieb Mathieu Desnoyers
> <mathieu.desnoyers at efficios.com>:
>>
>> ----- On May 19, 2021, at 8:11 AM, lttng-dev lttng-dev at lists.lttng.org wrote:
>>
>> > Hello,
>> >
>> > Several context fields will cause a syscall atleast the first time a
>> > tracepoint is
>> > recorded. For example all of the following:
>> >
>> > `lttng add-context -c chan --userspace --type=vpid --type=vtid --type=procname`
>> >
>> > Each of them seems cached in TLS however, and most should never change
>> > after startup.
>> >
>> > As I am using Lttng over Xenomai, syscalls are strictly forbidden, I
>> > would like to have some function that prepares all data, which I can
>> > call on each thread before it switches to realtime work.
>> >
>> > Kinda similar to urcu_bp_register_thread, I'd like to have some
>> > `lttng_ust_warmup_thread` function that fetches the context values
>> > that can be cached. (urcu_bp_register_thread should be called there
>> > aswell)
>> > I considered just doing a tracepoint, but AFAIK the channel can be
>> > changed/configured after the process is running. So this is not robust
>> > enough.
>>
>> The new lttng_ust_init_thread() API in lttng-ust 2.13 would be the right
>> place to do this I think:
>>
>> /*
>>  * Initialize this thread's LTTng-UST data structures. There is
>>  * typically no need to call this, because LTTng-UST initializes its
>>  * per-thread data structures lazily, but it should be called explicitly
>>  * upon creation of each thread before signal handlers nesting over
>>  * those threads use LTTng-UST tracepoints.
>>  */
>>
>> It would make sense that this new initialization helper also initializes
>> all contexts which cache the result of a system call. Considering that
>> contexts can be used from the filter and capture bytecode interpreter, as
>> well as contexts added to channels, I think we'd need to simply initialize
>> them all.
> 
> Yeah, just figured that it doesnt help at all if I do a tracepoint, as
> it might just be disabled ;)
> lttng_ust_init_thread() sounds right for that, maybe add one or 2 arguments for
> stuff you want initialized / dont want initialized over the default.
> 
> I take that the downside of eager initialization is potentially wasted
> resources (now ignoring any one-time runtime cost).

I would not want to introduce too much coupling between the application and
the tracer though. The public API I've added for the 2.13 release cycle takes
no argument, and I'm not considering changing that at this stage (we are already
at -rc2, so we're past the API freeze).

I'd be open to adding an extra API with a different purpose though. Currently
lttng_ust_init_thread is meant to initialize per-thread data structures for
tracing signal handlers.

Your use-case is different: you aim at tracing from a context which cannot
issue system calls. Basically, any attempt to issue a system call from that
thread after this is a no-go. I would be tempted to introduce something like
"lttng_ust_{set,clear}_thread_no_syscall" or such, which would have the following
effects when set:

* Force immediate initialization of all thread's cached context information,
* Set a TLS variable flag indicating that the tracer should not do any system
  call whatsoever. The tracer could either use dummy data (zeroes), log an error,
  or abort() the process if a thread in no_syscall mode attempts to issue a system
  call. This could be dynamically selected by a new environment variable.
* Prevent threads in no_syscall mode from calling the write() system call on
  sub-buffer switch (of course the read-timer channel option is preferred).

Thoughts ?

Thanks,

Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com