[lttng-dev] URCU_CALL_RCU_RT and its implications

[Mark E. Dawson, Jr.]

Yes, this is QSBR flavor. 
Regarding the context to your last question, I'm curious about how an updater in QSBR mode using call_rcu *without properly registering itself as a reader* could ever work properly. In our case, this was the initial mistake we noticed that we were doing, yet we only encountered the aforementioned startup error a relatively small percentage of the time (i.e., the reclamation thread only performing a cleanup once).

      From: Mathieu Desnoyers <mathieu.desnoyers at efficios.com>
 To: "Mark E. Dawson, Jr." <medawsonjr at yahoo.com> 
Cc: lttng-dev <lttng-dev at lists.lttng.org>
 Sent: Thursday, December 8, 2016 3:46 PM
 Subject: Re: [lttng-dev] URCU_CALL_RCU_RT and its implications
   
----- On Dec 8, 2016, at 2:53 PM, lttng-dev <lttng-dev at lists.lttng.org> wrote:

All,
In the documentation for the create_call_rcu_data() call, we have the following description of the URCU_CALL_RCU_RT flag:

struct call_rcu_data *create_call_rcu_data(unsigned long flags, int cpu_affinity): Returns a handle that can be passed to the other functions in this list. The flags argument can be zero, or can be URCU_CALL_RCU_RT if the application threads associated with the new callback-invocation thread are to get real-time response from call_rcu() by avoiding the need to call into the kernel to wake up the callback-invocation thread.

So setting "flags" to 0 will simply use the futex mechanism for wakeups. What exactly happens when set to the RT option? Is there something else that needs to be configured from the OS side to make the aforementioned option work correctly?

No, it's just that the call rcu worker thread will rely on periodic polling
of the call rcu work queue.



I'm asking because we're seeing intermittent issues whereby on application startup, the reclamation thread (call_rcu) does exactly one cleanup, but then never does it again.  We have the URCU_CALL_RCU_RT flag set.  Originally, we thought it was because the updater was not appropriately registered as a reader. But after ensuring that is the case, it still happens in 10% of startups. 
Which URCU flavor are you using ? You may have a thread stuck as
being a RCU reader forever, which would cause this kind of issue.

For instance, unbalanced rcu read-side lock/unlock, or blocking on
poll() or other while holding a rcu read-side lock, or blocking on
pthread_join() while holding rcu read-side lock...

And if you use the QSBR flavor, remember that registered threads
are by default in read-side. So you need to mark them as explicitly
"offline" whenever you do any blocking op (e.g. poll(), blocking read(),
pthread_join....).




Lastly, we wonder why it would even work correctly *at all* when we failed to register the updater as a reader thread.
I don't understand the context of this question. Can you provide more info ?

Thanks,

Mathieu





_______________________________________________
lttng-dev mailing list
lttng-dev at lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev


-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.lttng.org/pipermail/lttng-dev/attachments/20161208/a3c20ef1/attachment.html>