<html><head></head><body><div style="color:#000; background-color:#fff; font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:16px"><div id="yui_3_16_0_ym19_1_1481240908655_4439"><span id="yui_3_16_0_ym19_1_1481240908655_4438">Hmmm, it just dawned on me that since in QSBR mode the updater itself is a registered reader, then it, too, much use the rcu_thread_offline/online() functions around its network I/O system calls. Doh!</span></div><div class="qtdSeparateBR" id="yui_3_16_0_ym19_1_1481240908655_4452"><br><br></div><div class="yahoo_quoted" id="yui_3_16_0_ym19_1_1481240908655_4467" style="display: block;"> <div style="font-family: HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif; font-size: 16px;" id="yui_3_16_0_ym19_1_1481240908655_4466"> <div style="font-family: HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif; font-size: 16px;" id="yui_3_16_0_ym19_1_1481240908655_4465"> <div dir="ltr" id="yui_3_16_0_ym19_1_1481240908655_4464"> <font size="2" face="Arial" id="yui_3_16_0_ym19_1_1481240908655_4463"> <hr size="1" id="yui_3_16_0_ym19_1_1481240908655_4462"> <b><span style="font-weight:bold;">From:</span></b> Mathieu Desnoyers <mathieu.desnoyers@efficios.com><br> <b><span style="font-weight: bold;">To:</span></b> "Mark E. Dawson, Jr." <medawsonjr@yahoo.com> <br><b><span style="font-weight: bold;">Cc:</span></b> lttng-dev <lttng-dev@lists.lttng.org><br> <b><span style="font-weight: bold;">Sent:</span></b> Thursday, December 8, 2016 3:46 PM<br> <b><span style="font-weight: bold;">Subject:</span></b> Re: [lttng-dev] URCU_CALL_RCU_RT and its implications<br> </font> </div> <div class="y_msg_container" id="yui_3_16_0_ym19_1_1481240908655_4468"><br><div id="yiv8687194669"><div id="yui_3_16_0_ym19_1_1481240908655_4471"><div style="font-family:arial, helvetica, sans-serif;font-size:12pt;color:#000000;" id="yui_3_16_0_ym19_1_1481240908655_4470"><div id="yui_3_16_0_ym19_1_1481240908655_4469"><span id="yiv8687194669zwchr">----- On Dec 8, 2016, at 2:53 PM, lttng-dev <lttng-dev@lists.lttng.org> wrote:<br clear="none"></span></div><div id="yui_3_16_0_ym19_1_1481240908655_4474"><blockquote style="border-left:2px solid #1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica, Arial, sans-serif;font-size:12pt;" id="yui_3_16_0_ym19_1_1481240908655_4473"><div style="color:#000;background-color:#fff;font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:16px;" id="yui_3_16_0_ym19_1_1481240908655_4472"><div id="yiv8687194669yui_3_16_0_ym19_1_1481226442633_2947">All,</div><div id="yiv8687194669yui_3_16_0_ym19_1_1481226442633_2947"><br clear="none"></div><div id="yiv8687194669yui_3_16_0_ym19_1_1481226442633_2947">In the documentation for the create_call_rcu_data() call, we have the following description of the URCU_CALL_RCU_RT flag:</div><div id="yiv8687194669yui_3_16_0_ym19_1_1481226442633_2947"><br clear="none"></div><blockquote id="yiv8687194669yui_3_16_0_ym19_1_1481226442633_3052" style="margin:0 0 0 40px;border:none;padding:0px;"><div dir="ltr" id="yiv8687194669yui_3_16_0_ym19_1_1481226442633_2947"><code id="yiv8687194669yui_3_16_0_ym19_1_1481226442633_2989">struct call_rcu_data *create_call_rcu_data(unsigned long flags, int cpu_affinity)</code><span id="yiv8687194669yui_3_16_0_ym19_1_1481226442633_2990" style="font-family:Times;font-size:medium;">: Returns a handle that can be passed to the other functions in this list. The </span><code id="yiv8687194669yui_3_16_0_ym19_1_1481226442633_2991">flags</code><span id="yiv8687194669yui_3_16_0_ym19_1_1481226442633_2992" style="font-family:Times;font-size:medium;"> argument can be zero, or can be </span><code id="yiv8687194669yui_3_16_0_ym19_1_1481226442633_2993">URCU_CALL_RCU_RT</code><span id="yiv8687194669yui_3_16_0_ym19_1_1481226442633_2994" style="font-family:Times;font-size:medium;"> if the application threads associated with the new callback-invocation thread are to get real-time response from </span><code id="yiv8687194669yui_3_16_0_ym19_1_1481226442633_2995">call_rcu()</code><span id="yiv8687194669yui_3_16_0_ym19_1_1481226442633_2996" style="font-family:Times;font-size:medium;"> by avoiding the need to call into the kernel to wake up the callback-invocation thread.</span></div><div dir="ltr" id="yiv8687194669yui_3_16_0_ym19_1_1481226442633_2947"><span style="font-family:Times;font-size:medium;"><br clear="none"></span></div></blockquote><div id="yiv8687194669yui_3_16_0_ym19_1_1481226442633_3169">So setting "flags" to 0 will simply use the futex mechanism for wakeups. What exactly happens when set to the RT option? Is there something else that needs to be configured from the OS side to make the aforementioned option work correctly?</div></div></blockquote><div><br clear="none"></div><div>No, it's just that the call rcu worker thread will rely on periodic polling<br clear="none"></div><div>of the call rcu work queue.<br clear="none"></div><div><br clear="none"></div><blockquote style="border-left:2px solid #1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica, Arial, sans-serif;font-size:12pt;"><div style="color:#000;background-color:#fff;font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:16px;"><br clear="none"><div id="yiv8687194669yui_3_16_0_ym19_1_1481226442633_3170">I'm asking because we're seeing intermittent issues whereby on application startup, the reclamation thread (call_rcu) does exactly one cleanup, but then never does it again. We have the URCU_CALL_RCU_RT flag set. Originally, we thought it was because the updater was not appropriately registered as a reader. But after ensuring that is the case, it still happens in 10% of startups. </div></div></blockquote><div>Which URCU flavor are you using ? You may have a thread stuck as<br clear="none"></div><div>being a RCU reader forever, which would cause this kind of issue.<br clear="none"></div><div><br clear="none"></div><div>For instance, unbalanced rcu read-side lock/unlock, or blocking on<br clear="none"></div><div>poll() or other while holding a rcu read-side lock, or blocking on<br clear="none"></div><div>pthread_join() while holding rcu read-side lock...<br clear="none"></div><div><br clear="none"></div><div>And if you use the QSBR flavor, remember that registered threads<br clear="none"></div><div>are by default in read-side. So you need to mark them as explicitly<br clear="none"></div><div>"offline" whenever you do any blocking op (e.g. poll(), blocking read(),<br clear="none"></div><div>pthread_join....).<br clear="none"></div><div><br clear="none"></div><div><br clear="none"></div><blockquote style="border-left:2px solid #1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica, Arial, sans-serif;font-size:12pt;"><div style="color:#000;background-color:#fff;font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:16px;"><br clear="none"><div id="yiv8687194669yui_3_16_0_ym19_1_1481226442633_3171">Lastly, we wonder why it would even work correctly *at all* when we failed to register the updater as a reader thread.</div></div></blockquote><div>I don't understand the context of this question. Can you provide more info ?<br clear="none"></div><div><br clear="none"></div><div>Thanks,<br clear="none"></div><div><br clear="none"></div><div>Mathieu<div class="yiv8687194669yqt2659051997" id="yiv8687194669yqtfd37054"><br clear="none"></div></div><div class="yiv8687194669yqt2659051997" id="yiv8687194669yqtfd80120"><div><br clear="none"></div><div><br clear="none"></div><div><br clear="none"></div></div><blockquote style="border-left:2px solid #1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica, Arial, sans-serif;font-size:12pt;"><div class="yiv8687194669yqt2659051997" id="yiv8687194669yqtfd57349"><br clear="none">_______________________________________________<br clear="none">lttng-dev mailing list<br clear="none">lttng-dev@lists.lttng.org<br clear="none">https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev</div><br clear="none"></blockquote></div><div><br clear="none"></div><div>-- <br clear="none"></div><div>Mathieu Desnoyers<br clear="none">EfficiOS Inc.<br clear="none">http://www.efficios.com</div></div></div></div><br><br></div> </div> </div> </div></div></body></html>