<html><head></head><body><div style="color:#000; background-color:#fff; font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:16px"><div id="yui_3_16_0_ym19_1_1481233834337_5494">Yes, this is QSBR flavor. </div><div id="yui_3_16_0_ym19_1_1481233834337_5494"><br></div><div id="yui_3_16_0_ym19_1_1481233834337_5494">Regarding the context to your last question, I'm curious about how an updater in QSBR mode using call_rcu *without properly registering itself as a reader* could ever work properly. In our case, this was the initial mistake we noticed that we were doing, yet we only encountered the aforementioned startup error a relatively small percentage of the time (i.e., the reclamation thread only performing a cleanup once).</div><div class="qtdSeparateBR" id="yui_3_16_0_ym19_1_1481233834337_5507"><br><br></div><div class="yahoo_quoted" id="yui_3_16_0_ym19_1_1481233834337_5519" style="display: block;"> <div style="font-family: HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif; font-size: 16px;" id="yui_3_16_0_ym19_1_1481233834337_5518"> <div style="font-family: HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif; font-size: 16px;" id="yui_3_16_0_ym19_1_1481233834337_5517"> <div dir="ltr" id="yui_3_16_0_ym19_1_1481233834337_5657"> <font size="2" face="Arial" id="yui_3_16_0_ym19_1_1481233834337_5656"> <hr size="1"> <b><span style="font-weight:bold;">From:</span></b> Mathieu Desnoyers <mathieu.desnoyers@efficios.com><br> <b><span style="font-weight: bold;">To:</span></b> "Mark E. Dawson, Jr." <medawsonjr@yahoo.com> <br><b><span style="font-weight: bold;">Cc:</span></b> lttng-dev <lttng-dev@lists.lttng.org><br> <b><span style="font-weight: bold;">Sent:</span></b> Thursday, December 8, 2016 3:46 PM<br> <b><span style="font-weight: bold;">Subject:</span></b> Re: [lttng-dev] URCU_CALL_RCU_RT and its implications<br> </font> </div> <div class="y_msg_container" id="yui_3_16_0_ym19_1_1481233834337_5516"><br><div id="yiv0967250936"><div id="yui_3_16_0_ym19_1_1481233834337_5515"><div style="font-family:arial, helvetica, sans-serif;font-size:12pt;color:#000000;" id="yui_3_16_0_ym19_1_1481233834337_5514"><div id="yui_3_16_0_ym19_1_1481233834337_5658"><span id="yiv0967250936zwchr">----- On Dec 8, 2016, at 2:53 PM, lttng-dev <lttng-dev@lists.lttng.org> wrote:<br clear="none"></span></div><div id="yui_3_16_0_ym19_1_1481233834337_5513"><blockquote style="border-left:2px solid #1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica, Arial, sans-serif;font-size:12pt;" id="yui_3_16_0_ym19_1_1481233834337_5621"><div style="color:#000;background-color:#fff;font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:16px;" id="yui_3_16_0_ym19_1_1481233834337_5620"><div id="yiv0967250936yui_3_16_0_ym19_1_1481226442633_2947">All,</div><div id="yiv0967250936yui_3_16_0_ym19_1_1481226442633_2947"><br clear="none"></div><div id="yiv0967250936yui_3_16_0_ym19_1_1481226442633_2947">In the documentation for the create_call_rcu_data() call, we have the following description of the URCU_CALL_RCU_RT flag:</div><div id="yiv0967250936yui_3_16_0_ym19_1_1481226442633_2947"><br clear="none"></div><blockquote id="yiv0967250936yui_3_16_0_ym19_1_1481226442633_3052" style="margin:0 0 0 40px;border:none;padding:0px;"><div dir="ltr" id="yiv0967250936yui_3_16_0_ym19_1_1481226442633_2947"><code id="yiv0967250936yui_3_16_0_ym19_1_1481226442633_2989">struct call_rcu_data *create_call_rcu_data(unsigned long flags, int cpu_affinity)</code><span id="yiv0967250936yui_3_16_0_ym19_1_1481226442633_2990" style="font-family:Times;font-size:medium;">: Returns a handle that can be passed to the other functions in this list. The </span><code id="yiv0967250936yui_3_16_0_ym19_1_1481226442633_2991">flags</code><span id="yiv0967250936yui_3_16_0_ym19_1_1481226442633_2992" style="font-family:Times;font-size:medium;"> argument can be zero, or can be </span><code id="yiv0967250936yui_3_16_0_ym19_1_1481226442633_2993">URCU_CALL_RCU_RT</code><span id="yiv0967250936yui_3_16_0_ym19_1_1481226442633_2994" style="font-family:Times;font-size:medium;"> if the application threads associated with the new callback-invocation thread are to get real-time response from </span><code id="yiv0967250936yui_3_16_0_ym19_1_1481226442633_2995">call_rcu()</code><span id="yiv0967250936yui_3_16_0_ym19_1_1481226442633_2996" style="font-family:Times;font-size:medium;"> by avoiding the need to call into the kernel to wake up the callback-invocation thread.</span></div><div dir="ltr" id="yiv0967250936yui_3_16_0_ym19_1_1481226442633_2947"><span style="font-family:Times;font-size:medium;"><br clear="none"></span></div></blockquote><div id="yiv0967250936yui_3_16_0_ym19_1_1481226442633_3169">So setting "flags" to 0 will simply use the futex mechanism for wakeups. What exactly happens when set to the RT option? Is there something else that needs to be configured from the OS side to make the aforementioned option work correctly?</div></div></blockquote><div><br clear="none"></div><div>No, it's just that the call rcu worker thread will rely on periodic polling<br clear="none"></div><div>of the call rcu work queue.<br clear="none"></div><div><br clear="none"></div><blockquote style="border-left:2px solid #1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica, Arial, sans-serif;font-size:12pt;" id="yui_3_16_0_ym19_1_1481233834337_5628"><div style="color:#000;background-color:#fff;font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:16px;" id="yui_3_16_0_ym19_1_1481233834337_5627"><br clear="none"><div id="yiv0967250936yui_3_16_0_ym19_1_1481226442633_3170">I'm asking because we're seeing intermittent issues whereby on application startup, the reclamation thread (call_rcu) does exactly one cleanup, but then never does it again. We have the URCU_CALL_RCU_RT flag set. Originally, we thought it was because the updater was not appropriately registered as a reader. But after ensuring that is the case, it still happens in 10% of startups. </div></div></blockquote><div>Which URCU flavor are you using ? You may have a thread stuck as<br clear="none"></div><div id="yui_3_16_0_ym19_1_1481233834337_5629">being a RCU reader forever, which would cause this kind of issue.<br clear="none"></div><div id="yui_3_16_0_ym19_1_1481233834337_5631"><br clear="none"></div><div id="yui_3_16_0_ym19_1_1481233834337_5524">For instance, unbalanced rcu read-side lock/unlock, or blocking on<br clear="none"></div><div>poll() or other while holding a rcu read-side lock, or blocking on<br clear="none"></div><div>pthread_join() while holding rcu read-side lock...<br clear="none"></div><div><br clear="none"></div><div>And if you use the QSBR flavor, remember that registered threads<br clear="none"></div><div>are by default in read-side. So you need to mark them as explicitly<br clear="none"></div><div id="yui_3_16_0_ym19_1_1481233834337_5522">"offline" whenever you do any blocking op (e.g. poll(), blocking read(),<br clear="none"></div><div>pthread_join....).<br clear="none"></div><div><br clear="none"></div><div><br clear="none"></div><blockquote style="border-left:2px solid #1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica, Arial, sans-serif;font-size:12pt;"><div style="color:#000;background-color:#fff;font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:16px;"><br clear="none"><div id="yiv0967250936yui_3_16_0_ym19_1_1481226442633_3171">Lastly, we wonder why it would even work correctly *at all* when we failed to register the updater as a reader thread.</div></div></blockquote><div>I don't understand the context of this question. Can you provide more info ?<br clear="none"></div><div><br clear="none"></div><div>Thanks,<br clear="none"></div><div><br clear="none"></div><div id="yui_3_16_0_ym19_1_1481233834337_5512">Mathieu<div class="yiv0967250936yqt4998000334" id="yiv0967250936yqtfd47668"><br clear="none"></div></div><div class="yiv0967250936yqt4998000334" id="yiv0967250936yqtfd59846"><div id="yui_3_16_0_ym19_1_1481233834337_5624"><br clear="none"></div><div><br clear="none"></div><div><br clear="none"></div></div><blockquote style="border-left:2px solid #1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica, Arial, sans-serif;font-size:12pt;"><div class="yiv0967250936yqt4998000334" id="yiv0967250936yqtfd70691"><br clear="none">_______________________________________________<br clear="none">lttng-dev mailing list<br clear="none">lttng-dev@lists.lttng.org<br clear="none">https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev</div><br clear="none"></blockquote></div><div><br clear="none"></div><div>-- <br clear="none"></div><div>Mathieu Desnoyers<br clear="none">EfficiOS Inc.<br clear="none">http://www.efficios.com</div></div></div></div><br><br></div> </div> </div> </div></div></body></html>