[lttng-dev] URCU and pthread mutex/cond variable hang

[Mathieu Desnoyers]

----- Original Message -----
> From: "Jeff Layton" <jlayton at poochiereds.net>
> To: lttng-dev at lists.lttng.org
> Sent: Tuesday, March 3, 2015 3:40:14 PM
> Subject: [lttng-dev] URCU and pthread mutex/cond variable hang
> 
> I've been using urcu to develop some userland code, and I've run into a
> problem that I don't quite understand. If I register a thread and then
> have that thread block on a pthread condition variable, then it seems
> to cause synchronize_rcu in another thread to hang.
> 
> I've attached a testcase that demonstrates the problem. You have to
> build it and link it against -lpthread and -lurcu-qsbr:
> 
>     $ gcc -Wall -o ./urcu_hang -lpthread -lurcu-qsbr urcu_hang.c
> 
> ...it will run fine. If you comment out the rcu_thread_offline/online
> calls though, it will hang.
> 
> Why? Is this expected behavior or a bug in urcu?

This is because you are using the urcu QSBR flavor. For this
flavor, the default state of a registered thread is to be
within a RCU read-side critical section, thus to block
synchronize_rcu() until the next rcu_quiescent_state() call
or until the next extended quiescent state (thread offline).

Therefore, you need to take care to put the threads in
offline mode before you issue blocking operations that
depend on completion of synchronize_rcu() to proceed further,
which is the case of test_rcu() in your test program. Otherwise
you create a deadlock, where main() is in a RCU read-side critical
section while it blocks awaiting on the pthread cond var.
Unfortuntately, the test_rcu() thread is unable to issue
the pthread cond signal, because it is blocked on synchronize_rcu(),
because main is itself in a RCU critical section.

Another alternative would be to use the other URCU flavors such
as urcu-mb, urcu-signal or urcu-membarrier. Their default state
is to be in a RCU quiescent state, which is IMHO more intuitive
for the users. But QSBR is the fastest flavor, but it comes at
the expense of a somewhat more complex API.

Hoping this explanation helps,

Thanks,

Mathieu



> 
> Thanks in advance!
> --
> Jeff Layton <jlayton at poochiereds.net>
> 
> _______________________________________________
> lttng-dev mailing list
> lttng-dev at lists.lttng.org
> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> 

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com