[lttng-dev] Userspace RCU: workqueue with batching, cheap wakeup, and work stealing

[Mathieu Desnoyers]

Hi Lai,

I pushed a few updates within the past days, including one
fixing this issue. The updated branch is available at:

git://git.urcu.so/urcu.git
branch: urcu/workqueue-wakeup

gitweb:
http://git.lttng.org/?p=userspace-rcu.git;a=shortlog;h=refs/heads/urcu/workqueue-wakeup

Please let me know if you notice anything else,

Thanks!

Mathieu

----- Original Message -----
> From: "Lai Jiangshan" <laijs at cn.fujitsu.com>
> To: "Mathieu Desnoyers" <mathieu.desnoyers at efficios.com>
> Cc: "Ben Maurer" <bmaurer at fb.com>, "lttng-dev" <lttng-dev at lists.lttng.org>, "Yannick Brosseau" <scientist at fb.com>,
> "Paul E. McKenney" <paulmck at linux.vnet.ibm.com>, "Stephen Hemminger" <stephen at networkplumber.org>
> Sent: Wednesday, October 22, 2014 6:57:18 AM
> Subject: Re: Userspace RCU: workqueue with batching, cheap wakeup, and work stealing
> 
> Hi
> 
> I just glance the code quickly...
> 
> __urcu_steal_work() and __urcu_wakeup_siblings don't test
> if they visit the queue->sibling_head.
> 
> All the tasks may continue stealing works without any progress.
> 
> Thanks,
> Lai
> 
> On 10/19/2014 05:13 AM, Mathieu Desnoyers wrote:
> > Hi,
> > 
> > I have written an implementation of a workqueue based
> > on wfcqueue (fifo workqueue), wfstack (lifo wait queue)
> > including a work-stealing scheme. I think this could
> > scale very naturally to large workloads, and it skips
> > the overhead of sys_futex when wakees are already running.
> > Since the wakeup scheme is very lightweight when work
> > is ongoing, we use it extensively.
> > 
> > It also makes extensive use of wfcqueue "splice" operation
> > to move work batch in bulk amongst queue and worker
> > threads. It is a O(1) operation which does not need to
> > touch the content of the queue.
> > 
> > RCU is used to lookup the sibling worker threads for
> > work stealing and to wake the siblings of a thread
> > busy handling work.
> > 
> > It's at:
> > 
> > git://git.urcu.so/urcu.git
> > branch: urcu/workqueue-wakeup
> > 
> > or gitweb of the top commit:
> > http://git.lttng.org/?p=userspace-rcu.git;a=commit;h=2c6a5414ed2ab44be24211d15a5d9f1f40dbfc74
> > 
> > Note that this is work in progress, compile-tested only.
> > I also need to write extensive tests.
> > 
> > Feedback is welcome,
> > 
> > Thanks!
> > 
> > Mathieu
> > 
> 
> 

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com