[lttng-dev] urcu workqueue thread uses 99% of cpu while workqueue is empty

[Minlan Wang]

Hi, Mathieu,
The commit on branch stable-0.12 correponds to the tarball we downloaded is this:

commit d5277e807192178ddb79f56ecbbd5ac3c4994f60 (HEAD -> v0.12.1.b, tag: v0.12.1)
Author: Mathieu Desnoyers <mathieu.desnoyers at efficios.com>
Date:   Wed Apr 22 08:51:41 2020 -0400

    Version 0.12.1

    Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers at efficios.com>

The OS we are using is CentOS Linux release 7.9.2009 (Core), not CentOS 8.2
as mentioned before. And the kernel version is: 3.10.0-1160.el7.x86_64.

On Tue, Jun 14, 2022 at 11:53:16AM -0400, Mathieu Desnoyers wrote:
> Also, I notice that you appear to be using an internal liburcu API (not public)
> from outside of the liburcu project, which is not really expected.
We are trying to move some linux kernel module function into userspace, and
found that the urcu internal workqueue.h has all the things we need for a
replace for kernel workqueue, so we decided to give it a try.

> 
> If your process forks without exec, make sure you wire up the equivalent of
> rculfhash pthread_atfork functions which call urcu_workqueue_pause_worker(),
> urcu_workqueue_resume_worker() and urcu_workqueue_create_worker().
There's no fork/exec in the process who is calling alloc_workqueue, and the
threads who are enqueue work into the workqueue is created by calling
pthread_create.

> 
> Also, can you validate of you have many workqueue worker threads trying to
> dequeue from the same workqueue in parallel ? This is unsupported and would
> cause the kind of issues you are observing here.
The workqueue thread is created by calling urcu_workqueue_create in the code
below, and it is the only thread which will dequeue work from the workqueue.
Though, there are multiple threads who will enqueue work by calling
urcu_workqueue_queue_work(wq, work, work->func).
---
static void workqueue_init_fn(struct urcu_workqueue *workqueue, void *priv)
{
	pthread_t tid;
	const char *name;
	char thread_name[16] = {0};

	if (!priv)
		return;

	name = (const char *)priv;
	tid = pthread_self();
	
	memcpy(thread_name, name, 15);
	if (pthread_setname_np(tid, thread_name)) {
		pr_err("failed to set thread name for workqueue %s\n", name);
	}

	urcu_memb_register_thread();
}

static void workqueue_finalize_fn(struct urcu_workqueue *workqueue, void
				  *priv)
{
	urcu_memb_unregister_thread();
	if (priv)
		free(priv);
}

struct workqueue_struct *alloc_workqueue(const char *fmt,
					 unsigned int flags,
					 int max_active, ...)
{
	const char *name;

	name = strdup(fmt);
	if (!name) {
		pr_err("failed to dup name for workqueue %s\n", fmt);
		return NULL;
	}

	return urcu_workqueue_create(0, -1, (void *)name,
			      NULL,			/* grace */
			      workqueue_init_fn,	/* init */
			      workqueue_finalize_fn,	/* finalize */
			      NULL,			/* before wait */
			      NULL,			/* after wake up */
			      NULL,			/* before pasue */
			      NULL);			/* after resume */
}
---

B.R
Minlan