[lttng-dev] Alternative to signals/sys_membarrier() in liburcu

[Michael Sullivan]

I've been looking at the RCU library (as part of gathering examples for my
research on weak memory models) and was thinking about ways to force other
threads to issue barriers. Since it seems like sys_membarrier() never made
it into the kernel, I was pondering whether there was some other way to
more or less get its effect; as it turns out, there is, but it is a hack:
mprotect(2).

When a thread revokes write permissions on a page, the kernel needs to do a
TLB shootdown to make sure that none of the other CPUs running code in that
address space have a writable mapping for that page cached. In Linux, this
is done by forcing code to invalidate the mappings to run on every CPU in
the address space, and waiting for completion. The code for the "run this
function on another CPU" mechanism forces the target CPU to issue an
smp_mb().

(In practice TLB shootdowns are done when permissions are added, not just
when they are removed, but they needn't be; faults caused by using a cached
entry with less permissions can be fixed up by the page fault handler.
They're also needed when unmapping memory, but mprotect() seems cheaper
than having to mmap() and munmap(). Also TLB shootdowns aren't needed if
the page is non-present because it's never been backed or has been swapped
out, so mlock(2) is used to keep it in place).

I hacked this up and in my limited testing, it does seem to have way better
write side performance than the signal version has. That said, it is also
super hacky and is certainly depending on behaviors of mprotect() that are
not actually specified. It would be unusual, I think, to implement
mprotect() in a way where this didn't work? It may well be to janky to
actually be useful, though.

I can send the code if you're interested.

-Michael Sullivan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lttng.org/pipermail/lttng-dev/attachments/20150210/70f63254/attachment.html>