[lttng-dev] Using CRIU with LTTng-instrumented applications

[Daniel Glöckner]

Hi,

we are trying to use LTTng in an application that will be dumped and restored
across reboots with CRIU. The problem is that CRIU is unable to restore the
unix sockets used to communicate with lttng-sessiond and would try to restore
the lttng-ust-wait-8 shared memories to the contents they had when the
application was dumped. The good thing is that our application knows when it
will be dumped by CRIU and could tell liblttng-ust to prepare for that.

I see two possibilities:

1. Drop the connection to lttng-sessiond and catch the thread with a mutex or
   semaphore when it wants to access wait_shm_mmap. Temporarily unmap the
   shared memory and let CRIU do its work before allowing the thread to
   continue

2. Cause the threads to terminate. Do the steps done in
   lttng_ust_after_fork_child, but allow CRIU to be called before
   lttng_ust_ctor is called again.

I suspect the two ust_listener_thread instances will be in one of two states
most of the time. Either there is a connection to lttng-sessiond and the thread
is waiting inside ustcomm_recv_unix_sock or there is no connection and the
thread is doing FUTEX_WAIT on the shared memory. We can make it stop waiting
inside ustcomm_recv_unix_sock by using shutdown() with SHUT_RD on the socket
from another thread. The only way to interrupt FUTEX_WAIT without writing to
the shared memory is probably to send a signal. I have seen warnings in the
code advising against pthread_join, but I think with shutdown() and
pthread_kill we can make the thread reevaluate lttng_ust_comm_should_quit in a
safe way (or cause it to reacquire the mutex/semaphore mentioned above in
possibility 1).

Any thoughts on this? Which direction looks more promising? Is there any
chance to get helper functions like this accepted into liblttng-ust?

Best regards,

  Daniel


-- 
Dipl.-Math. Daniel Glöckner, emlix GmbH, http://www.emlix.com
Fon +49 551 30664-0, Fax +49 551 30664-11,
Berliner Straße 12, 37073 Göttingen, Germany
Sitz der Gesellschaft: Göttingen, Amtsgericht Göttingen HR B 3160
Geschäftsführung: Heike Jordan, Dr. Uwe Kracke
Ust-IdNr.: DE 205 198 055

emlix - your embedded linux partner