[lttng-dev] RFC: Fix crash in dlerror()

[Stefan Seefeld]

I have been looking into an issue with the malloc wrapper, where an
unsuccessful call to dlopen(), followed by a call to dlerror(), would
result in a segmentation fault when the malloc wrapper is being used.

The problem is the following:

The functions dlopen() and dlsym() make use of a global (though
thread-local) "result" structure to hold the error state, to allow a
subsequent call to dlerror() to report it.

As it turns out, dlerror() itself may implicitly call realloc(), which,
if it hasn't been used before, triggers our wrapper to call dlsym(). So,
while dlerror() inspects said result structure, dlsym() re-initializes
it, causing the crash...

This is arguably a bug in the dlfcn functions. The attached patch
attempts to fix this by moving the initialization of the realloc()
wrapper (i.e., the loading of the symbol) into the constructor. This
fixes the crash that I'm observing, but since none of these dependencies
are specified or documented, this change may cause other issues elsewhere.

Are there any objections to this approach ? If not, I'll submit a formal
patch for this.

Thanks,
        Stefan

-- 
Stefan Seefeld
CodeSourcery / Mentor Graphics
http://www.mentor.com/embedded-software/

-------------- next part --------------
diff --git a/liblttng-ust-libc-wrapper/lttng-ust-malloc.c b/liblttng-ust-libc-wrapper/lttng-ust-malloc.c
index 33ed18b..3f391db 100644
--- a/liblttng-ust-libc-wrapper/lttng-ust-malloc.c
+++ b/liblttng-ust-libc-wrapper/lttng-ust-malloc.c
@@ -54,6 +54,14 @@ static void *static_calloc(size_t nmemb, size_t size)
 	return &static_calloc_buf[prev_offset];
 }
 
+static void *(*plibc_realloc)(void *ptr, size_t size);
+
+__attribute__((constructor))
+static void init()
+{
+	plibc_realloc = dlsym(RTLD_NEXT, "realloc");
+}
+
 void *malloc(size_t size)
 {
 	static void *(*plibc_malloc)(size_t size);
@@ -119,7 +127,6 @@ void *calloc(size_t nmemb, size_t size)
 
 void *realloc(void *ptr, size_t size)
 {
-	static void *(*plibc_realloc)(void *ptr, size_t size);
 	void *retval;
 
 	if (plibc_realloc == NULL) {