[lttng-dev] Double free or corruption error (fasttop)

[Shehab Elsayed]

Hello All,

I am trying to instrument a pthread application using the provided pthread
wrapper, but I sometimes run into a "Double free or corruption error (
fasttop)" error.

Here is a description of what I have tried and noticed:
1- The problem isn't consistent. It sometimes happen and sometimes works as
expected.
2- From my experiments, the problem happens (more frequently at least) when
adding performance counter contexts (I tried cycles, cpu_cycles and
instructions).
3- I am testing using lu_ncb from splash3 benchmark suite after setting LD_
PRELOAD to use the pthread wrapper as described in the LTTng documents.
4- Here is the backtrace printed after exiting:
======= Backtrace: =========
/lib64/libc.so.6([Thread 0x7ffff5611700 (LWP 97229) exited]
/usr/local/lib/liblttng-ust.so.0(lttng_destroy_context+0x35)[0x7ffff7471575]
/usr/local/lib/liblttng-ust.so.0(lttng
_session_destroy+0x21c)[0x7ffff747363c]
/usr/local/lib/liblttng-ust.so.0(+0x1e906)[0x7ffff746d906]
/usr/local/lib/liblttng-ust.so.0(lttng_ust_objd_unref+0x9f)[0x7ffff746dccf]
/usr/local/lib/liblttng-ust.so.0(lttng_ust_objd_unref+0x9f)[0x7ffff746dccf]
/usr/local/lib/liblttng-ust.so.0(lttng_ust_objd_unref+0x9f)[0x7ffff746dccf]
/usr/local/lib/liblttng-ust.so.0(lttng_ust_abi_exit+0x68)[0x7ffff746ead8]
/usr/local/lib/liblttng-ust.so.0(+0x191d3)[0x7ffff74681d3]
/usr/local/lib/liblttng-ust.so.0(lttng_ust_exit+0x67)[0x7ffff745ed57]
/lib64/ld-linux-x86-64.so.2(+0xf85a)[0x7ffff7dec85a]
/lib64/libc.so.6(+0x38a49)[0x7ffff6ca6a49]
/lib64/libc.so.6(+0x38a95)[0x7ffff6ca6a95]
/aenao-99/elsayed9/LTTng/data/scripts/tmp/lu_ncb[0x401b51]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7ffff6c8fb35]
/aenao-99/elsayed9/LTTng/data/scripts/tmp/lu_ncb[0x401c44]
5- Also, this is a backtrace I obtained from gdb:
#0  0x00007ffff6eac1d7 in raise () from /lib64/libc.so.6
#1  0x00007ffff6ead8c8 in abort () from /lib64/libc.so.6
#2  0x00007ffff6eebf07 in __libc_message () from /lib64/libc.so.6
#3  0x00007ffff6ef3503 in _int_free () from /lib64/libc.so.6
#4  0x00007ffff768ad25 in lttng_destroy_perf_counter_field (
    field=<optimized out>) at lttng-context-perf-counters.c:418
#5  0x00007ffff767a575 in lttng_destroy_context (
    ctx=0x7ffff0011090) at lttng-context.c:278
#6  0x00007ffff767c63c in _lttng_channel_unmap (
    lttng_chan=0x7ffff0010f40) at lttng-events.c:172
#7  lttng_session_destroy (session=0x7ffff0000900)
    at lttng-events.c:247
#8  0x00007ffff7676906 in lttng_release_session (
    objd=<optimized out>) at lttng-ust-abi.c:601
#9  0x00007ffff7676ccf in lttng_ust_objd_unref (id=1,
    is_owner=<optimized out>) at lttng-ust-abi.c:216
#10 0x00007ffff7676ccf in lttng_ust_objd_unref (id=2,
    is_owner=<optimized out>) at lttng-ust-abi.c:216
#11 0x00007ffff7676ccf in lttng_ust_objd_unref (id=id at entry=18,
    is_owner=is_owner at entry=1) at lttng-ust-abi.c:216
#12 0x00007ffff7677ad8 in objd_table_destroy ()
    at lttng-ust-abi.c:235
#13 lttng_ust_abi_exit () at lttng-ust-abi.c:1002
#14 0x00007ffff76711d3 in lttng_ust_cleanup (exiting=1)
    at lttng-ust-comm.c:1807
#15 0x00007ffff7667d57 in lttng_ust_exit ()
    at lttng-ust-comm.c:1874
#16 0x00007ffff7dec85a in _dl_fini ()
   from /lib64/ld-linux-x86-64.so.2
#17 0x00007ffff6eafa49 in __run_exit_handlers ()
   from /lib64/libc.so.6
#18 0x00007ffff6eafa95 in exit () from /lib64/libc.so.6
#19 0x0000000000401b51 in main (argc=<optimized out>,
    argv=<optimized out>) at lu.c:368

Any ideas, why this happens and how to fix it?

Thanks,
Shehab
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.lttng.org/pipermail/lttng-dev/attachments/20180316/c21e8a04/attachment.html>