[lttng-dev] Double free or corruption error (fasttop)
Shehab Elsayed
shehabyomn at gmail.com
Mon Mar 19 12:36:42 EDT 2018
Hi Mathieu,
Thank you very much for your reply.
I manually built lttng-ust from source (commit #:
8a208943e21700211beee3ea64180a5a534c7d2a).
This is how I set up the tracing session:
1- lttng create lu_ncb_8_native -o {path}
2- lttng enable-event --userspace lttng_ust_pthread:pthread_mutex_lock_req
lttng enable-event --userspace lttng_ust_pthread:pthread_mutex_lock_acq
lttng enable-event --userspace
lttng_ust_pthread:pthread_mutex_lock_trylock
lttng enable-event --userspace
lttng_ust_pthread:pthread_mutex_lock_unlock
3- lttng add-context -u -t procname
lttng add-context -u -t vpid
lttng add-context -u -t pthread_id
lttng add-context -u -t vtid
lttng add-context -u -t ip
lttng add-context -u -t perf:thread:cpu-cycles
lttng add-context -u -t perf:thread:cycles
lttng add-context -u -t perf:thread:instructions
4- lttng start
5- LD_PRELOAD=/usr/local/lib/liblttng-ust-pthread-wrapper.so ./lu_ncb -p8
-n8096 -b32
6- lttng stop
7- lttng destroy
Shehab Y. Elsayed, MSc.
PhD Student
The Edwards S. Rogers Sr. Dept. of Electrical and Computer Engineering
University of Toronto
E-mail: shehabyomn at gmail.com
<https://webmail.rice.edu/imp/message.php?mailbox=INBOX&index=11#>
On Mon, Mar 19, 2018 at 12:21 PM, Mathieu Desnoyers <
mathieu.desnoyers at efficios.com> wrote:
>
>
> ----- On Mar 16, 2018, at 5:37 PM, Shehab Elsayed <shehabyomn at gmail.com>
> wrote:
>
> Hello All,
>
> I am trying to instrument a pthread application using the provided pthread
> wrapper, but I sometimes run into a "Double free or corruption error (
> fasttop)" error.
>
>
> Please provide more information about the version of lttng-ust you are
> using, and how you setup
> your tracing session.
>
> Thanks,
>
> Mathieu
>
>
>
> Here is a description of what I have tried and noticed:
> 1- The problem isn't consistent. It sometimes happen and sometimes works
> as expected.
> 2- From my experiments, the problem happens (more frequently at least)
> when adding performance counter contexts (I tried cycles, cpu_cycles and
> instructions).
> 3- I am testing using lu_ncb from splash3 benchmark suite after setting LD
> _PRELOAD to use the pthread wrapper as described in the LTTng documents.
> 4- Here is the backtrace printed after exiting:
> ======= Backtrace: =========
> /lib64/libc.so.6([Thread 0x7ffff5611700 (LWP 97229) exited]
> /usr/local/lib/liblttng-ust.so.0(lttng_destroy_context+
> 0x35)[0x7ffff7471575]
> /usr/local/lib/liblttng-ust.so.0(lttng_session_destroy+
> 0x21c)[0x7ffff747363c]
> /usr/local/lib/liblttng-ust.so.0(+0x1e906)[0x7ffff746d906]
> /usr/local/lib/liblttng-ust.so.0(lttng_ust_objd_unref+
> 0x9f)[0x7ffff746dccf]
> /usr/local/lib/liblttng-ust.so.0(lttng_ust_objd_unref+
> 0x9f)[0x7ffff746dccf]
> /usr/local/lib/liblttng-ust.so.0(lttng_ust_objd_unref+
> 0x9f)[0x7ffff746dccf]
> /usr/local/lib/liblttng-ust.so.0(lttng_ust_abi_exit+0x68)[0x7ffff746ead8]
> /usr/local/lib/liblttng-ust.so.0(+0x191d3)[0x7ffff74681d3]
> /usr/local/lib/liblttng-ust.so.0(lttng_ust_exit+0x67)[0x7ffff745ed57]
> /lib64/ld-linux-x86-64.so.2(+0xf85a)[0x7ffff7dec85a]
> /lib64/libc.so.6(+0x38a49)[0x7ffff6ca6a49]
> /lib64/libc.so.6(+0x38a95)[0x7ffff6ca6a95]
> /aenao-99/elsayed9/LTTng/data/scripts/tmp/lu_ncb[0x401b51]
> /lib64/libc.so.6(__libc_start_main+0xf5)[0x7ffff6c8fb35]
> /aenao-99/elsayed9/LTTng/data/scripts/tmp/lu_ncb[0x401c44]
> 5- Also, this is a backtrace I obtained from gdb:
> #0 0x00007ffff6eac1d7 in raise () from /lib64/libc.so.6
> #1 0x00007ffff6ead8c8 in abort () from /lib64/libc.so.6
> #2 0x00007ffff6eebf07 in __libc_message () from /lib64/libc.so.6
> #3 0x00007ffff6ef3503 in _int_free () from /lib64/libc.so.6
> #4 0x00007ffff768ad25 in lttng_destroy_perf_counter_field (
> field=<optimized out>) at lttng-context-perf-counters.c:418
> #5 0x00007ffff767a575 in lttng_destroy_context (
> ctx=0x7ffff0011090) at lttng-context.c:278
> #6 0x00007ffff767c63c in _lttng_channel_unmap (
> lttng_chan=0x7ffff0010f40) at lttng-events.c:172
> #7 lttng_session_destroy (session=0x7ffff0000900)
> at lttng-events.c:247
> #8 0x00007ffff7676906 in lttng_release_session (
> objd=<optimized out>) at lttng-ust-abi.c:601
> #9 0x00007ffff7676ccf in lttng_ust_objd_unref (id=1,
> is_owner=<optimized out>) at lttng-ust-abi.c:216
> #10 0x00007ffff7676ccf in lttng_ust_objd_unref (id=2,
> is_owner=<optimized out>) at lttng-ust-abi.c:216
> #11 0x00007ffff7676ccf in lttng_ust_objd_unref (id=id at entry=18,
> is_owner=is_owner at entry=1) at lttng-ust-abi.c:216
> #12 0x00007ffff7677ad8 in objd_table_destroy ()
> at lttng-ust-abi.c:235
> #13 lttng_ust_abi_exit () at lttng-ust-abi.c:1002
> #14 0x00007ffff76711d3 in lttng_ust_cleanup (exiting=1)
> at lttng-ust-comm.c:1807
> #15 0x00007ffff7667d57 in lttng_ust_exit ()
> at lttng-ust-comm.c:1874
> #16 0x00007ffff7dec85a in _dl_fini ()
> from /lib64/ld-linux-x86-64.so.2
> #17 0x00007ffff6eafa49 in __run_exit_handlers ()
> from /lib64/libc.so.6
> #18 0x00007ffff6eafa95 in exit () from /lib64/libc.so.6
> #19 0x0000000000401b51 in main (argc=<optimized out>,
> argv=<optimized out>) at lu.c:368
>
> Any ideas, why this happens and how to fix it?
>
> Thanks,
> Shehab
>
>
>
> _______________________________________________
> lttng-dev mailing list
> lttng-dev at lists.lttng.org
> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>
>
> --
> Mathieu Desnoyers
> EfficiOS Inc.
> http://www.efficios.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.lttng.org/pipermail/lttng-dev/attachments/20180319/33d2483e/attachment-0001.html>
More information about the lttng-dev
mailing list