[lttng-dev] segfault when exiting process with background thread

[Mathieu Desnoyers]

Hi Jeffrey, 

I CC'd your @live.com address on the RFC patch. Please let me know if the approach 
is OK with you (and try it out). 

See: 
https://lists.lttng.org/pipermail/lttng-dev/2016-March/025608.html 

Thanks, 

Mathieu 

----- On Mar 10, 2016, at 1:35 PM, Jeffrey Chen <cpthk at hotmail.com> wrote: 

> Thanks Mathieu.

> Is there a plan at LTTng side to fix this issue? If so, we could wait for the
> fix. If not, we will have to workaround the problem for now (probably by
> applying your fix ourselves). Thanks.

> From: Mathieu Desnoyers <mathieu.desnoyers at efficios.com>
> Sent: Tuesday, March 8, 2016 6:09 PM
> To: Jonathan Rajotte Julien
> Cc: lttng-dev
> Subject: Re: [lttng-dev] segfault when exiting process with background thread
> The attached patch works around the issue. The problem is
> the tracepoint destructors that run in parallel with the thread
> that still exists.

> If we remove this, we leak the liblttng-ust-tracepoint.so.0 shared
> object on dlclose of the instrumented code. Not sure if we should
> care though...

> Thoughts ?

> Thanks,

> Mathieu

> ----- On Mar 8, 2016, at 7:35 PM, Mathieu Desnoyers
> <mathieu.desnoyers at efficios.com> wrote:

>> Reproduced it on master (ust and tools).

>> Repro with:

>> lttng create
>> lttng enable-event -u -a
>> lttng start

>> then run hello a few times, it eventually segfaults in the spawned thread.

>> Looking into it.

>> Thanks,

>> Mathieu

>> ----- On Mar 8, 2016, at 7:12 PM, Jonathan Rajotte Julien
>> <Jonathan.rajotte-julien at efficios.com> wrote:

>>> Hi Jeffrey,

>>> Could you provide the version for lttng-ust, lttng-tools, lttng-modules (if
>>> installed)?

>>> Were you able to reproduce the issue on master (git) ?

>>> What are the exact order of command to reproduce the problem ?

>>> Thanks

>>> On 2016-03-08 06:43 PM, Jeffrey Chen wrote:

>>>> Hi LTTng community:

>>>> We are using LTTng for our production environment. We have been noticing
>>>> segfault problem when our process exit. We think the problem is due to the
>>>> background thread continue to write traces, while LTTng cleanup its memory. We
>>>> do not know how to fix this issue without changing LTTng code base. I have
>>>> written a very simple app that could repro the problem. Most of my codes are
>>>> copy pasted from the LTTng doc sample. Is there any fix that the LTTng side
>>>> could do?

>>>> hello.c

>>>> #include <stdio.h>
>>>> #include "hello-tp.h"
>>>> #include <pthread.h>

>>>> void* doSomeThing(void *arg)
>>>> {
>>>> int x;
>>>> for (x = 0; x < 100000; ++x) {
>>>> tracepoint(hello_world, my_first_tracepoint, x, "test");
>>>> }
>>>> }

>>>> int main(int argc, char *argv[])
>>>> {
>>>> int x;

>>>> getchar();

>>>> pthread_t inc_x_thread;

>>>> if(pthread_create(&inc_x_thread, NULL, doSomeThing, NULL)) {

>>>> fprintf(stderr, "Error creating thread\n");
>>>> return 1;
>>>> }

>>>> tracepoint(hello_world, my_first_tracepoint, 23, "hi there!");

>>>> return 0;
>>>> }

>>>> hello-tp.c

>>>> #define TRACEPOINT_CREATE_PROBES
>>>> #define TRACEPOINT_DEFINE

>>>> #include "hello-tp.h"

>>>> hello-tp.h

>>>> #undef TRACEPOINT_PROVIDER
>>>> #define TRACEPOINT_PROVIDER hello_world

>>>> #undef TRACEPOINT_INCLUDE
>>>> #define TRACEPOINT_INCLUDE "./hello-tp.h"

>>>> #if !defined(_HELLO_TP_H) || defined(TRACEPOINT_HEADER_MULTI_READ)
>>>> #define _HELLO_TP_H

>>>> #include <lttng/tracepoint.h>

>>>> TRACEPOINT_EVENT(
>>>> hello_world,
>>>> my_first_tracepoint,
>>>> TP_ARGS(
>>>> int, my_integer_arg,
>>>> char*, my_string_arg
>>>> ),
>>>> TP_FIELDS(
>>>> ctf_string(my_string_field, my_string_arg)
>>>> ctf_integer(int, my_integer_field, my_integer_arg)
>>>> )
>>>> )

>>>> #endif /* _HELLO_TP_H */

>>>> #include <lttng/tracepoint-event.h>

>>>> Compile

>>>> gcc -c -I. hello-tp.c
>>>> gcc -c hello.c
>>>> gcc -o hello hello.o hello-tp.o -llttng-ust -ldl -lpthread

>>>> _______________________________________________
>>>> lttng-dev mailing list lttng-dev at lists.lttng.org
>>>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

>>> --
>>> Jonathan R. Julien
>>> Efficios

>>> _______________________________________________
>>> lttng-dev mailing list
>>> lttng-dev at lists.lttng.org
>>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

>> --
>> Mathieu Desnoyers
>> EfficiOS Inc.
>> http://www.efficios.com

>> _______________________________________________
>> lttng-dev mailing list
>> lttng-dev at lists.lttng.org
>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

> --
> Mathieu Desnoyers
> EfficiOS Inc.
> http://www.efficios.com

-- 
Mathieu Desnoyers 
EfficiOS Inc. 
http://www.efficios.com 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.lttng.org/pipermail/lttng-dev/attachments/20160310/f84389e9/attachment.html>