[lttng-dev] Tracepoints for pthreads conditional variables
Shehab Elsayed
shehabyomn at gmail.com
Thu Jul 5 17:36:30 EDT 2018
Any suggestions where I should investigate to fix this bug? Also, please
let me know whether the git repo I shared is sufficient.
Thanks,
Shehab
Shehab Y. Elsayed, MSc.
PhD Student
The Edwards S. Rogers Sr. Dept. of Electrical and Computer Engineering
University of Toronto
E-mail: shehabyomn at gmail.com
<https://webmail.rice.edu/imp/message.php?mailbox=INBOX&index=11#>
On Tue, May 29, 2018 at 10:36 AM, Shehab Elsayed <shehabyomn at gmail.com>
wrote:
> I was wondering if you had a chance to look over this issue. Thanks!
>
> Shehab Y. Elsayed, MSc.
> PhD Student
> The Edwards S. Rogers Sr. Dept. of Electrical and Computer Engineering
> University of Toronto
> E-mail: shehabyomn at gmail.com
> <https://webmail.rice.edu/imp/message.php?mailbox=INBOX&index=11#>
>
> On Thu, May 3, 2018 at 4:37 PM, Shehab Elsayed <shehabyomn at gmail.com>
> wrote:
>
>> Thanks for the reply. Here is a link to the repo
>> https://github.com/ShehabElsayed/LTTng_debugging.git
>>
>> It contains the LTTng source code I am using and one of the benchmarks
>> that is causing problems.
>>
>> Shehab Y. Elsayed, MSc.
>> PhD Student
>> The Edwards S. Rogers Sr. Dept. of Electrical and Computer Engineering
>> University of Toronto
>> E-mail: shehabyomn at gmail.com
>> <https://webmail.rice.edu/imp/message.php?mailbox=INBOX&index=11#>
>>
>> On Tue, May 1, 2018 at 10:39 AM, Jonathan Rajotte-Julien <
>> jonathan.rajotte-julien at efficios.com> wrote:
>>
>>> Hi Shehab,
>>>
>>> Please provide a link to a lttng-ust git tree or an actual patch (git
>>> format-patch).
>>> This will help any person who might be interested in helping you
>>> including
>>> myself.
>>>
>>> Cheers
>>>
>>> On Tue, May 01, 2018 at 09:31:52AM -0400, Shehab Elsayed wrote:
>>> > Hi,
>>> >
>>> > I am trying to create wrappers for pthreads conditional variables
>>> functions
>>> > (wait, signal and broadcast) to insert tracepoints. I followed the same
>>> > approach done for the mutex functions already provided with lttng,
>>> however
>>> > I am running into some problems. Mainly the applications seem to get
>>> stuck
>>> > when the conditional variable wrappers are enabled.
>>> >
>>> > Here is my implementation for the wrapper functions:
>>> > int pthread_cond_wait(pthread_cond_t *condition, pthread_mutex_t
>>> > *mutex)
>>> >
>>> > {
>>> >
>>> > static int (*cond_wait)(pthread_cond_t *, pthread_mutex_t
>>> > *);
>>> > int retval;
>>> >
>>> >
>>> >
>>> > if (!cond_wait)
>>> > {
>>> >
>>> > cond_wait = dlsym(RTLD_NEXT, "pthread_cond_wait");
>>> >
>>> > if (!cond_wait)
>>> > {
>>> >
>>> > if (thread_in_trace)
>>> > {
>>> >
>>> >
>>> > abort();
>>> >
>>> >
>>> > }
>>> >
>>> > fprintf(stderr, "unable to initialize pthread wrapper
>>> > library.\n");
>>> > return EINVAL;
>>> >
>>> >
>>> > }
>>> >
>>> >
>>> > }
>>> >
>>> > if (thread_in_trace)
>>> > {
>>> >
>>> > return cond_wait(condition, mutex);
>>> >
>>> >
>>> > }
>>> >
>>> >
>>> >
>>> > thread_in_trace =
>>> > 1;
>>> >
>>> > tracepoint(lttng_ust_pthread, pthread_cond_wait_begin, condition,
>>> > mutex,
>>> >
>>> > LTTNG_UST_CALLER_IP());
>>> >
>>> > retval = cond_wait(condition, mutex);
>>> >
>>> > tracepoint(lttng_ust_pthread, pthread_cond_wait_end, condition,
>>> > mutex,
>>> >
>>> > LTTNG_UST_CALLER_IP());
>>> >
>>> > thread_in_trace =
>>> > 0;
>>> >
>>> > return retval;
>>> >
>>> > }
>>> >
>>> >
>>> >
>>> > int pthread_cond_signal(pthread_cond_t
>>> > *condition)
>>> >
>>> > {
>>> >
>>> > static int (*cond_signal)(pthread_cond_t
>>> > *);
>>> >
>>> > int retval;
>>> >
>>> >
>>> >
>>> > if (!cond_signal)
>>> > {
>>> >
>>> > cond_signal = dlsym(RTLD_NEXT, "pthread_cond_signal");
>>> >
>>> > if (!cond_signal)
>>> > {
>>> >
>>> > if (thread_in_trace)
>>> > {
>>> >
>>> >
>>> > abort();
>>> >
>>> >
>>> > }
>>> >
>>> > fprintf(stderr, "unable to initialize pthread wrapper
>>> > library.\n");
>>> > return EINVAL;
>>> >
>>> >
>>> > }
>>> >
>>> >
>>> > }
>>> >
>>> > if (thread_in_trace)
>>> > {
>>> >
>>> > return cond_signal(condition);
>>> >
>>> >
>>> > }
>>> >
>>> >
>>> >
>>> > thread_in_trace =
>>> > 1;
>>> >
>>> > tracepoint(lttng_ust_pthread, pthread_cond_signal_begin,
>>> > condition,
>>> > LTTNG_UST_CALLER_IP());
>>> >
>>> > retval = cond_signal(condition);
>>> >
>>> > tracepoint(lttng_ust_pthread, pthread_cond_signal_end,
>>> > condition,
>>> > LTTNG_UST_CALLER_IP());
>>> >
>>> > thread_in_trace =
>>> > 0;
>>> >
>>> > return retval;
>>> >
>>> > }
>>> >
>>> >
>>> >
>>> > int pthread_cond_broadcast(pthread_cond_t
>>> > *condition)
>>> >
>>> > {
>>> >
>>> > static int (*cond_broadcast)(pthread_cond_t
>>> > *);
>>> > int retval;
>>> >
>>> >
>>> >
>>> > if (!cond_broadcast)
>>> > {
>>> >
>>> > cond_broadcast = dlsym(RTLD_NEXT, "pthread_cond_broadcast");
>>> >
>>> > if (!cond_broadcast)
>>> > {
>>> >
>>> > if (thread_in_trace)
>>> > {
>>> >
>>> >
>>> > abort();
>>> >
>>> >
>>> > }
>>> >
>>> > fprintf(stderr, "unable to initialize pthread wrapper
>>> > library.\n");
>>> > return EINVAL;
>>> >
>>> >
>>> > }
>>> >
>>> >
>>> > }
>>> >
>>> > if (thread_in_trace)
>>> > {
>>> >
>>> > return cond_broadcast(condition);
>>> >
>>> >
>>> > }
>>> >
>>> >
>>> >
>>> > thread_in_trace =
>>> > 1;
>>> >
>>> > tracepoint(lttng_ust_pthread, pthread_cond_broadcast_begin,
>>> > condition,
>>> > LTTNG_UST_CALLER_IP());
>>> >
>>> > retval = cond_broadcast(condition);
>>> >
>>> > tracepoint(lttng_ust_pthread, pthread_cond_broadcast_end,
>>> > condition,
>>> > LTTNG_UST_CALLER_IP());
>>> >
>>> > thread_in_trace =
>>> > 0;
>>> >
>>> > return retval;
>>> >
>>> > }
>>> >
>>> > Things I have tried:
>>> > 1- Comment out pthread_cond_wait function --> segmentation fault
>>> > 2- Comment out pthread_cond_signal and _broadcast --> stuck (I can see
>>> the
>>> > program is still running but nor forward progress is being made)
>>> compared
>>> > to the case when the wrappers are not preloaded.
>>> > 3- Comment out all tracepoint related code (basically the wrapper just
>>> call
>>> > the corresponding pthreads function) --> same results as points 1 and
>>> 2.
>>> >
>>> > Any idea what might be causing this or how I could debug this problem?
>>> >
>>> > Thank you very much in advance.
>>> >
>>> > Best Regards,
>>> > Shehab
>>>
>>> > int pthread_cond_wait(pthread_cond_t *condition, pthread_mutex_t
>>> *mutex)
>>> > {
>>>
>>>
>>>
>>> > static int (*cond_wait)(pthread_cond_t *, pthread_mutex_t *);
>>>
>>> > int retval;
>>>
>>> >
>>>
>>> > if (!cond_wait) {
>>>
>>> > cond_wait = dlsym(RTLD_NEXT, "pthread_cond_wait");
>>>
>>> > if (!cond_wait) {
>>>
>>> > if (thread_in_trace) {
>>>
>>> > abort();
>>>
>>> > }
>>>
>>> > fprintf(stderr, "unable to initialize pthread wrapper
>>> library.\n");
>>> > return EINVAL;
>>>
>>> > }
>>>
>>> > }
>>>
>>> > if (thread_in_trace) {
>>>
>>> > return cond_wait(condition, mutex);
>>>
>>> > }
>>>
>>> >
>>>
>>> > thread_in_trace = 1;
>>>
>>> > tracepoint(lttng_ust_pthread, pthread_cond_wait_begin, condition,
>>> mutex,
>>> > LTTNG_UST_CALLER_IP());
>>>
>>> > retval = cond_wait(condition, mutex);
>>>
>>> > tracepoint(lttng_ust_pthread, pthread_cond_wait_end, condition,
>>> mutex,
>>> > LTTNG_UST_CALLER_IP());
>>>
>>> > thread_in_trace = 0;
>>>
>>> > return retval;
>>>
>>> > }
>>>
>>> >
>>>
>>> > int pthread_cond_signal(pthread_cond_t *condition)
>>>
>>> > {
>>>
>>> > static int (*cond_signal)(pthread_cond_t *);
>>>
>>> > int retval;
>>>
>>> >
>>>
>>> > if (!cond_signal) {
>>>
>>> > cond_signal = dlsym(RTLD_NEXT, "pthread_cond_signal");
>>>
>>> > if (!cond_signal) {
>>>
>>> > if (thread_in_trace) {
>>>
>>> > abort();
>>>
>>> > }
>>>
>>> > fprintf(stderr, "unable to initialize pthread wrapper
>>> library.\n");
>>> > return EINVAL;
>>>
>>> > }
>>>
>>> > }
>>>
>>> > if (thread_in_trace) {
>>>
>>> > return cond_signal(condition);
>>>
>>> > }
>>>
>>> >
>>>
>>> > thread_in_trace = 1;
>>>
>>> > tracepoint(lttng_ust_pthread, pthread_cond_signal_begin, condition,
>>>
>>> > LTTNG_UST_CALLER_IP());
>>>
>>> > retval = cond_signal(condition);
>>>
>>> > tracepoint(lttng_ust_pthread, pthread_cond_signal_end, condition,
>>>
>>> > LTTNG_UST_CALLER_IP());
>>>
>>> > thread_in_trace = 0;
>>>
>>> > return retval;
>>>
>>> > }
>>>
>>> >
>>>
>>> > int pthread_cond_broadcast(pthread_cond_t *condition)
>>>
>>> > {
>>>
>>> > static int (*cond_broadcast)(pthread_cond_t *);
>>>
>>> > int retval;
>>>
>>> >
>>>
>>> > if (!cond_broadcast) {
>>>
>>> > cond_broadcast = dlsym(RTLD_NEXT, "pthread_cond_broadcast");
>>>
>>> > if (!cond_broadcast) {
>>>
>>> > if (thread_in_trace) {
>>>
>>> > abort();
>>>
>>> > }
>>>
>>> > fprintf(stderr, "unable to initialize pthread wrapper
>>> library.\n");
>>> > return EINVAL;
>>>
>>> > }
>>>
>>> > }
>>>
>>> > if (thread_in_trace) {
>>>
>>> > return cond_broadcast(condition);
>>>
>>> > }
>>>
>>> >
>>>
>>> > thread_in_trace = 1;
>>>
>>> > tracepoint(lttng_ust_pthread, pthread_cond_broadcast_begin,
>>> condition,
>>> > LTTNG_UST_CALLER_IP());
>>>
>>> > retval = cond_broadcast(condition);
>>>
>>> > tracepoint(lttng_ust_pthread, pthread_cond_broadcast_end,
>>> condition,
>>> > LTTNG_UST_CALLER_IP());
>>>
>>> > thread_in_trace = 0;
>>>
>>> > return retval;
>>>
>>> > }
>>>
>>> > _______________________________________________
>>> > lttng-dev mailing list
>>> > lttng-dev at lists.lttng.org
>>> > https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>>
>>>
>>> --
>>> Jonathan Rajotte-Julien
>>> EfficiOS
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.lttng.org/pipermail/lttng-dev/attachments/20180705/87b82350/attachment-0001.html>
More information about the lttng-dev
mailing list