[ltt-dev] [UST issue]: Waiting for ustd to shutdown...Segmentation fault

Fri Jul 30 14:31:54 EDT 2010

2010/7/30 srikanth krishnakar <skrishnakar at gmail.com>:
> Hello Alexis,
>
> Thanks for your Quick Response !!!
>
> There is no trace data collected as per the size of the metadata_* files &
> ust_*

See below.

>
> I am getting the following error while running the lttv as shown below:
>
> -----------------------------
> root at x86-generic:~# lttv -m textDump -t
> /home/root/.usttraces/x86-generic-20100730150429647915588/2886_5499715367809623712/
>
> ** (process:2907): WARNING **: Unsupported trace version : 2.6
>
> ** (process:2907): WARNING **: parse_trace_header error
>

What version of lttv are you using? This "Unsupported trace version"
sounds to me like you are using an older version. If I am not
mistaken, you need at least 0.12.30 to be able to load v2.6 traces.

> ** (process:2907): WARNING **: Unsupported trace version : 2.6
>
> ** (process:2907): WARNING **: parse_trace_header error
>
> ** (process:2907): WARNING **: Unsupported trace version : 2.6
>
> ** (process:2907): WARNING **: parse_trace_header error
>
> ** (process:2907): WARNING **: Unsupported trace version : 2.6
>
> ** (process:2907): WARNING **: parse_trace_header error
>
> ** (process:2907): WARNING **: Trace
> /home/root/.usttraces/x86-generic-20100730150429647915588/2886_5499715367809623712
> has no metadata tracefile
>
> ** (process:2907): CRITICAL **: cannot open trace
> /home/root/.usttraces/x86-generic-20100730150429647915588/2886_5499715367809623712/
> Segmentation fault
> root at x86-generic:~#
> -----------------------------
>
> When looking into the size of trace files:
>
> root at x86-generic:~# du -sh
> /home/root/.usttraces/x86-generic-20100730150429647915588/2886_5499715367809623712/*
> 4.0K
> /home/root/.usttraces/x86-generic-20100730150429647915588/2886_5499715367809623712/metadata_0
> 4.0K
> /home/root/.usttraces/x86-generic-20100730150429647915588/2886_5499715367809623712/metadata_1
> 4.0K
> /home/root/.usttraces/x86-generic-20100730150429647915588/2886_5499715367809623712/ust_0
> 4.0K
> /home/root/.usttraces/x86-generic-20100730150429647915588/2886_5499715367809623712/ust_1
> root at x86-generic:~#
>
> They all seem to be empty.

The sizes are not unusual if you are tracing a simple test application
with one or two markers. What exactly is in usttest.c? Did you copy
the sample code from section 4.1 of the manual? I am tracing this code
just fine with UST 0.5 and liburcu 0.46, as you mentioned you are
using. I am getting 4.0K files that load fine in lttv 0.12.33.

>
> For reference I have attached the strace log for usttrace.
>
> Thanks,
> Srikanth
> ---------
> On Fri, Jul 30, 2010 at 9:12 PM, Alexis Hallé <alexis.halle at polymtl.ca>
> wrote:
>>
>> Hi Srikanth,
>>
>> 2010/7/30 srikanth krishnakar <skrishnakar at gmail.com>:
>> > Hello Pierre/Mathieu,
>> >
>> > I have followed the instructions for UST(user space tracing)  from
>> > http://lttng.org/files/ust/manual/ust.html
>> >
>> > 1. Installed liburcu & ust
>> >
>> > liburcu-0.46
>> > ust-0.5
>> >
>> > 2. Compiled the usttest.c
>> >
>> > root at x86:~/ust-test# gcc -o usttest -lust usttest.c
>> >
>> > root at x86-generic:~/ust-test# ldd ./usttest
>> >         linux-gate.so.1 =>  (0xb789f000)
>> >         libust.so.0 => /usr/lib/libust.so.0 (0xb787a000)
>> >         libc.so.6 => /lib/libc.so.6 (0xb772d000)
>> >         /lib/ld-linux.so.2 (0xb78a0000)
>> >         liburcu-bp.so.0 => /usr/lib/liburcu-bp.so.0 (0xb772a000)
>> >         librt.so.1 => /lib/librt.so.1 (0xb7715000)
>> >         libpthread.so.0 => /lib/libpthread.so.0 (0xb76fb000)
>> > root at x86-generic:~/ust-test#
>> >
>> > Package Versions:
>> >
>> > ###################
>> > GLIBC: 2.11.1
>> > GCC: 4.4
>> > libglib-2.0-0
>> > ###################
>> >
>> > 3. Now start tracing :
>> >
>> > root at x86-generic:~/ust-test# usttrace ./usttest
>> > Waiting for ustd to shutdown...
>>
>> The usttrace script starts a daemon on-the-fly for this tracing
>> session. When it is done tracing it sends this daemon a SIGTERM signal
>> and waits for it to shutdown. This message is perfectly normal.
>>
>> > Trace was output in:
>> > /home/root/.usttraces/x86-generic-20100730144935535976601
>> > root at x86-generic:~/ust-test#
>> >
>> > Seems some issue in executing of ustctl..
>> >
>> > Now checking the presence of trace file:
>> >
>> > root at x86-generic:~/.usttraces/x86-generic-20100730144935535976601# du
>> > -sh *
>> > 20K     2851_5499711528108666788
>> > 0       app.log
>> > 0       ustd.log
>> > root at x86-generic:~/.usttraces/x86-generic-20100730144935535976601#
>> >
>> > 4. Dumping the result on console :
>> >
>> >
>> > root at x86-generic:~# lttv -m textDump -t
>> > /home/root/.usttraces/x86-generic-20100730144935535976601
>> >
>> > ** (process:2862): WARNING **: Unsupported trace version : 2.6
>> >
>> > ** (process:2862): WARNING **: parse_trace_header error
>> >
>> > ** (process:2862): WARNING **: Unsupported trace version : 2.6
>> >
>> > ** (process:2862): WARNING **: parse_trace_header error
>> >
>> > ** (process:2862): WARNING **: Unsupported trace version : 2.6
>> >
>> > ** (process:2862): WARNING **: parse_trace_header error
>> >
>> > ** (process:2862): WARNING **: Unsupported trace version : 2.6
>> >
>> > ** (process:2862): WARNING **: parse_trace_header error
>> >
>> > ** (process:2862): WARNING **: Trace
>> > /home/root/.usttraces/x86-20100730144935535976601 has no metadata
>> > tracefile
>> >
>> > ** (process:2862): CRITICAL **: cannot open trace
>> > /home/root/.usttraces/x86-20100730144935535976601
>> > Segmentation fault
>> > root at x86-generic:~#
>> >
>> > DMESG OUTPUT:
>> >
>> > root at x86-generic:~# dmesg | grep -ir ltt
>> > [ 5108.713591] lttv.real[2862]: segfault at 40 ip b7732222 sp bfd88e90
>> > error
>> > 4 in libglib-2.0.so.0.2200.1[b7714000+b6000]
>> > root at x86-generic:~#
>> >
>> > What is mismatching section/package that is causing this segfault ?
>> >
>> > Your inputs are appreciated !! Thanks !!
>>
>> When you look inside
>> /home/root/.usttraces/x86-generic-20100730144935535976601, you can see
>> a directory named 2851_5499711528108666788. This is the directory
>> containing the actual trace files, and it is this path that you must
>> pass to lttv with the -t switch.
>>
>> >
>> >
>> > Regards,
>> > Srikanth Krishnakar
>> > **********************
>> >
>> > _______________________________________________
>> > ltt-dev mailing list
>> > ltt-dev at lists.casi.polymtl.ca
>> > http://lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev
>> >
>> >
>>
>> Regards,
>>
>> --
>> Alexis Hallé
>> Software Engineering student
>> Intern, Tracing & Monitoring Project
>> École Polytechnique de Montréal
>
>
>
> --
> "The Good You Do, The Best You GET"
>
> Regards
> Srikanth Krishnakar
> **********************
>

-- 
Alexis Hallé
Software Engineering student
Intern, Tracing & Monitoring Project
École Polytechnique de Montréal