[ltt-dev] lttctl -D option hangs while trying to destroy trace
fazlin ahamed
fazlincse at gmail.com
Sat Oct 31 10:09:14 EDT 2009
I got un update...
The machine tat i was facing problem had a stripped down version of 2.6.29
kernel with very few kernel configuration options enabled.. (only those
which are really required)..
I got another machine running 2.6.29 but with most of the kernel
configuration option enabled. In this i applied the same patch and lttctl
starts and destroys trace without any problem.
This makes me think whether i have missed any kernel configuration option
that are required for LTT. I have supplied same value to all kernel config
options supplied by LTT. But i guess there is some related kernel
configuration option that i am missing.
Anyone has any idea what might be the missing link??? Or can someone pls let
me know what are the basic options tat have to be enabled for LTT..
Any help is appreciated.
Thanks,
Fazlin
On Sat, Oct 31, 2009 at 2:46 AM, fazlin ahamed <fazlincse at gmail.com> wrote:
> One more query in the meantime i try with 2.6.31..
>
> This machine where i use 2.6.29 is a kind of embedded machine where in
> space is very limited. Hence my lttctl and lttd are on a NFS mounted
> directory and also i store the trace data in a NFS mounted directory. Does
> this affect the performance or will this may be the reason for hang??
>
> Thanks,
> Fazlin
>
>
> On Fri, Oct 30, 2009 at 10:52 PM, Mathieu Desnoyers <
> compudj at krystal.dyndns.org> wrote:
>
>> * fazlin ahamed (fazlincse at gmail.com) wrote:
>> > Ok i ll try out with 2.6.31 and let you know the results.. But the prob
>> is i
>> > would have to make it working for 2.6.29 anyway as it is the kernel tat
>> we
>> > ll be using (we have made few modifications in 2.6.29 to suit our
>> needs)...
>> >
>>
>> If you try with 2.6.31 and it works, then bissecting from 2.6.29 to
>> 2.6.31 will help you finding the fix.
>>
>> Mathieu
>>
>> > I debugged the problem more and here are my updates:
>> >
>> > Once "lttctl -D -w /tmp/tdir/trace3 trace3" cmd hangs, i did ctrl+C to
>> kill
>> > the process.
>> >
>> > After tat ps -aef shows two lttd process running. From the code i
>> understood
>> > tat one was started wen i started trace and other was started wen i
>> issued
>> > "lttctl -D" command.
>> >
>> > And if i kill both these lttd process using "kill" command, i see lot of
>> > "unread channel" messages in dmesg. I have pasted few below:
>> >
>> > LTT : unread channel metadata offset is 262144 and cons_off : 0 (cpu 0)
>> > LTT : metadata : commit count : 262144, subbuf size 262144
>> > LTT : unread channel metadata offset is 262144 and cons_off : 0 (cpu 1)
>> > LTT : metadata : commit count : 262144, subbuf size 262144
>> > LTT : Tracing not active for trace trace1
>> > LTT : unread channel metadata offset is 262144 and cons_off : 0 (cpu 0)
>> > LTT : metadata : commit count : 262144, subbuf size 262144
>> > LTT : unread channel metadata offset is 262144 and cons_off : 0 (cpu 1)
>> > LTT : metadata : commit count : 262144, subbuf size 262144
>> > LTT : unread channel metadata offset is 262144 and cons_off : 0 (cpu 0)
>> > LTT : metadata : commit count : 262144, subbuf size 262144
>> > LTT : unread channel metadata offset is 262144 and cons_off : 0 (cpu 1)
>> > LTT : metadata : commit count : 262144, subbuf size 262144
>> >
>> > My guess from the above msg was there are some trace data in buffers tat
>> > couldnt be read to userspace since i killed "lttctl" and "lttd". Am i
>> > guessing rite?
>> >
>> > I also tried to run "lttd" by myself and i got the following error:
>> >
>> > [root@(none) /]# /tmp/ltt/ltt-control-0.70-18082009/lttd/lttd -t
>> > /tmp/tdir/trace2 -a -c /tmp/debugfs/ltt -v -d -n
>> > Linux Trace Toolkit Trace Daemon 0.70-18082009
>> >
>> > Reading from debugfs directory : /tmp/debugfs/ltt
>> > Writing to trace directory : /tmp/tdir/trace2
>> >
>> > Creating trace subdirectory /tmp/tdir/trace2
>> > Adding inotify for channel /tmp/debugfs/ltt/
>> > Added inotify for channel /tmp/debugfs/ltt/, wd 4294967295
>> > Channel file : /tmp/debugfs/ltt/write_event
>> > Opening file.
>> > Appending to file /tmp/tdir/trace2/write_event as requested
>> > Channel file : /tmp/debugfs/ltt/kprobes
>> > Entering channel subdirectory...
>> > Creating trace subdirectory /tmp/tdir/trace2/kprobes
>> > Adding inotify for channel /tmp/debugfs/ltt/kprobes/
>> > Added inotify for channel /tmp/debugfs/ltt/kprobes/, wd 4294967295
>> > Channel file : /tmp/debugfs/ltt/kprobes/list
>> > Opening file.
>> > Appending to file /tmp/tdir/trace2/kprobes/list as requested
>> > Channel file : /tmp/debugfs/ltt/kprobes/disable
>> > Opening file.
>> > ...
>> > ...
>> > ...
>> > Creating trace subdirectory /tmp/tdir/trace2/control
>> > Adding inotify for channel /tmp/debugfs/ltt/control/
>> > Added inotify for channel /tmp/debugfs/ltt/control/, wd 4294967295
>> > Error in getting the number of subbuffers: Inappropriate ioctl for
>> device
>> >
>> > I am not sure whether these are related or valid. But my guess is
>> > destroy_trace is sleeping/waiting for someone else (maybe lttd) and it
>> is
>> > not able to recover.
>> >
>> > Is there any fix for this problem already in latest ltt-control or lttng
>> > kernel patches??
>> >
>> > Thanks in advance,
>> > Fazlin
>> >
>> > On Fri, Oct 30, 2009 at 7:36 PM, Mathieu Desnoyers <
>> > compudj at krystal.dyndns.org> wrote:
>> >
>> > > * fazlin ahamed (fazlincse at gmail.com) wrote:
>> > > > Hi all,
>> > > >
>> > > > I am new to using lttng and previously i was using lttng in 2.6.18
>> > > without
>> > > > any problems. Now when i switched to 2.6.29, i downloaded the
>> > > corresponding
>> > > > patches (patch-2.6.29-lttng-0.121) and ltt-control tool
>> > > > (ltt-control-0.70-18082009)..
>> > > >
>> > > > I patched the kernel and tried to start/stop tracing using following
>> > > > commands:
>> > > >
>> > > > lttctl -C -w /tmp/tdir/trace3 -o channel.all.overwrite=1 trace3
>> > > >
>> > > > This command ran successfully. And wen i executed
>> > > >
>> > > > lttctl -D -w /tmp/tdir/trace3 trace3
>> > > >
>> > > > The cmd nevers returns... when i debugged further i found tat
>> "write"
>> > > call
>> > > > to write trace3 to debugfs destroy_trace file hangs.
>> > > >
>> > > > Can anyone throw some light on this problem... forgive me if it is
>> silly
>> > > > question..
>> > > >
>> > >
>> > > You might want to try with a newer LTTng version (for kernel 2.6.31)
>> and
>> > > see if you can reproduce the problem.
>> > >
>> > > Thanks,
>> > >
>> > > Mathieu
>> > >
>> > >
>> > > > thanks in advance,
>> > > > Fazlin
>> > >
>> > > > _______________________________________________
>> > > > ltt-dev mailing list
>> > > > ltt-dev at lists.casi.polymtl.ca
>> > > > http://lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev
>> > >
>> > >
>> > > --
>> > > Mathieu Desnoyers
>> > > OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE
>> 9A68
>> > >
>>
>> --
>> Mathieu Desnoyers
>> OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE
>> 9A68
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20091031/588ee2aa/attachment-0003.htm>
More information about the lttng-dev
mailing list