[ltt-dev] lttctl -D option hangs while trying to destroy trace

[Mathieu Desnoyers]

* fazlin ahamed (fazlincse at gmail.com) wrote:
> Ok i ll try out with 2.6.31 and let you know the results.. But the prob is i
> would have to make it working for 2.6.29 anyway as it is the kernel tat we
> ll be using (we have made few modifications in 2.6.29 to suit our needs)...
> 

If you try with 2.6.31 and it works, then bissecting from 2.6.29 to
2.6.31 will help you finding the fix.

Mathieu

> I debugged the problem more and here are my updates:
> 
> Once "lttctl -D -w /tmp/tdir/trace3 trace3" cmd hangs, i did ctrl+C to kill
> the process.
> 
> After tat ps -aef shows two lttd process running. From the code i understood
> tat one was started wen i started trace and other was started wen i issued
> "lttctl -D" command.
> 
> And if i kill both these lttd process using "kill" command, i see lot of
> "unread channel" messages in dmesg. I have pasted few below:
> 
> LTT : unread channel metadata offset is 262144 and cons_off : 0 (cpu 0)
> LTT : metadata : commit count : 262144, subbuf size 262144
> LTT : unread channel metadata offset is 262144 and cons_off : 0 (cpu 1)
> LTT : metadata : commit count : 262144, subbuf size 262144
> LTT : Tracing not active for trace trace1
> LTT : unread channel metadata offset is 262144 and cons_off : 0 (cpu 0)
> LTT : metadata : commit count : 262144, subbuf size 262144
> LTT : unread channel metadata offset is 262144 and cons_off : 0 (cpu 1)
> LTT : metadata : commit count : 262144, subbuf size 262144
> LTT : unread channel metadata offset is 262144 and cons_off : 0 (cpu 0)
> LTT : metadata : commit count : 262144, subbuf size 262144
> LTT : unread channel metadata offset is 262144 and cons_off : 0 (cpu 1)
> LTT : metadata : commit count : 262144, subbuf size 262144
> 
> My guess from the above msg was there are some trace data in buffers tat
> couldnt be read to userspace since i killed "lttctl" and "lttd". Am i
> guessing rite?
> 
> I also tried to run "lttd" by myself and i got the following error:
> 
> [root@(none) /]# /tmp/ltt/ltt-control-0.70-18082009/lttd/lttd -t
> /tmp/tdir/trace2 -a -c /tmp/debugfs/ltt -v -d -n
> Linux Trace Toolkit Trace Daemon 0.70-18082009
> 
> Reading from debugfs directory : /tmp/debugfs/ltt
> Writing to trace directory : /tmp/tdir/trace2
> 
> Creating trace subdirectory /tmp/tdir/trace2
> Adding inotify for channel /tmp/debugfs/ltt/
> Added inotify for channel /tmp/debugfs/ltt/, wd 4294967295
> Channel file : /tmp/debugfs/ltt/write_event
> Opening file.
> Appending to file /tmp/tdir/trace2/write_event as requested
> Channel file : /tmp/debugfs/ltt/kprobes
> Entering channel subdirectory...
> Creating trace subdirectory /tmp/tdir/trace2/kprobes
> Adding inotify for channel /tmp/debugfs/ltt/kprobes/
> Added inotify for channel /tmp/debugfs/ltt/kprobes/, wd 4294967295
> Channel file : /tmp/debugfs/ltt/kprobes/list
> Opening file.
> Appending to file /tmp/tdir/trace2/kprobes/list as requested
> Channel file : /tmp/debugfs/ltt/kprobes/disable
> Opening file.
> ...
> ...
> ...
> Creating trace subdirectory /tmp/tdir/trace2/control
> Adding inotify for channel /tmp/debugfs/ltt/control/
> Added inotify for channel /tmp/debugfs/ltt/control/, wd 4294967295
> Error in getting the number of subbuffers: Inappropriate ioctl for device
> 
> I am not sure whether these are related or valid. But my guess is
> destroy_trace is sleeping/waiting for someone else (maybe lttd) and it is
> not able to recover.
> 
> Is there any fix for this problem already in latest ltt-control or lttng
> kernel patches??
> 
> Thanks in advance,
> Fazlin
> 
> On Fri, Oct 30, 2009 at 7:36 PM, Mathieu Desnoyers <
> compudj at krystal.dyndns.org> wrote:
> 
> > * fazlin ahamed (fazlincse at gmail.com) wrote:
> > > Hi all,
> > >
> > > I am new to using lttng and previously i was using lttng in 2.6.18
> > without
> > > any problems. Now when i switched to 2.6.29, i downloaded the
> > corresponding
> > > patches (patch-2.6.29-lttng-0.121) and ltt-control tool
> > > (ltt-control-0.70-18082009)..
> > >
> > > I patched the kernel and tried to start/stop tracing using following
> > > commands:
> > >
> > > lttctl -C -w /tmp/tdir/trace3 -o channel.all.overwrite=1 trace3
> > >
> > > This command ran successfully. And wen i executed
> > >
> > > lttctl -D -w /tmp/tdir/trace3 trace3
> > >
> > > The cmd nevers returns... when i debugged further i found tat "write"
> > call
> > > to write trace3 to debugfs destroy_trace file hangs.
> > >
> > > Can anyone throw some light on this problem... forgive me if it is silly
> > > question..
> > >
> >
> > You might want to try with a newer LTTng version (for kernel 2.6.31) and
> > see if you can reproduce the problem.
> >
> > Thanks,
> >
> > Mathieu
> >
> >
> > > thanks in advance,
> > > Fazlin
> >
> > > _______________________________________________
> > > ltt-dev mailing list
> > > ltt-dev at lists.casi.polymtl.ca
> > > http://lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev
> >
> >
> > --
> > Mathieu Desnoyers
> > OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68
> >

-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68