[ltt-dev] Kernel crashes when creating a trace

[Mathieu Desnoyers]

* Ashwin Tanugula (ashwin.tanugula at broadcom.com) wrote:
> Mathieu,
> 
> I made the same changes to the kernel with patch-2.6.31.3-lttng-0.162 and I don't see a kernel crash. (I hope this helps to narrow down the problem)
> 
> However the trace data is not generated.
> 

Also try upgrading to the brand new ltt-control 0.74. I just saw that a
return value was missing from the __lttctl_set_channel_subbuf_size
function in liblttctl. It could also help.

Thanks,

Mathieu

> Here is what I see
> 
> # /root/ltt-control-0.72-23102009/lttctl/.libs/lttctl -D trace1
> Linux Trace Toolkit Trace Control 0.72-23102009
> 
> Controlling trace : trace1
> 
> lttctl: Pausing trace
> lttctl: LTT: 171 events written in channel vm_state (cpu 0, index 0)
> Destroying traceLTT: 43 events written in channel task_state (cpu 0, index 0)
> 
> LTT: 1 events written in channel swap_state (cpu 0, index 0)
> LTT: 32 events written in channel softirq_state (cpu 0, index 0)
> LTT: 11 events written in channel rcu (cpu 1, index 0)
> LTT: 132 events written in channel rcu (cpu 0, index 0)
> LTT: 2 events written in channel netif_state (cpu 0, index 0)
> LTT: 7 events written in channel net (cpu 0, index 0)
> LTT: 30 events written in channel mm (cpu 1, index 0)
> LTT: 97 events written in channel mm (cpu 0, index 0)
> LTT: 39092 events written in channel kernel (cpu 1, index 0)
> LTT: 52378 events written in channel kernel (cpu 0, index 0)
> LTT: 1 events written in channel jbd2 (cpu 1, index 0)
> LTT: 1 events written in channel jbd2 (cpu 0, index 0)
> LTT: 10 events written in channel irq_state (cpu 0, index 0)
> LTT: 1 events written in channel global_state (cpu 0, index 0)
> LTT: 73 events written in channel fs (cpu 1, index 0)
> LTT: 199 events written in channel fs (cpu 0, index 0)
> LTT: 140 events written in channel fd_state (cpu 0, index 0)
> LTT: 25 events written in channel block (cpu 1, index 0)
> LTT: 15 events written in channel block (cpu 0, index 0)
> LTT: 338 events written in channel metadata (cpu 0, index 0)
> # ls -l /tmp/trace1/
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 block_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 block_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 ext4_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 ext4_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 fd_state_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 fd_state_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 fs_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 fs_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 global_state_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 global_state_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 ipc_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 ipc_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 irq_state_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 irq_state_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 jbd2_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 jbd2_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 kernel_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 kernel_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 metadata_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 metadata_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 mm_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 mm_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 module_state_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 module_state_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 net_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 net_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 netif_state_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 netif_state_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 pm_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 pm_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 rcu_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 rcu_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 softirq_state_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 softirq_state_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 swap_state_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 swap_state_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 task_state_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 task_state_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 userspace_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 userspace_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 vm_state_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 vm_state_1
> 
> 
> Am I doing something wrong?
> 
> Thanks,
> Ashwin
> -----Original Message-----
> From: Ashwin Tanugula 
> Sent: Thursday, November 12, 2009 5:10 PM
> To: Mathieu Desnoyers
> Cc: ltt-dev at lists.casi.polymtl.ca
> Subject: RE: Kernel crashes when creating a trace
> 
> Hi Mathieu,
>  
> >Can you also try with a return 1; at the beginning of
> 
> >trace_clock_async_tsc_read ?
> 
> I added "return 1;" to trace_clock_async_tsc_read and removed it from get_trace_clock and put_trace_clock functions.
> 
> And, it does not hang, but the kernel crashes when I try to stop tracing and destroy the channels.
> 
> Here is the what I have in the script. (debugfs is mounted)
> 
> 
> export LTT_DAEMON=/root/ltt-control-0.72-23102009/lttd/lttd
> ltt-control-0.72-23102009/lttctl/ltt-armall
> cd ltt-control-0.72-23102009/lttctl/.libs/
> lttctl -C -w /tmp/trace1 trace1
> 
> 
> To stop tracing 
> 
> /root/ltt-control-0.72-23102009/lttctl/.libs/lttctl -D trace1
> 
> Please look at the attached log.
> 
> Thanks,
> Ashwin
> 

-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68