[ltt-dev] Kernel crashes when creating a trace

[Mathieu Desnoyers]

* Ashwin Tanugula (ashwin.tanugula at broadcom.com) wrote:
> Mathieu,
> 
> I made the same changes to the kernel with patch-2.6.31.3-lttng-0.162 and I don't see a kernel crash. (I hope this helps to narrow down the problem)
> 
> However the trace data is not generated.
> 

Hrm, weird. Can you try again with the patch I provided on the newer
lttng, but keep disabling the timestamping infrastructure for now.

It should not crash anymore. If we get this right, then we can turn our
focus back on the trace clock.

Thanks !

Mathieu

> Here is what I see
> 
> # /root/ltt-control-0.72-23102009/lttctl/.libs/lttctl -D trace1
> Linux Trace Toolkit Trace Control 0.72-23102009
> 
> Controlling trace : trace1
> 
> lttctl: Pausing trace
> lttctl: LTT: 171 events written in channel vm_state (cpu 0, index 0)
> Destroying traceLTT: 43 events written in channel task_state (cpu 0, index 0)
> 
> LTT: 1 events written in channel swap_state (cpu 0, index 0)
> LTT: 32 events written in channel softirq_state (cpu 0, index 0)
> LTT: 11 events written in channel rcu (cpu 1, index 0)
> LTT: 132 events written in channel rcu (cpu 0, index 0)
> LTT: 2 events written in channel netif_state (cpu 0, index 0)
> LTT: 7 events written in channel net (cpu 0, index 0)
> LTT: 30 events written in channel mm (cpu 1, index 0)
> LTT: 97 events written in channel mm (cpu 0, index 0)
> LTT: 39092 events written in channel kernel (cpu 1, index 0)
> LTT: 52378 events written in channel kernel (cpu 0, index 0)
> LTT: 1 events written in channel jbd2 (cpu 1, index 0)
> LTT: 1 events written in channel jbd2 (cpu 0, index 0)
> LTT: 10 events written in channel irq_state (cpu 0, index 0)
> LTT: 1 events written in channel global_state (cpu 0, index 0)
> LTT: 73 events written in channel fs (cpu 1, index 0)
> LTT: 199 events written in channel fs (cpu 0, index 0)
> LTT: 140 events written in channel fd_state (cpu 0, index 0)
> LTT: 25 events written in channel block (cpu 1, index 0)
> LTT: 15 events written in channel block (cpu 0, index 0)
> LTT: 338 events written in channel metadata (cpu 0, index 0)
> # ls -l /tmp/trace1/
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 block_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 block_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 ext4_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 ext4_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 fd_state_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 fd_state_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 fs_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 fs_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 global_state_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 global_state_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 ipc_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 ipc_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 irq_state_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 irq_state_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 jbd2_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 jbd2_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 kernel_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 kernel_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 metadata_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 metadata_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 mm_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 mm_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 module_state_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 module_state_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 net_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 net_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 netif_state_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 netif_state_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 pm_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 pm_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 rcu_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 rcu_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 softirq_state_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 softirq_state_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 swap_state_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 swap_state_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 task_state_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 task_state_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 userspace_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 userspace_1
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 vm_state_0
> -rwxr-xr-x    1 root     root            0 Jan  1 00:03 vm_state_1
> 
> 
> Am I doing something wrong?
> 
> Thanks,
> Ashwin
> -----Original Message-----
> From: Ashwin Tanugula 
> Sent: Thursday, November 12, 2009 5:10 PM
> To: Mathieu Desnoyers
> Cc: ltt-dev at lists.casi.polymtl.ca
> Subject: RE: Kernel crashes when creating a trace
> 
> Hi Mathieu,
>  
> >Can you also try with a return 1; at the beginning of
> 
> >trace_clock_async_tsc_read ?
> 
> I added "return 1;" to trace_clock_async_tsc_read and removed it from get_trace_clock and put_trace_clock functions.
> 
> And, it does not hang, but the kernel crashes when I try to stop tracing and destroy the channels.
> 
> Here is the what I have in the script. (debugfs is mounted)
> 
> 
> export LTT_DAEMON=/root/ltt-control-0.72-23102009/lttd/lttd
> ltt-control-0.72-23102009/lttctl/ltt-armall
> cd ltt-control-0.72-23102009/lttctl/.libs/
> lttctl -C -w /tmp/trace1 trace1
> 
> 
> To stop tracing 
> 
> /root/ltt-control-0.72-23102009/lttctl/.libs/lttctl -D trace1
> 
> Please look at the attached log.
> 
> Thanks,
> Ashwin
> 

-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68