[ltt-dev] Some LTTng 0.43 issue on ARM
Mathieu Desnoyers
compudj at krystal.dyndns.org
Tue Aug 4 09:18:27 EDT 2009
* Bryan Hsieh (bryanhsieh at fihspec.com) wrote:
> Hi Mathieu, All
>
> Please check my comment which marked by [Bryan], Thanks!
> -----------------------------------------------------------------------------
> [Bryan]
> All the related LTT functions are build-in not build as modules, Please check attachment- LTTng Kernel Config..txt, I am not sure if Kernel->user space relay support (formerly relayfs) need to build-in.
> Is there any item I missed?
> -----------------------------------------------------------------------------
> [ ] Kernel->user space relay support (formerly relayfs)
LTTng does not need relayfs anymore. So this is ok.
> [*] Activate tracepoints
> [*] Activate markers
> Linux Trace Toolkit --->
> <*> Compile lttng tracing probes
> [*] LTTng fine-grained timestamping
> [*] Linux Trace Toolkit Instrumentation Support
> <*> Linux Trace Toolkit High-speed Lockless Data Relay
> <*> Linux Trace Toolkit Lock-Protected Data Relay
> [*] Debug check for random access in ltt relay buffers
> <*> Linux Trace Toolkit Serializer
> <*> Linux Trace Toolkit Marker Control
> <*> Linux Trace Toolkit Tracer
> [*] Align Linux Trace Toolkit Traces
> [*] Add event size field to LTT events for tracer debugging
> [*] Support trace extraction from crash dump
> <*> Linux Trace Toolkit Netlink Controller
> <*> Linux Trace Toolkit State Dump
> -----------------------------------------------------------------------------
> [Bryan]
> I am sure that the external utilities are build-in busybox, please check the attachment - " busybox version and command query.txt".
>
> Finally, I figure out why the ltt-armall doesn't work correctly in my target system, root cause is the output pipeline will be empty after grep twice, even I use a temp file to avoid connecting two grep commands directly, it's still empty. So I copy the /proc/ltt to my local desktop to do ltt-armall script, then generate the /proc/ltt which been filtered by following command.
> MARKERS=`cat /proc/ltt|grep -v %k|awk '{print $2}'|sort -u|grep -v ^core_|grep -v ^locking_|grep -v ^lockdep_`
> I rename it as ltt_filter and copy to my target system path /system/bin/, then modify ltt-armall by following modification and run it.
> MARKERS=`cat /system/bin/ltt_filter`
>
> The execution result on screen:
> # ltt-armall-1
> Connecting all markers
> Connecting fs_buffer_wait_end
> Connecting fs_buffer_wait_start
> Connecting fs_close
> Connecting fs_exec
> Connecting fs_ioctl
> Connecting fs_llseek
> Connecting fs_lseek
> Connecting fs_open
> Connecting fs_pollfd
> Connecting fs_pread64
> Connecting fs_pwrite64
> Connecting fs_read
> Connecting fs_readv
> Connecting fs_select
> Connecting fs_write
> Connecting fs_writev
> Connecting input_event
> Connecting ipc_msg_create
> Connecting ipc_sem_create
> Connecting ipc_shm_create
> Connecting kernel_arch_ipc_call
> Connecting kernel_arch_kthread_create
> Connecting kernel_arch_syscall_entry
> Connecting kernel_arch_syscall_exit
> Connecting kernel_arch_trap_entry
> Connecting kernel_arch_trap_exit
> Connecting kernel_irq_entry
> Connecting kernel_irq_exit
> Connecting kernel_kthread_stop
> Connecting kernel_kthread_stop_ret
> Connecting kernel_module_free
> Connecting kernel_module_load
> Connecting kernel_printk
> Connecting kernel_process_exit
> Connecting kernel_process_fork
> Connecting kernel_process_free
> Connecting kernel_process_wait
> Connecting kernel_sched_migrate_task
> Connecting kernel_sched_schedule
> Connecting kernel_sched_try_wakeup
> Connecting kernel_sched_wait_task
> Connecting kernel_sched_wakeup_new_task
> Connecting kernel_send_signal
> Connecting kernel_softirq_entry
> Connecting kernel_softirq_exit
> Connecting kernel_softirq_raise
> Connecting kernel_tasklet_high_entry
> Connecting kernel_tasklet_high_exit
> Connecting kernel_tasklet_low_entry
> Connecting kernel_tasklet_low_exit
> Connecting kernel_timer_itimer_expired
> Connecting kernel_timer_itimer_set
> Connecting kernel_timer_set
> Connecting kernel_timer_timeout
> Connecting kernel_timer_update_time
> Connecting kernel_vprintk
> Connecting list_file_descriptor
> Connecting list_interrupt
> Connecting list_module
> Connecting list_network_ip_interface
> Connecting list_network_ipv4_interface
> Connecting list_process_state
> Connecting list_statedump_end
> Connecting list_vm_map
> Connecting mm_handle_fault_entry
> Connecting mm_handle_fault_exit
> Connecting mm_huge_page_alloc
> Connecting mm_huge_page_free
> Connecting mm_page_alloc
> Connecting mm_page_free
> Connecting mm_wait_on_page_end
> Connecting mm_wait_on_page_start
> Connecting rcu_classic_callback
> Connecting rcu_classic_call_rcu
> Connecting rcu_classic_call_rcu_bh
> Connecting statedump_softirq_vec
> Connecting high-rate markers to tap
> Connecting fs_buffer_wait_end
> Connecting fs_buffer_wait_start
> Connecting fs_close
> Connecting fs_exec
> Connecting fs_ioctl
> Connecting fs_llseek
> Connecting fs_lseek
> Connecting fs_open
> Connecting fs_pollfd
> Connecting fs_pread64
> Connecting fs_pwrite64
> Connecting fs_read
> Connecting fs_readv
> Connecting fs_select
> Connecting fs_write
> Connecting fs_writev
> Connecting input_event
> Connecting ipc_msg_create
> Connecting ipc_sem_create
> Connecting ipc_shm_create
> Connecting kernel_arch_ipc_call
> Connecting kernel_arch_kthread_create
> Connecting kernel_arch_syscall_entry
> Connecting kernel_arch_syscall_exit
> Connecting kernel_arch_trap_entry
> Connecting kernel_arch_trap_exit
> Connecting kernel_irq_entry
> Connecting kernel_irq_exit
> Connecting kernel_kthread_stop
> Connecting kernel_kthread_stop_ret
> Connecting kernel_module_free
> Connecting kernel_module_load
> Connecting kernel_printk
> Connecting kernel_process_exit
> Connecting kernel_process_fork
> Connecting kernel_process_free
> Connecting kernel_process_wait
> Connecting kernel_sched_migrate_task
> Connecting kernel_sched_schedule
> Connecting kernel_sched_try_wakeup
> Connecting kernel_sched_wait_task
> Connecting kernel_sched_wakeup_new_task
> Connecting kernel_send_signal
> Connecting kernel_softirq_entry
> Connecting kernel_softirq_exit
> Connecting kernel_softirq_raise
> Connecting kernel_tasklet_high_entry
> Connecting kernel_tasklet_high_exit
> Connecting kernel_tasklet_low_entry
> Connecting kernel_tasklet_low_exit
> Connecting kernel_timer_itimer_expired
> Connecting kernel_timer_itimer_set
> Connecting kernel_timer_set
> Connecting kernel_timer_timeout
> Connecting kernel_timer_update_time
> Connecting kernel_vprintk
> Connecting list_file_descriptor
> Connecting list_interrupt
> Connecting list_module
> Connecting list_network_ip_interface
> Connecting list_network_ipv4_interface
> Connecting list_process_state
> Connecting list_statedump_end
> Connecting list_vm_map
> Connecting mm_handle_fault_entry
> Connecting mm_handle_fault_exit
> Connecting mm_huge_page_alloc
> Connecting mm_huge_page_free
> Connecting mm_page_alloc
> Connecting mm_page_free
> Connecting mm_wait_on_page_end
> Connecting mm_wait_on_page_start
> Connecting rcu_classic_callback
> Connecting rcu_classic_call_rcu
> Connecting rcu_classic_call_rcu_bh
> Connecting statedump_softirq_vec
> -----------------------------------------------------------------------------
> [Bryan] It's that correct result?
Yes, this output is correct. So you probably need to recompile busybox
with awk and/or sed and/or sort support.....
> -----------------------------------------------------------------------------
> # cat /proc/ltt
> marker: kernel_arch_kthread_create format: "pid %ld fn %p" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_arch_syscall_exit format: " " state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_arch_syscall_entry format: "syscall_id %d ip #p%ld" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_arch_ipc_call format: "call %u first %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_arch_trap_exit format: " " state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_arch_trap_entry format: "trap_id %ld ip #p%ld" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: statedump_softirq_vec format: "id %d address %p symbol %s" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: list_module format: "name %s state %d refcount %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: core_marker_format format: "name %s format %s" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: core_marker_format format: "name %s format %s" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: list_statedump_end format: " " state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: list_process_state format: "pid %d parent_pid %d name %s type %d mode %d submode %d status %d tgid %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: list_file_descriptor format: "filename %s pid %d fd %u" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: list_vm_map format: "pid %d start %lu end %lu flags %lu pgoff %lu inode %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: list_interrupt format: "name %s action %s irq_id %u" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: list_network_ip_interface format: "name %s address #4u%lu up %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: list_network_ipv4_interface format: "name %s address #4u%lu up %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: core_marker_id format: "name %s id %hu int #1u%zu long #1u%zu pointer #1u%zu size_t #1u%zu alignment #1u%u" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: core_marker_id format: "name %s id %hu int #1u%zu long #1u%zu pointer #1u%zu size_t #1u%zu alignment #1u%u" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: core_marker_format format: "name %s format %s" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: core_marker_id format: "name %s id %hu int #1u%zu long #1u%zu pointer #1u%zu size_t #1u%zu alignment #1u%u" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: core_marker_id format: "name %s id %hu int #1u%zu long #1u%zu pointer #1u%zu size_t #1u%zu alignment #1u%u" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_module_load format: "name %s" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_module_free format: "name %s" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_vprintk format: "loglevel %c string %s ip %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_printk format: "ip %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_timer_timeout format: "pid %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_timer_update_time format: "jiffies #8u%llu xtime_sec %ld xtime_nsec %ld walltomonotonic_sec %ld walltomonotonic_nsec %ld" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_timer_set format: "expires %lu function %p data %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_timer_itimer_set format: "which %d interval_sec %ld interval_usec %ld value_sec %ld value_usec %ld" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_timer_itimer_expired format: "pid %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_process_fork format: "parent_pid %d child_pid %d child_tgid %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_process_wait format: "pid %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_process_exit format: "pid %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_process_free format: "pid %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_send_signal format: "pid %d signal %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_sched_migrate_task format: "pid %d state %ld dest_cpu %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_sched_schedule format: "prev_pid %d next_pid %d prev_state %ld" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_sched_wakeup_new_task format: "pid %d state %ld cpu_id %u" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_sched_try_wakeup format: "pid %d state %ld cpu_id %u" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_sched_wait_task format: "pid %d state %ld" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_kthread_stop_ret format: "ret %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_kthread_stop format: "pid %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_tasklet_high_exit format: "func %p data %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_tasklet_high_entry format: "func %p data %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_tasklet_low_exit format: "func %p data %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_tasklet_low_entry format: "func %p data %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_softirq_raise format: "softirq_id %u" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_softirq_exit format: "softirq_id %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_softirq_entry format: "softirq_id %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_irq_exit format: "handled #1u%u" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_irq_entry format: "irq_id %u kernel_mode %u ip %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: mm_page_alloc format: "order %u pfn %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: mm_page_free format: "order %u pfn %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: mm_handle_fault_exit format: "res %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: mm_handle_fault_entry format: "address %lu ip #p%ld write_access %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: mm_huge_page_alloc format: "pfn %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: mm_huge_page_free format: "pfn %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: mm_wait_on_page_end format: "pfn %lu bit_nr %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: mm_wait_on_page_start format: "pfn %lu bit_nr %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_pollfd format: "fd %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_select format: "fd %d timeout #8d%lld" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_writev format: "fd %lu vlen %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_readv format: "fd %lu vlen %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_pwrite64 format: "fd %u count %zu pos %llu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_pread64 format: "fd %u count %zu pos %llu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_write format: "fd %u count %zu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_read format: "fd %u count %zu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_llseek format: "fd %u offset %lld origin %u" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_lseek format: "fd %u offset %ld origin %u" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_close format: "fd %u" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_open format: "fd %d filename %s" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_ioctl format: "fd %u cmd %u arg %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_exec format: "filename %s" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_buffer_wait_end format: "bh %p" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_buffer_wait_start format: "bh %p" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: ipc_shm_create format: "id %ld flags %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: ipc_sem_create format: "id %ld flags %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: ipc_msg_create format: "id %ld flags %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: lockdep_lock_release format: "retaddr 0x%lX lock %p nested %d" state: 0 call: 0xc0093374 probe single : 0xc00920ec
> marker: lockdep_lock_acquire format: "retaddr 0x%lX subclass %u lock %p trylock %d read %d hardirqs_off %d" state: 0 call: 0xc0093374 probe single : 0xc00920ec
> marker: lockdep_softirqs_off format: "retaddr 0x%lX" state: 0 call: 0xc0093374 probe single : 0xc00920ec
> marker: lockdep_softirqs_on format: "retaddr 0x%lX" state: 0 call: 0xc0093374 probe single : 0xc00920ec
> marker: lockdep_hardirqs_off format: "retaddr 0x%lX" state: 0 call: 0xc0093374 probe single : 0xc00920ec
> marker: lockdep_hardirqs_on format: "retaddr 0x%lX" state: 0 call: 0xc0093374 probe single : 0xc00920ec
> marker: rcu_classic_call_rcu_bh format: "func %p ip 0x%lX" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: rcu_classic_call_rcu format: "func %p ip 0x%lX" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: rcu_classic_callback format: "func %p" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: input_event format: "type %u code %u value %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> -----------------------------------------------------------------------------
> [Bryan] But the /proc/ltt is still same, it's that correct? What's the expected result?
This output is good. busybox problem, as I said above. Try each command
used by busybox to test them.
> -----------------------------------------------------------------------------
> # df
> /dev: 39564K total, 0K used, 39564K available (block size 4096)
> /sqlite_stmt_journals: 4096K total, 0K used, 4096K available (block size 4096)
> /system: 72704K total, 71680K used, 1024K available (block size 4096)
> /data: 66816K total, 37812K used, 29004K available (block size 4096)
> /cache: 67584K total, 1156K used, 66428K available (block size 4096)
> /sdcard: 991708K total, 147040K used, 844668K available (block size 4096)
> -----------------------------------------------------------------------------
> [Bryan] There is 29004K available for trace log file /data/ltt/trace1. Is that enough?
No. I would save it to /sdcard.
> -----------------------------------------------------------------------------
> # cat /proc/meminfo
> MemTotal: 79128 kB
> MemFree: 2440 kB
> Buffers: 36 kB
> Cached: 29872 kB
> SwapCached: 0 kB
> Active: 61384 kB
> Inactive: 5052 kB
> SwapTotal: 0 kB
> SwapFree: 0 kB
> Dirty: 0 kB
> Writeback: 0 kB
> AnonPages: 36552 kB
> Mapped: 24888 kB
> Slab: 3856 kB
> SReclaimable: 904 kB
> SUnreclaim: 2952 kB
> PageTables: 3660 kB
> NFS_Unstable: 0 kB
> Bounce: 0 kB
> WritebackTmp: 0 kB
> CommitLimit: 39564 kB
> Committed_AS: 1032216 kB
> VmallocTotal: 409600 kB
> VmallocUsed: 52908 kB
> VmallocChunk: 332796 kB
> -----------------------------------------------------------------------------
> [Bryan]
> There is 2440 kB free memory. Is that enough?
I think that yes. The page allocator will kick out Cached pages when
needed.
> -----------------------------------------------------------------------------
> # lttctl -n trace1 -d -l /sys/kernel/debug/ltt -t /data/ltt/trace1
> Linux Trace Toolkit Trace Control 0.55-16102008
>
> Controlling trace : trace1
>
> Linux Trace Toolkit Trace Daemon 0.55-16102008
>
> Reading from debugfs directory : /sys/kernel/debug/ltt/trace1
> Writing to trace directory : /data/ltt/trace1
> -----------------------------------------------------------------------------
> [Bryan] do some tests on target device, then check the channel and trace
> -----------------------------------------------------------------------------
> # ls -l /sys/kernel/debug/ltt/trace1
> -r-------- root root 0 1980-01-06 00:10 cpu_0
> drwxr-xr-x root root 1980-01-06 00:10 control
> # ls -l /sys/kernel/debug/ltt/trace1/control
> -r-------- root root 0 1980-01-06 00:10 network_0
> -r-------- root root 0 1980-01-06 00:10 modules_0
> -r-------- root root 0 1980-01-06 00:10 processes_0
> -r-------- root root 0 1980-01-06 00:10 interrupts_0
> -r-------- root root 0 1980-01-06 00:10 metadata_0
> # ls -l /data/ltt/trace1
> drwxrwxrwx root root 1980-01-06 00:10 control
> -rwxrwxrwx root root 29949952 1980-01-06 00:10 cpu_0
> # ls -l /data/ltt/trace1/control
> -rwxrwxrwx root root 0 1980-01-06 00:10 metadata_0
> -rwxrwxrwx root root 0 1980-01-06 00:10 interrupts_0
> -rwxrwxrwx root root 0 1980-01-06 00:10 processes_0
> -rwxrwxrwx root root 0 1980-01-06 00:10 modules_0
> -rwxrwxrwx root root 0 1980-01-06 00:10 network_0
> -----------------------------------------------------------------------------
> [Bryan] It seems that the capacity has been exhausted by cpu_0, because 29004K available.
> It's that correct that cpu_0 will grow to so huge (29M)?
29M is not huge. It is small :) Especially if the system is very busy.
> -----------------------------------------------------------------------------
> # lttctl -n trace1 -d -R
> Linux Trace Toolkit Trace Control 0.55-16102008
>
> Controlling trace : trace1
> ^Z[1] + Stopped lttctl -n trace1 -d -R
> -----------------------------------------------------------------------------
> [Bryan] hang in here...I have to break it by Ctrl+Z, What's problem?
It's waiting for lttd to release the trace channels, but I guess that
the lack of space is causing errors, so lttd keeps running.
> -----------------------------------------------------------------------------
> # ls -l /data/ltt/trace1/
> drwxrwxrwx root root 1980-01-06 00:10 control
> -rwxrwxrwx root root 29949952 1980-01-06 00:10 cpu_0
> # ls -l /data/ltt/trace1/control
> -rwxrwxrwx root root 0 1980-01-06 00:10 metadata_0
> -rwxrwxrwx root root 0 1980-01-06 00:10 interrupts_0
> -rwxrwxrwx root root 0 1980-01-06 00:10 processes_0
> -rwxrwxrwx root root 0 1980-01-06 00:10 modules_0
> -rwxrwxrwx root root 0 1980-01-06 00:10 network_0
> # ls -l /sys/kernel/debug/ltt/trace1
> -r-------- root root 0 1980-01-06 00:10 cpu_0
> drwxr-xr-x root root 1980-01-06 00:10 control
> # ls -l /sys/kernel/debug/ltt/trace1/control
> -r-------- root root 0 1980-01-06 00:10 network_0
> -r-------- root root 0 1980-01-06 00:10 modules_0
> -r-------- root root 0 1980-01-06 00:10 processes_0
> -r-------- root root 0 1980-01-06 00:10 interrupts_0
> -r-------- root root 0 1980-01-06 00:10 metadata_0
> # df
> /dev: 39564K total, 0K used, 39564K available (block size 4096)
> /sqlite_stmt_journals: 4096K total, 0K used, 4096K available (block size 4096)
> /system: 72704K total, 71680K used, 1024K available (block size 4096)
> /data: 66816K total, 66732K used, 84K available (block size 4096)
> /cache: 67584K total, 1156K used, 66428K available (block size 4096)
> /sdcard: 991708K total, 147040K used, 844668K available (block size 4096)
> -----------------------------------------------------------------------------
> [Bryan] I think the trace is incomplete.
Yes, here /data only has 84K avail. It's full.
Mathieu
> -----------------------------------------------------------------------------
>
> Please help me! Thank you very much!
>
> Best regards,
> Bryan Hsieh
>
> SPEC, FIH Co., Ltd.
> Tel:+886-3-270-0000 #2723
> Fax:+886-3-270-0001
>
>
> -----Original Message-----
> From: Mathieu Desnoyers [mailto:compudj at krystal.dyndns.org]
> Sent: Friday, July 31, 2009 11:43 PM
> To: Bryan Hsieh
> Cc: 'Ajit R'; 'ltt-dev at lists.casi.polymtl.ca'; Tommy Peng
> Subject: Re: [ltt-dev] Some LTTng 0.43 issue on ARM
>
> * Bryan Hsieh (bryanhsieh at fihspec.com) wrote:
> > Hi Ajit,
> >
> > I appreciate your reply!
> > But there is a problem here.
> >
> > My installation of Busybox is 1.12.4.
>
> It sounds like either awk or sed or not built in your busybox
> installation.
>
> Can you provide the output of
>
> cat /proc/ltt
>
> on your target system ?
>
> Check the content of ltt-armall to see which external programs it needs.
>
> Also, if you built the lttng probes as modules, you may need to load
> them explicitely.
>
> Mathieu
>
>
> >
> > I follow your suggestion to test on x86 Desktop ARM simulator, the message said that can't find /proc/ltt. I think it should be, because I didn't patch to ARM simulator Linux kernel.
> >
> > [sbox-ARM: ~/ltt-control-0.55-16102008/lttctl] > ltt-armall
> > Connecting all markers
> > cat: /proc/ltt: No such file or directory
> > Connecting high-rate markers to tap
> > cat: /proc/ltt: No such file or directory
> >
> > It should be work on my target device, but it seems not.
> > I can't find /ltt under /proc
> > Maybe it's root cause, but why there is no /ltt/proc? And why there is no " cat: /proc/ltt: No such file or directory" message in my target device during running ltt-armall?
> >
> > BTW, I mount debugfs in init.c
> > Mount("debugfs", "/sys/kernel/debug", "debugfs", 0, NULL);
> > It should not be a problem.
> >
> > -----------------------------------------------------------------------------------------------------------Another problem:
> > I command as following steps.
> > # lttctl -n trace1 -q
> > Linux Trace Toolkit Trace Control 0.55-16102008
> > Controlling trace : trace1
> >
> > # lttctl -n trace1 -r
> > Linux Trace Toolkit Trace Control 0.55-16102008
> > Controlling trace : trace1
> >
> > >>> Android shell hang in here, I have to press Ctrl+Z to force stopping - lttctl -n trace1 -r .Why? Is anything wrong?
> > # ls -l ../tmp/trace1
> > drwxrwxrwx root root 1980-01-07 06:37 control
> > -rwxrwxrwx root root 1048576 1980-01-07 06:37 cpu_0
> > # ls -l ../tmp/trace1/control
> > -rwxrwxrwx root root 0 1980-01-07 06:37 metadata_0
> > -rwxrwxrwx root root 0 1980-01-07 06:37 interrupts_0
> > -rwxrwxrwx root root 10240 1980-01-07 06:37 processes_0
> > -rwxrwxrwx root root 65536 1980-01-07 06:37 modules_0
> > -rwxrwxrwx root root 65536 1980-01-07 06:37 network_0
> >
> > The Channel in /sys/kernel/debug/ltt/trace1 is clear. Does it cause by destroy channel command?
> >
> > The metadata_0 and interrupts_0 are empty, Is that correct? I've done some key press, the interrupts_o shouldn't be empt.
> >
> > And those files almost filled by 0 except for the short header I can't make sure, it looks like incomplete.
> >
> > I hope you can help me to solve these issues, I am appreciate you so much..
> >
> > Best regards,
> > Bryan Hsieh
> >
> > SPEC, FIH Co., Ltd.
> > Tel:+886-3-270-0000 #2723
> > Fax:+886-3-270-0001
> >
> >
> > -----Original Message-----
> > From: Ajit R [mailto:ajitrzid at yahoo.com]
> > Sent: Friday, July 31, 2009 6:17 PM
> > To: 'ltt-dev at lists.casi.polymtl.ca'; Bryan Hsieh
> > Cc: Tommy Peng
> > Subject: Re: [ltt-dev] Some LTTng 0.43 issue on ARM
> >
> >
> > Hi Bryan,
> > My replies with the [Ajit] tag.
> > LTT Gods correct me if I am wrong.
> >
> > Thanks
> > Ajit
> >
> > --- On Fri, 7/31/09, Bryan Hsieh <bryanhsieh at fihspec.com> wrote:
> >
> > > From: Bryan Hsieh <bryanhsieh at fihspec.com>
> > > Subject: [ltt-dev] Some LTTng 0.43 issue on ARM
> > > To: "'ltt-dev at lists.casi.polymtl.ca'" <ltt-dev at lists.casi.polymtl.ca>
> > > Cc: "Tommy Peng" <tommypeng at fihspec.com>
> > > Date: Friday, July 31, 2009, 12:19 PM
> >
> > > Hi Mathieu,
> > > all,
> > >
> > >
> > >
> > >
> > > I've ported
> > > LTTng 0.43 on Android successfully,
> > > but I am wondering if the result is correct, please help me
> > > to verify, thank
> > > you very much!
> > >
> > >
> > >
> > >
> > > My Traced
> > > machine:
> > >
> > > ARM11 (ARM1136J(F)-S) / Linux
> > > 2.6..27
> > > (Android 1.5)
> > >
> > > My
> > > visualization machine:
> > >
> > > X86 / Sun xVM Virtual Box (Unbuntu
> > > 8.10)
> > >
> > >
> > >
> > > LTTng: 0.43
> > >
> > > LTTV:
> > > 0.11.2
> > >
> > > LTT Control: 0.55
> > >
> > >
> > >
> > >
> > > I
> > > connect to Android
> > > shell by ADB tool after Android bring up, then I command as
> > > following step, the
> > > command line options are different with LTTng manual,
> > > because the example in manual
> > > is for old version.
> > >
> > >
> > >
> > >
> > > # ltt-armall
> > >
> > [ Ajit ]
> > ltt-armall is a shell script. It connects + arms the markers in the
> > kernel. So depending on your kernel + LTT version it would write some
> > "text" into the /proc/ltt directory or a "1" into files in
> > the debugfs directory.
> > Make sure, the binaries for running the script are available in your target environment. For eg. awk, grep, sed etc. Busybox versions might have their limitations. If they do not work, you could do equivalent things on your Desktop and then write a simple script, doing the same on your target.
> > Once you run "ltt-armall", you should see some debug statements in your "dmesg" like "connect" or "arm" etc.
> >
> > >
> > > Connecting all
> > > markers
> > >
> > > Connecting high-rate
> > > markers to tap
> > >
> > > # lttctl
> > > -n trace1 -d -l /sys/kernel/debug/ltt -t
> > > /system/traces/trace1
> > >
> > [Ajit]
> > Not sure if your "-l" option is right. you have to point it to the place where you mounted your debugfs.
> >
> > >
> > > Linux Trace Toolkit
> > > Trace Control 0.55-16102008
> > >
> > >
> > >
> > >
> > > Controlling trace :
> > > trace1
> > >
> > >
> > >
> > >
> > > Linux Trace Toolkit
> > > Trace Daemon 0.55-16102008
> > >
> > >
> > >
> > >
> > > Reading from debugfs
> > > directory : /sys/kernel/debug/ltt/trace1
> > >
> > >
> > > Writing to trace
> > > directory : /system/traces/trace1
> > >
> > >
> > >
> > >
> > > It
> > > seems the LTT control
> > > and daemon work fine. I run some scenarios for 1 minute on
> > > Android, but the trace
> > > files are still empty (all files length are
> > > 0)
> > >
> > > What's
> > > wrong? Is
> > > there any step I missed? Or the LTT control and daemon
> > > don't work?
> > >
> > >
> > >
> > >
> > > # ls
> > > -l ../../sys/kernel/debug/ltt/
> > >
> > >
> > >
> > > drwxr-xr-x
> > > root
> > > root
> > > 1980-01-07 01:13 trace1
> > >
> > > # ls
> > > -l
> > > ../../sys/kernel/debug/ltt/trace1
> > >
> > >
> > > -r--------
> > > root
> > > root
> > > 0
> > > 1980-01-07 01:13 cpu_0
> > >
> >
> > [Ajit]
> > If you have connected and armed it right, you should see the cpu_0
> > file increasing in size. After sometime, disconnect by running the
> > ltt-control -n whatever-name -R
> > and then you will see files in the control directory (network, process, etc ) grown in size. Tar and move the trace directory to your host and do the needful.
> >
> >
> > > drwxr-xr-x
> > > root
> > > root
> > > 1980-01-07 01:13 control
> > >
> > > # ls
> > > -l
> > > ../../sys/kernel/debug/ltt/trace1/control
> > >
> > >
> > > -r--------
> > > root
> > > root
> > > 0
> > > 1980-01-07 01:13 network_0
> > >
> > > -r--------
> > > root
> > > root
> > > 0
> > > 1980-01-07 01:13 modules_0
> > >
> > > -r--------
> > > root
> > > root
> > > 0
> > > 1980-01-07 01:13 processes_0
> > >
> > > -r--------
> > > root
> > > root
> > > 0
> > > 1980-01-07 01:13 interrupts_0
> > >
> > > -r--------
> > > root
> > > root
> > > 0 1980-01-07 01:13 metadata_0
> > >
> > > # ls -l
> > > .../traces/
> > >
> > >
> > >
> > > drwxrwxrwx
> > > root
> > > root
> > > 1980-01-07 01:13 trace1
> > >
> > > # ls -l
> > > .../traces/trace1
> > >
> > > drwxrwxrwx
> > > root
> > > root
> > > 1980-01-07 01:13 control
> > >
> > > -rwxrwxrwx
> > > root
> > > root
> > > 0
> > > 1980-01-07 01:13 cpu_0
> > >
> > > # ls -l
> > > .../traces/trace1/control
> > >
> > >
> > >
> > > -rwxrwxrwx
> > > root
> > > root
> > > 0
> > > 1980-01-07 01:13 metadata_0
> > >
> > > -rwxrwxrwx
> > > root
> > > root
> > > 0
> > > 1980-01-07 01:13 interrupts_0
> > >
> > > -rwxrwxrwx
> > > root
> > > root
> > > 0
> > > 1980-01-07 01:13 processes_0
> > >
> > > -rwxrwxrwx
> > > root
> > > root
> > > 0
> > > 1980-01-07 01:13 modules_0
> > >
> > > -rwxrwxrwx
> > > root
> > > root
> > > 0
> > > 1980-01-07 01:13 network_0
> > >
> > >
> > >
> > >
> > > BTW,
> > > I would like to
> > > know what is the channel concept for LTTng? What's
> > > the relationship between
> > > trace and channel? What's the normal, flight recorder
> > > and hybrid mode?
> > >
> > > Which
> > > files should I get
> > > for LTTV on another visualization
> > > machine?
> > >
> > > So far, I
> > > can't find related document or the
> > > information isn't enough for me.
> > >
> > >
> > >
> > >
> > > Thanks for your
> > > help!
> > >
> > >
> > >
> > >
> > > Best
> > > regards,
> > >
> > > Bryan Hsieh
> > >
> > > SPEC, FIH
> > > Co., Ltd.
> > >
> > > Tel:+886-3-270-0000 #2723
> > >
> > > Fax:+886-3-270-0001
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> >
> > >
> > > _______________________________________________
> > > ltt-dev mailing list
> > > ltt-dev at lists.casi.polymtl.ca
> > > http://lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev
> > >
> >
> >
> >
> > This e-mail message together with any attachments thereto (if any) is confidential, protected under an enforceable non-disclosure agreement,
> > intended only for the use of the named recipient(s) above and may contain information that is privileged, belonging to professional work products
> > or exempt from disclosure under applicable laws. Any unauthorized review, use, copying, disclosure, or distribution of any information contained
> > in or attached to this transmission is STRICTLY PROHIBITED and may be against the laws. If you have received this message in error, or are not
> > the named recipient(s), please immediately notify the sender by e-mail or telephone at IT support hotline 886-3-2700000 EXT 1333 and delete this
> > e-mail message and any attached documentation from your computer. Receipt by anyone other than the intended recipient(s) is not a waiver of any
> > attorney-client or work product privilege. Thank you!
> >
> >
> > _______________________________________________
> > ltt-dev mailing list
> > ltt-dev at lists.casi.polymtl.ca
> > http://lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev
> >
>
> --
> Mathieu Desnoyers
> OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
>
> This e-mail message together with any attachments thereto (if any) is confidential, protected under an enforceable non-disclosure agreement,
> intended only for the use of the named recipient(s) above and may contain information that is privileged, belonging to professional work products
> or exempt from disclosure under applicable laws. Any unauthorized review, use, copying, disclosure, or distribution of any information contained
> in or attached to this transmission is STRICTLY PROHIBITED and may be against the laws. If you have received this message in error, or are not
> the named recipient(s), please immediately notify the sender by e-mail or telephone at IT support hotline 886-3-2700000 EXT 1333 and delete this
> e-mail message and any attached documentation from your computer. Receipt by anyone other than the intended recipient(s) is not a waiver of any
> attorney-client or work product privilege. Thank you!
Content-Description: busybox version and command query.txt
> # grep --help
> BusyBox v1.12.4 (2009-07-29 23:17:25 CST) multi-call binary
>
> Usage: grep [-HhrilLnqvsoeFEABC] PATTERN [FILEs...]
>
> Search for PATTERN in each FILE or standard input
>
> Options:
> -H Prefix output lines with filename where match was found
> -h Suppress the prefixing filename on output
> -r Recurse subdirectories
> -i Ignore case distinctions
> -l List names of files that match
> -L List names of files that do not match
> -n Print line number with output lines
> -q Quiet. Return 0 if PATTERN is found, 1 otherwise
> -v Select non-matching lines
> -s Suppress file open/read error messages
> -c Only print count of matching lines
> -o Show only the part of a line that matches PATTERN
> -m MAX Match up to MAX times per file
> -F PATTERN is a set of newline-separated strings
> -E PATTERN is an extended regular expression
> -e PTRN Pattern to match
> -f FILE Read pattern from file
> -A Print NUM lines of trailing context
> -B Print NUM lines of leading context
> -C Print NUM lines of output context
>
> # awk --help
> BusyBox v1.12.4 (2009-07-29 23:17:25 CST) multi-call binary
>
> Usage: awk [OPTION]... [program-text] [FILE...]
>
> Options:
> -v var=val Set variable
> -F sep Use sep as field separator
> -f file Read program from file
>
> # sort --help
> BusyBox v1.12.4 (2009-07-29 23:17:25 CST) multi-call binary
>
> Usage: sort [-nrugMcszbdfimSTokt] [-o FILE] [-k start[.offset][opts][,end[.offset][opts]] [-t CHAR] [FILE]...
>
> Sort lines of text
>
> Options:
> -b Ignore leading blanks
> -c Check whether input is sorted
> -d Dictionary order (blank or alphanumeric only)
> -f Ignore case
> -g General numerical sort
> -i Ignore unprintable characters
> -k Sort key
> -M Sort month
> -n Sort numbers
> -o Output to file
> -k Sort by key
> -t CHAR Key separator
> -r Reverse sort order
> -s Stable (don't sort ties alphabetically)
> -u Suppress duplicate lines
> -z Lines are terminated by NUL, not newline
> -mST Ignored for GNU compatibility
>
> # cat --help
> unknown option -- -usage: cat [-beflnstv] [-] [file ...]
> #
>
Content-Description: LTTng Execution Log.txt
> # ltt-armall-1
> Connecting all markers
> Connecting fs_buffer_wait_end
> Connecting fs_buffer_wait_start
> Connecting fs_close
> Connecting fs_exec
> Connecting fs_ioctl
> Connecting fs_llseek
> Connecting fs_lseek
> Connecting fs_open
> Connecting fs_pollfd
> Connecting fs_pread64
> Connecting fs_pwrite64
> Connecting fs_read
> Connecting fs_readv
> Connecting fs_select
> Connecting fs_write
> Connecting fs_writev
> Connecting input_event
> Connecting ipc_msg_create
> Connecting ipc_sem_create
> Connecting ipc_shm_create
> Connecting kernel_arch_ipc_call
> Connecting kernel_arch_kthread_create
> Connecting kernel_arch_syscall_entry
> Connecting kernel_arch_syscall_exit
> Connecting kernel_arch_trap_entry
> Connecting kernel_arch_trap_exit
> Connecting kernel_irq_entry
> Connecting kernel_irq_exit
> Connecting kernel_kthread_stop
> Connecting kernel_kthread_stop_ret
> Connecting kernel_module_free
> Connecting kernel_module_load
> Connecting kernel_printk
> Connecting kernel_process_exit
> Connecting kernel_process_fork
> Connecting kernel_process_free
> Connecting kernel_process_wait
> Connecting kernel_sched_migrate_task
> Connecting kernel_sched_schedule
> Connecting kernel_sched_try_wakeup
> Connecting kernel_sched_wait_task
> Connecting kernel_sched_wakeup_new_task
> Connecting kernel_send_signal
> Connecting kernel_softirq_entry
> Connecting kernel_softirq_exit
> Connecting kernel_softirq_raise
> Connecting kernel_tasklet_high_entry
> Connecting kernel_tasklet_high_exit
> Connecting kernel_tasklet_low_entry
> Connecting kernel_tasklet_low_exit
> Connecting kernel_timer_itimer_expired
> Connecting kernel_timer_itimer_set
> Connecting kernel_timer_set
> Connecting kernel_timer_timeout
> Connecting kernel_timer_update_time
> Connecting kernel_vprintk
> Connecting list_file_descriptor
> Connecting list_interrupt
> Connecting list_module
> Connecting list_network_ip_interface
> Connecting list_network_ipv4_interface
> Connecting list_process_state
> Connecting list_statedump_end
> Connecting list_vm_map
> Connecting mm_handle_fault_entry
> Connecting mm_handle_fault_exit
> Connecting mm_huge_page_alloc
> Connecting mm_huge_page_free
> Connecting mm_page_alloc
> Connecting mm_page_free
> Connecting mm_wait_on_page_end
> Connecting mm_wait_on_page_start
> Connecting rcu_classic_callback
> Connecting rcu_classic_call_rcu
> Connecting rcu_classic_call_rcu_bh
> Connecting statedump_softirq_vec
> Connecting high-rate markers to tap
> Connecting fs_buffer_wait_end
> Connecting fs_buffer_wait_start
> Connecting fs_close
> Connecting fs_exec
> Connecting fs_ioctl
> Connecting fs_llseek
> Connecting fs_lseek
> Connecting fs_open
> Connecting fs_pollfd
> Connecting fs_pread64
> Connecting fs_pwrite64
> Connecting fs_read
> Connecting fs_readv
> Connecting fs_select
> Connecting fs_write
> Connecting fs_writev
> Connecting input_event
> Connecting ipc_msg_create
> Connecting ipc_sem_create
> Connecting ipc_shm_create
> Connecting kernel_arch_ipc_call
> Connecting kernel_arch_kthread_create
> Connecting kernel_arch_syscall_entry
> Connecting kernel_arch_syscall_exit
> Connecting kernel_arch_trap_entry
> Connecting kernel_arch_trap_exit
> Connecting kernel_irq_entry
> Connecting kernel_irq_exit
> Connecting kernel_kthread_stop
> Connecting kernel_kthread_stop_ret
> Connecting kernel_module_free
> Connecting kernel_module_load
> Connecting kernel_printk
> Connecting kernel_process_exit
> Connecting kernel_process_fork
> Connecting kernel_process_free
> Connecting kernel_process_wait
> Connecting kernel_sched_migrate_task
> Connecting kernel_sched_schedule
> Connecting kernel_sched_try_wakeup
> Connecting kernel_sched_wait_task
> Connecting kernel_sched_wakeup_new_task
> Connecting kernel_send_signal
> Connecting kernel_softirq_entry
> Connecting kernel_softirq_exit
> Connecting kernel_softirq_raise
> Connecting kernel_tasklet_high_entry
> Connecting kernel_tasklet_high_exit
> Connecting kernel_tasklet_low_entry
> Connecting kernel_tasklet_low_exit
> Connecting kernel_timer_itimer_expired
> Connecting kernel_timer_itimer_set
> Connecting kernel_timer_set
> Connecting kernel_timer_timeout
> Connecting kernel_timer_update_time
> Connecting kernel_vprintk
> Connecting list_file_descriptor
> Connecting list_interrupt
> Connecting list_module
> Connecting list_network_ip_interface
> Connecting list_network_ipv4_interface
> Connecting list_process_state
> Connecting list_statedump_end
> Connecting list_vm_map
> Connecting mm_handle_fault_entry
> Connecting mm_handle_fault_exit
> Connecting mm_huge_page_alloc
> Connecting mm_huge_page_free
> Connecting mm_page_alloc
> Connecting mm_page_free
> Connecting mm_wait_on_page_end
> Connecting mm_wait_on_page_start
> Connecting rcu_classic_callback
> Connecting rcu_classic_call_rcu
> Connecting rcu_classic_call_rcu_bh
> Connecting statedump_softirq_vec
>
> # cat /proc/ltt
> marker: kernel_arch_kthread_create format: "pid %ld fn %p" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_arch_syscall_exit format: " " state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_arch_syscall_entry format: "syscall_id %d ip #p%ld" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_arch_ipc_call format: "call %u first %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_arch_trap_exit format: " " state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_arch_trap_entry format: "trap_id %ld ip #p%ld" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: statedump_softirq_vec format: "id %d address %p symbol %s" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: list_module format: "name %s state %d refcount %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: core_marker_format format: "name %s format %s" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: core_marker_format format: "name %s format %s" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: list_statedump_end format: " " state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: list_process_state format: "pid %d parent_pid %d name %s type %d mode %d submode %d status %d tgid %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: list_file_descriptor format: "filename %s pid %d fd %u" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: list_vm_map format: "pid %d start %lu end %lu flags %lu pgoff %lu inode %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: list_interrupt format: "name %s action %s irq_id %u" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: list_network_ip_interface format: "name %s address #4u%lu up %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: list_network_ipv4_interface format: "name %s address #4u%lu up %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: core_marker_id format: "name %s id %hu int #1u%zu long #1u%zu pointer #1u%zu size_t #1u%zu alignment #1u%u" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: core_marker_id format: "name %s id %hu int #1u%zu long #1u%zu pointer #1u%zu size_t #1u%zu alignment #1u%u" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: core_marker_format format: "name %s format %s" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: core_marker_id format: "name %s id %hu int #1u%zu long #1u%zu pointer #1u%zu size_t #1u%zu alignment #1u%u" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: core_marker_id format: "name %s id %hu int #1u%zu long #1u%zu pointer #1u%zu size_t #1u%zu alignment #1u%u" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_module_load format: "name %s" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_module_free format: "name %s" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_vprintk format: "loglevel %c string %s ip %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_printk format: "ip %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_timer_timeout format: "pid %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_timer_update_time format: "jiffies #8u%llu xtime_sec %ld xtime_nsec %ld walltomonotonic_sec %ld walltomonotonic_nsec %ld" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_timer_set format: "expires %lu function %p data %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_timer_itimer_set format: "which %d interval_sec %ld interval_usec %ld value_sec %ld value_usec %ld" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_timer_itimer_expired format: "pid %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_process_fork format: "parent_pid %d child_pid %d child_tgid %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_process_wait format: "pid %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_process_exit format: "pid %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_process_free format: "pid %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_send_signal format: "pid %d signal %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_sched_migrate_task format: "pid %d state %ld dest_cpu %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_sched_schedule format: "prev_pid %d next_pid %d prev_state %ld" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_sched_wakeup_new_task format: "pid %d state %ld cpu_id %u" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_sched_try_wakeup format: "pid %d state %ld cpu_id %u" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_sched_wait_task format: "pid %d state %ld" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_kthread_stop_ret format: "ret %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_kthread_stop format: "pid %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_tasklet_high_exit format: "func %p data %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_tasklet_high_entry format: "func %p data %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_tasklet_low_exit format: "func %p data %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_tasklet_low_entry format: "func %p data %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_softirq_raise format: "softirq_id %u" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_softirq_exit format: "softirq_id %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_softirq_entry format: "softirq_id %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_irq_exit format: "handled #1u%u" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: kernel_irq_entry format: "irq_id %u kernel_mode %u ip %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: mm_page_alloc format: "order %u pfn %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: mm_page_free format: "order %u pfn %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: mm_handle_fault_exit format: "res %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: mm_handle_fault_entry format: "address %lu ip #p%ld write_access %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: mm_huge_page_alloc format: "pfn %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: mm_huge_page_free format: "pfn %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: mm_wait_on_page_end format: "pfn %lu bit_nr %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: mm_wait_on_page_start format: "pfn %lu bit_nr %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_pollfd format: "fd %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_select format: "fd %d timeout #8d%lld" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_writev format: "fd %lu vlen %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_readv format: "fd %lu vlen %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_pwrite64 format: "fd %u count %zu pos %llu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_pread64 format: "fd %u count %zu pos %llu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_write format: "fd %u count %zu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_read format: "fd %u count %zu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_llseek format: "fd %u offset %lld origin %u" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_lseek format: "fd %u offset %ld origin %u" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_close format: "fd %u" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_open format: "fd %d filename %s" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_ioctl format: "fd %u cmd %u arg %lu" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_exec format: "filename %s" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_buffer_wait_end format: "bh %p" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: fs_buffer_wait_start format: "bh %p" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: ipc_shm_create format: "id %ld flags %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: ipc_sem_create format: "id %ld flags %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: ipc_msg_create format: "id %ld flags %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: lockdep_lock_release format: "retaddr 0x%lX lock %p nested %d" state: 0 call: 0xc0093374 probe single : 0xc00920ec
> marker: lockdep_lock_acquire format: "retaddr 0x%lX subclass %u lock %p trylock %d read %d hardirqs_off %d" state: 0 call: 0xc0093374 probe single : 0xc00920ec
> marker: lockdep_softirqs_off format: "retaddr 0x%lX" state: 0 call: 0xc0093374 probe single : 0xc00920ec
> marker: lockdep_softirqs_on format: "retaddr 0x%lX" state: 0 call: 0xc0093374 probe single : 0xc00920ec
> marker: lockdep_hardirqs_off format: "retaddr 0x%lX" state: 0 call: 0xc0093374 probe single : 0xc00920ec
> marker: lockdep_hardirqs_on format: "retaddr 0x%lX" state: 0 call: 0xc0093374 probe single : 0xc00920ec
> marker: rcu_classic_call_rcu_bh format: "func %p ip 0x%lX" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: rcu_classic_call_rcu format: "func %p ip 0x%lX" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: rcu_classic_callback format: "func %p" state: 1 call: 0xc0093374 probe single : 0xc015d504
> marker: input_event format: "type %u code %u value %d" state: 1 call: 0xc0093374 probe single : 0xc015d504
>
> [Bryan] Maybe caused by privilege, the the /proc/ltt do not be replaced with content below. It's that will be a problem?
> connect statedump_softirq_vec ltt_tap_marker dynami
>
> # df
> /dev: 39564K total, 0K used, 39564K available (block size 4096)
> /sqlite_stmt_journals: 4096K total, 0K used, 4096K available (block size 4096)
> /system: 72704K total, 71680K used, 1024K available (block size 4096)
> /data: 66816K total, 37812K used, 29004K available (block size 4096)
> /cache: 67584K total, 1156K used, 66428K available (block size 4096)
> /sdcard: 991708K total, 147040K used, 844668K available (block size 4096)
>
> # lttctl -n trace1 -d -l /sys/kernel/debug/ltt -t /data/ltt/trace1
> Linux Trace Toolkit Trace Control 0.55-16102008
>
> Controlling trace : trace1
>
> Linux Trace Toolkit Trace Daemon 0.55-16102008
>
> Reading from debugfs directory : /sys/kernel/debug/ltt/trace1
> Writing to trace directory : /data/ltt/trace1
>
> [Bryan] there is 29004K available for /data/ltt/trace1, is that enough?
> [Bryan] do some tests on target device, then check the channel and trace
>
> # ls -l /sys/kernel/debug/ltt/trace1
> -r-------- root root 0 1980-01-06 00:10 cpu_0
> drwxr-xr-x root root 1980-01-06 00:10 control
> # ls -l /sys/kernel/debug/ltt/trace1/control
> -r-------- root root 0 1980-01-06 00:10 network_0
> -r-------- root root 0 1980-01-06 00:10 modules_0
> -r-------- root root 0 1980-01-06 00:10 processes_0
> -r-------- root root 0 1980-01-06 00:10 interrupts_0
> -r-------- root root 0 1980-01-06 00:10 metadata_0
> # ls -l /data/ltt/trace1
> drwxrwxrwx root root 1980-01-06 00:10 control
> -rwxrwxrwx root root 29949952 1980-01-06 00:10 cpu_0
> # ls -l /data/ltt/trace1/control
> -rwxrwxrwx root root 0 1980-01-06 00:10 metadata_0
> -rwxrwxrwx root root 0 1980-01-06 00:10 interrupts_0
> -rwxrwxrwx root root 0 1980-01-06 00:10 processes_0
> -rwxrwxrwx root root 0 1980-01-06 00:10 modules_0
> -rwxrwxrwx root root 0 1980-01-06 00:10 network_0
>
> [Bryan] It seems that the space has been exhausted by cpu_0, it's that correct that cpu_0 will grow to so huge(29M)?
> How much file space should I prepare for LTTng?
>
> # lttctl -n trace1 -d -R
> Linux Trace Toolkit Trace Control 0.55-16102008
>
> Controlling trace : trace1
>
> [Bryan] hang in here...I have to break it by Ctrl+Z, What's problem?
>
> ^Z[1] + Stopped lttctl -n trace1 -d -R
> # ls -l /data/ltt/trace1/
> drwxrwxrwx root root 1980-01-06 00:10 control
> -rwxrwxrwx root root 29949952 1980-01-06 00:10 cpu_0
> # ls -l /data/ltt/trace1/control
> -rwxrwxrwx root root 0 1980-01-06 00:10 metadata_0
> -rwxrwxrwx root root 0 1980-01-06 00:10 interrupts_0
> -rwxrwxrwx root root 0 1980-01-06 00:10 processes_0
> -rwxrwxrwx root root 0 1980-01-06 00:10 modules_0
> -rwxrwxrwx root root 0 1980-01-06 00:10 network_0
> # ls -l /sys/kernel/debug/ltt/trace1
> -r-------- root root 0 1980-01-06 00:10 cpu_0
> drwxr-xr-x root root 1980-01-06 00:10 control
> # ls -l /sys/kernel/debug/ltt/trace1/control
> -r-------- root root 0 1980-01-06 00:10 network_0
> -r-------- root root 0 1980-01-06 00:10 modules_0
> -r-------- root root 0 1980-01-06 00:10 processes_0
> -r-------- root root 0 1980-01-06 00:10 interrupts_0
> -r-------- root root 0 1980-01-06 00:10 metadata_0
> # df
> /dev: 39564K total, 0K used, 39564K available (block size 4096)
> /sqlite_stmt_journals: 4096K total, 0K used, 4096K available (block size 4096)
> /system: 72704K total, 71680K used, 1024K available (block size 4096)
> /data: 66816K total, 66732K used, 84K available (block size 4096)
> /cache: 67584K total, 1156K used, 66428K available (block size 4096)
> /sdcard: 991708K total, 147040K used, 844668K available (block size 4096)
>
>
>
Content-Description: LTTng Kernel Config.txt
>
> [ ] Kernel->user space relay support (formerly relayfs)
> [*] Activate tracepoints x x
> [*] Activate markers x x
> Linux Trace Toolkit --->
> <*> Compile lttng tracing probes x x
> [*] LTTng fine-grained timestamping x x
> [*] Linux Trace Toolkit Instrumentation Support x x
> <*> Linux Trace Toolkit High-speed Lockless Data Relay x x
> <*> Linux Trace Toolkit Lock-Protected Data Relay x x
> [*] Debug check for random access in ltt relay buffers x x
> <*> Linux Trace Toolkit Serializer x x
> <*> Linux Trace Toolkit Marker Control x x
> <*> Linux Trace Toolkit Tracer x x
> [*] Align Linux Trace Toolkit Traces x x
> [*] Add event size field to LTT events for tracer debugging x x
> [*] Support trace extraction from crash dump x x
> <*> Linux Trace Toolkit Netlink Controller x x
> <*> Linux Trace Toolkit State Dump
--
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
More information about the lttng-dev
mailing list