[ltt-dev] ltt trace files are empty
Mathieu Desnoyers
compudj at krystal.dyndns.org
Thu Apr 21 10:41:07 EDT 2011
* Dhirender Baliyan (dhir157 at gmail.com) wrote:
> Hi Mathieu,
>
> Thanks very much for helping us.
> Now i am getting the traces, The only thing now is when i make comparison of
> x86 traces and mips traces, I am not getting traces for the following marker
> on mips
>
> ** (process:102): WARNING **: No channel for marker named
> syscall_state.sys_call_table found
The syscall table is provided by the ltt-statedump kernel module by
architecture-specific code that iterates on the syscalls to dump their
names into the trace. So you just won't have the mapping from syscall
number to their handler name in the trace.
>
> ** (process:102): WARNING **: No channel for marker named
> kprobe_state.kprobe_table found
If CONFIG_KPROBE is not set in your kernel, you will get that. It means
you cannot connect dynamic probe points in your system with the LTTng
kprobe interface.
>
> ** (process:102): WARNING **: No marker named userspace.function_entry found
>
> ** (process:102): WARNING **: No marker named userspace.function_exit found
Don't worry about this one: it's just there for older
-finstrument-functions instrumentation for userspace with old LTTng
versions.
Thanks,
Mathieu
>
> can you tell me why these warning is coming.
>
> Thanks
> Dhirender
>
>
> On Tue, Apr 19, 2011 at 7:01 AM, Mathieu Desnoyers <
> compudj at krystal.dyndns.org> wrote:
>
> > A lot of stabilisation/testing work has been done between versions 0.88
> > and 0.170. I would recommend to backport something starting around the
> > 0.170 era. Some commits that are related to your warning there (taken
> > from the compatibility list):
> >
> > - LTTng 0.91 optimizes the write to page function (data copy to the
> > buffers).
> > - LTTng 0.103 integrates a patch to cache the page_address lookup.
> > - LTTng 0.110 moves to a pointer array instead of a linked list to
> > manage the buffer pages.
> >
> > Thanks,
> >
> > Mathieu
> >
> > * Dhirender Baliyan (dhir157 at gmail.com) wrote:
> > > Hi Mathieu,
> > >
> > > when i do dmesg, this crash is coming.
> > > <7>[4294720.204000] LTT state dump begin
> > > <7>[4294720.204000] LTT state dump thread start
> > > <7>[4294720.208000] LTT state dump end
> > > <4>[4294737.645000] Forward random access detected in ltt_relay.
> > Iterations
> > > 81, offset 1048576, orig iter->off 716800, iter->off 1048576 diff_offset
> > > 331776.
> > > <4>[4294737.659000] ------------[ cut here ]------------
> > > <4>[4294737.663000] WARNING: at ltt/ltt-relay-alloc.c:491
> > > ltt_relay_cache_page+0x194/0x1d8()
> > > <6>[4294737.671000] Modules linked in:
> > > <6>[4294737.671000] Call Trace:
> > > <6>[4294737.671000] [<8001ae2c>] dump_stack+0x8/0x34
> > > <6>[4294737.672000] [<8003548c>] warn_on_slowpath+0x60/0x88
> > > <6>[4294737.672000] [<80192298>] ltt_relay_cache_page+0x194/0x1d8
> > > <6>[4294737.672000] [<80190740>] ltt_relay_file_splice_read+0x118/0x270
> > > <6>[4294737.672000] [<800bbbbc>] sys_splice+0x1b8/0x31c
> > > <6>[4294737.672000] [<80009fc4>] stack_done+0x20/0x3c
> > >
> > >
> > > On Mon, Apr 18, 2011 at 5:39 PM, Dhirender Baliyan <dhir157 at gmail.com
> > >wrote:
> > >
> > > > Hi Mathieu,
> > > >
> > > > yes it is 32-bit,
> > > >
> > > > but the below changes were already there.
> > > > we are using NFS filesystem, I have tried it with flash also .. but
> > still
> > > > the files are empty.
> > > > your help is appricated
> > > >
> > > >
> > > > On Mon, Apr 18, 2011 at 4:33 PM, Mathieu Desnoyers <
> > > > compudj at krystal.dyndns.org> wrote:
> > > >
> > > >> My guess is that you are using a 32-bit userland, right ?
> > > >>
> > > >> Try backporting this patch from mainline first, it fixes the splice()
> > > >> system call for MIPS. By reading the commit date, it looks like it
> > came
> > > >> in during the 2.6.28 development cycle, so I'm not sure it made its
> > way
> > > >> into 2.6.28.
> > > >>
> > > >> commit 08d30879a6d3ce399f48def7c494853b5b5f2062
> > > >> Author: Ralf Baechle <ralf at linux-mips.org>
> > > >> Date: Wed Dec 3 16:33:17 2008 +0000
> > > >>
> > > >> MIPS: o32: Fix number of arguments to splice(2).
> > > >>
> > > >> The syscall code was assuming splice only takes 4 arguments so no
> > stack
> > > >> arguments were being copied from the userspace stack to the kernel
> > > >> stack.
> > > >> As the result splice was likely to fail with EINVAL.
> > > >>
> > > >> Signed-off-by: Ralf Baechle <ralf at linux-mips.org>
> > > >>
> > > >> diff --git a/arch/mips/kernel/scall32-o32.S
> > > >> b/arch/mips/kernel/scall32-o32.S
> > > >> index 34a4dbd..d0916a5 100644
> > > >> --- a/arch/mips/kernel/scall32-o32.S
> > > >> +++ b/arch/mips/kernel/scall32-o32.S
> > > >> @@ -624,7 +624,7 @@ einval: li v0, -ENOSYS
> > > >> sys sys_pselect6 6
> > > >> sys sys_ppoll 5
> > > >> sys sys_unshare 1
> > > >> - sys sys_splice 4
> > > >> + sys sys_splice 6
> > > >> sys sys_sync_file_range 7 /* 4305 */
> > > >> sys sys_tee 4
> > > >> sys sys_vmsplice 4
> > > >>
> > > >>
> > > >> * Dhirender Baliyan (dhir157 at gmail.com) wrote:
> > > >> > Hi mathieu,
> > > >> >
> > > >> > I understand this is the older version of lttng but when i tried it
> > on
> > > >> x86 ,
> > > >> > it is working.
> > > >> >
> > > >> > on x86 it is showing the traces but on mips the trace files are
> > empty.
> > > >> >
> > > >> >
> > > >> > Regards
> > > >> > Dhirender
> > > >> >
> > > >> >
> > > >> >
> > > >> > On Thu, Apr 14, 2011 at 2:43 AM, Mathieu Desnoyers <
> > > >> > compudj at krystal.dyndns.org> wrote:
> > > >> >
> > > >> > > * Dhirender Baliyan (dhir157 at gmail.com) wrote:
> > > >> > > > Hi Mathieu,
> > > >> > > >
> > > >> > > > I am using
> > > >> > > > 1) kernel 2.6.28.9( we have applied patches 2.6.28.4 to
> > 2.6.28.9)
> > > >> > > > 2)lttng -- patch-2.6.28.4-lttng-0.88
> > > >> > > > 2)lttctl 0.64
> > > >> > > > 3) lttv-0.12.10
> > > >> > > >
> > > >> > > > actually in our 2.6.28.9 kernel there is no support for kprobes
> > for
> > > >> MIPS.
> > > >> > > > hence in make menuconfig kprobe option is not coming under
> > lttng.
> > > >> > > >
> > > >> > > > would no kprobe support cause some problem ?
> > > >> > >
> > > >> > > No, I don't think it would. But we are talking about very very old
> > > >> LTTng
> > > >> > > versions here. It might not hurt to try a newer one. ;)
> > > >> > >
> > > >> > > Good luck,
> > > >> > >
> > > >> > > Mathieu
> > > >> > >
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > > On Wed, Apr 13, 2011 at 8:10 PM, Mathieu Desnoyers <
> > > >> > > > compudj at krystal.dyndns.org> wrote:
> > > >> > > >
> > > >> > > > > * Dhirender Baliyan (dhir157 at gmail.com) wrote:
> > > >> > > > > [...]
> > > >> > > > > > # lttctl -D trace1
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > > Linux Trace Toolkit Trace Control 0.64-14012009
> > > >> > > > > >
> > > >> > > > > > Controlling trace : trace1
> > > >> > > > > >
> > > >> > > > > > lttctl: Pausing trace
> > > >> > > > > > lttctl: Destroying trace
> > > >> > > > > > #ltt-disarmall
> > > >> > > > > > # lttv -m textDump -t /tmp/trace1
> > > >> > > > > > The input data file /var/tmp/trace1/metadata_0 does not
> > contain
> > > >> a
> > > >> > > trace
> > > >> > > > > > The input data file /var/tmp/trace1/block_0 does not contain
> > a
> > > >> trace
> > > >> > > > > > The input data file /var/tmp/trace1/fd_state_0 does not
> > contain
> > > >> a
> > > >> > > trace
> > > >> > > > > > The input data file /var/tmp/trace1/fs_0 does not contain a
> > > >> trace
> > > >> > > > > > The input data file /var/tmp/trace1/global_state_0 does not
> > > >> contain a
> > > >> > > > > trace
> > > >> > > > > > The input data file /var/tmp/trace1/input_0 does not contain
> > a
> > > >> trace
> > > >> > > > > > The input data file /var/tmp/trace1/ipc_0 does not contain a
> > > >> trace
> > > >> > > > > > The input data file /var/tmp/trace1/irq_state_0 does not
> > contain
> > > >> a
> > > >> > > trace
> > > >> > > > > > **
> > > >> > > > > > ERROR:tracefile.c:1355:map_block: assertion failed:
> > (block_num <
> > > >> > > > > > tf->num_blocks)
> > > >> > > > > > /bin/lttv: line 15: 101 Aborted $0.real $*
> > > >> > > > > >
> > > >> > > > > > # ls -l /tmp/trace1/
> > > >> > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:04
> > block_0
> > > >> > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:04
> > > >> fd_state_0
> > > >> > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:04 fs_0
> > > >> > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:04
> > > >> > > global_state_0
> > > >> > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:04
> > input_0
> > > >> > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:04
> > ipc_0
> > > >> > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:04
> > > >> irq_state_0
> > > >> > > > > > -rwxr-xr-x 1 root root 634880 Jan 1 00:04
> > kernel_0
> > > >> > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:04
> > > >> metadata_0
> > > >> > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:04 mm_0
> > > >> > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:04
> > > >> > > module_state_0
> > > >> > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:04
> > net_0
> > > >> > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:04
> > > >> netif_state_0
> > > >> > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:04
> > rcu_0
> > > >> > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:04
> > > >> > > softirq_state_0
> > > >> > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:04
> > > >> task_state_0
> > > >> > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:04
> > > >> userspace_0
> > > >> > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:04
> > > >> vm_state_0
> > > >> > > > >
> > > >> > > > > Please make sure you have enough free space on your device. It
> > > >> might be
> > > >> > > > > causing your problems. Also knowing which LTTng version you
> > use
> > > >> might
> > > >> > > > > help (you gave only the kernel version, not LTTng version). It
> > > >> might be
> > > >> > > > > a bug specific to this old LTTng version (which matched you
> > > >> 2.6.28.4
> > > >> > > > > kernel). Upgrading to a newer LTTng might help. Please see the
> > > >> > > > > compability list in the http://lttng.org download section,
> > where
> > > >> you
> > > >> > > can
> > > >> > > > > find information about changelogs and cross-compatibility
> > between
> > > >> tool
> > > >> > > > > versions.
> > > >> > > > >
> > > >> > > > > Thanks,
> > > >> > > > >
> > > >> > > > > Mathieu
> > > >> > > > >
> > > >> > > > > >
> > > >> > > > > > i am attching /tmp/trace1 and /mnt/debugfs
> > > >> > > > > >
> > > >> > > > > > still am not getting traces through lttv
> > > >> > > > > >
> > > >> > > > > > Regards
> > > >> > > > > > Dhirender
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > > On Wed, Apr 13, 2011 at 3:53 PM, Srikanth Krishnakar
> > > >> > > > > > <skrishnakar at gmail.com>wrote:
> > > >> > > > > >
> > > >> > > > > > > Dhirendar,
> > > >> > > > > > >
> > > >> > > > > > > You just connected the markers, where are the further
> > steps ?
> > > >> > > > > > >
> > > >> > > > > > > I mean :
> > > >> > > > > > >
> > > >> > > > > > > 1. ltt-armall
> > > >> > > > > > > 2. lttctl -C -w /tmp/trace1 trace1
> > > >> > > > > > >
> > > >> > > > > > > <Run your tests here, If you have any >
> > > >> > > > > > >
> > > >> > > > > > > 3. lttctl -D trace1
> > > >> > > > > > > 4. ltt-disarmall
> > > >> > > > > > > 5. lttv -m textDump -t /tmp/trace1
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > > > -skrishnakar
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > > > On 04/13/2011 02:38 PM, Dhirender Baliyan wrote:
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > > > Hi
> > > >> > > > > > >
> > > >> > > > > > > I am using
> > > >> > > > > > > 1) lttng 2.6.28.4
> > > >> > > > > > > 2)lttctl 0.64
> > > >> > > > > > > 3) lttv-0.12.10
> > > >> > > > > > >
> > > >> > > > > > > cross compiled it for mips
> > > >> > > > > > >
> > > >> > > > > > > my config option are CONFIG_LTT=y
> > > >> > > > > > > CONFIG_LTT_FILTER=y
> > > >> > > > > > > CONFIG_LTT_TRACEPROBES=y
> > > >> > > > > > > # CONFIG_HAVE_LTT_DUMP_TABLES is not set
> > > >> > > > > > > CONFIG_LTT_RELAY_ALLOC=y
> > > >> > > > > > > CONFIG_LTT_RELAY=y
> > > >> > > > > > > # CONFIG_LTT_RELAY_LOCKED is not set
> > > >> > > > > > > # CONFIG_LTT_RELAY_CHECK_RANDOM_ACCESS is not set
> > > >> > > > > > > CONFIG_LTT_SERIALIZE=y
> > > >> > > > > > > CONFIG_LTT_MARKER_CONTROL=y
> > > >> > > > > > > CONFIG_LTT_TRACER=y
> > > >> > > > > > > CONFIG_LTT_ALIGNMENT=y
> > > >> > > > > > > CONFIG_LTT_DEBUG_EVENT_SIZE=y
> > > >> > > > > > > CONFIG_LTT_USERSPACE_EVENT=y
> > > >> > > > > > > CONFIG_LTT_VMCORE=y
> > > >> > > > > > > CONFIG_LTT_NETLINK_CONTROL=y
> > > >> > > > > > > CONFIG_LTT_TRACE_CONTROL=y
> > > >> > > > > > > CONFIG_LTT_STATEDUMP=y
> > > >> > > > > > >
> > > >> > > > > > > now
> > > >> > > > > > > ltt-armall
> > > >> > > > > > > lttctl os running fine
> > > >> > > > > > >
> > > >> > > > > > > but when i do
> > > >> > > > > > > lttv -m textDump -t /tmp/trace1
> > > >> > > > > > > The input data file /var/tmp/trace1/metadata_0 does not
> > > >> contain a
> > > >> > > > > trace
> > > >> > > > > > > The input data file /var/tmp/trace1/block_0 does not
> > contain a
> > > >> > > trace
> > > >> > > > > > > The input data file /var/tmp/trace1/fd_state_0 does not
> > > >> contain a
> > > >> > > trace
> > > >> > > > > > > The input data file /var/tmp/trace1/fs_0 does not contain
> > a
> > > >> trace
> > > >> > > > > > > The input data file /var/tmp/trace1/global_state_0 does
> > not
> > > >> contain
> > > >> > > a
> > > >> > > > > trace
> > > >> > > > > > > The input data file /var/tmp/trace1/input_0 does not
> > contain a
> > > >> > > trace
> > > >> > > > > > > The input data file /var/tmp/trace1/ipc_0 does not contain
> > a
> > > >> trace
> > > >> > > > > > > The input data file /var/tmp/trace1/irq_state_0 does not
> > > >> contain a
> > > >> > > > > trace
> > > >> > > > > > > **
> > > >> > > > > > > ERROR:tracefile.c:1355:map_block: assertion failed:
> > (block_num
> > > >> <
> > > >> > > > > > > tf->num_blocks)
> > > >> > > > > > > /bin/lttv: line 15: 100 Aborted $0.real
> > $*
> > > >> > > > > > >
> > > >> > > > > > > this output is coming
> > > >> > > > > > > so i checked /tmp/trace1 file , then i found the files are
> > > >> having 0
> > > >> > > > > size
> > > >> > > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:07
> > > >> block_0
> > > >> > > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:07
> > > >> fd_state_0
> > > >> > > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:07
> > fs_0
> > > >> > > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:07
> > > >> > > global_state_0
> > > >> > > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:07
> > > >> input_0
> > > >> > > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:07
> > ipc_0
> > > >> > > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:07
> > > >> irq_state_0
> > > >> > > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:07
> > > >> kernel_0
> > > >> > > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:07
> > > >> metadata_0
> > > >> > > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:07
> > mm_0
> > > >> > > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:07
> > > >> > > module_state_0
> > > >> > > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:07
> > net_0
> > > >> > > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:07
> > > >> > > netif_state_0
> > > >> > > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:07
> > rcu_0
> > > >> > > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:07
> > > >> > > softirq_state_0
> > > >> > > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:07
> > > >> > > task_state_0
> > > >> > > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:07
> > > >> userspace_0
> > > >> > > > > > > -rwxr-xr-x 1 root root 0 Jan 1 00:07
> > > >> vm_state_0
> > > >> > > > > > >
> > > >> > > > > > > I am not getting why this is coming.
> > > >> > > > > > > Please help
> > > >> > > > > > > Thanks in advance
> > > >> > > > > > >
> > > >> > > > > > > Regards
> > > >> > > > > > > Dhiren
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > > > _______________________________________________
> > > >> > > > > > > ltt-dev mailing listltt-dev at lists.casi.polymtl.cahttp://
> > > >> > > > > lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > > > _______________________________________________
> > > >> > > > > > > ltt-dev mailing list
> > > >> > > > > > > ltt-dev at lists.casi.polymtl.ca
> > > >> > > > > > >
> > http://lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > >
> > > >> > > > >
> > > >> > > > >
> > > >> > > > > > _______________________________________________
> > > >> > > > > > ltt-dev mailing list
> > > >> > > > > > ltt-dev at lists.casi.polymtl.ca
> > > >> > > > > >
> > http://lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev
> > > >> > > > >
> > > >> > > > >
> > > >> > > > > --
> > > >> > > > > Mathieu Desnoyers
> > > >> > > > > Operating System Efficiency R&D Consultant
> > > >> > > > > EfficiOS Inc.
> > > >> > > > > http://www.efficios.com
> > > >> > > > >
> > > >> > >
> > > >> > > --
> > > >> > > Mathieu Desnoyers
> > > >> > > Operating System Efficiency R&D Consultant
> > > >> > > EfficiOS Inc.
> > > >> > > http://www.efficios.com
> > > >> > >
> > > >>
> > > >> --
> > > >> Mathieu Desnoyers
> > > >> Operating System Efficiency R&D Consultant
> > > >> EfficiOS Inc.
> > > >> http://www.efficios.com
> > > >>
> > > >
> > > >
> >
> > --
> > Mathieu Desnoyers
> > Operating System Efficiency R&D Consultant
> > EfficiOS Inc.
> > http://www.efficios.com
> >
--
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com
More information about the lttng-dev
mailing list