[ltt-dev] "Kernel/LTTV event size differs for event udpv4_rcv_extended: kernel 32, LTTV 28" error when view trace
Mathieu Desnoyers
compudj at krystal.dyndns.org
Wed Nov 10 10:14:37 EST 2010
* Mathieu Desnoyers (compudj at krystal.dyndns.org) wrote:
> * xufeng zhang (xufeng.zhang at windriver.com) wrote:
> > On 11/09/2010 07:57 PM, Mathieu Desnoyers wrote:
> >> * Mathieu Desnoyers (compudj at krystal.dyndns.org) wrote:
> >>
> >>> * xufeng zhang (xufeng.zhang at windriver.com) wrote:
> >>>
> >>>> On 11/08/2010 05:55 PM, Mathieu Desnoyers wrote:
> >>>>
> >>>>> * xufeng zhang (xufeng.zhang at windriver.com) wrote:
> >>>>>
> >>>>>
> >>>>>> On 11/08/2010 09:45 AM, Benjamin Poirier wrote:
> >>>>>>
> >>>>>>
> >>>>>>> On 05/11/10 02:24 AM, xufeng zhang wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>> Hi,
> >>>>>>>> I'm using ltt-control-0.86 and lttv-0.12.31 on ARM to trace the kernel,
> >>>>>>>> however,
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>> Hello,
> >>>>>>>
> >>>>>>> Which version of the kernel tracer patches are you using?
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>> I have query the developers who merged LTTng into kernel, they said
> >>>>>> there is no incompatibility
> >>>>>> between LTTV and LTTng, so does there any other reasons which lead to
> >>>>>> this error?
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Xufeng Zhang
> >>>>>>
> >>>>>>
> >>>>>>> -Ben
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>> I cannot view the trace after I have successfully generated, here is the
> >>>>>>>> error
> >>>>>>>> when I run 'lttv-gui -t trace1' command:
> >>>>>>>> ** Message: statistics viewer : background computation data ready.
> >>>>>>>>
> >>>>>>>> ** (lttv.real:6559): WARNING **: Cannot find pin_in in schedchange 671
> >>>>>>>>
> >>>>>>>> ** ERROR **: Kernel/LTTV event size differs for event
> >>>>>>>> udpv4_rcv_extended: kernel 32, LTTV 28
> >>>>>>>>
> >>>>>>>>
> >>>>> Please compare the /ltt/probes/net-extended-trace.c file from your
> >>>>> distro kernel tree to the:
> >>>>>
> >>>>> lttng-modules package, file: /probes/net-extended-trace.c
> >>>>>
> >>>>> Also compare the /ltt/ltt-type-serializer.h (or was it
> >>>>> include/linux/ltt-type-serializer.h ?) from your distro kernel with the
> >>>>> lttng-modules package:
> >>>>>
> >>>>> /ltt-type-serializer.h
> >>>>>
> >>>>> I remember having fixed this problem in the past months. Look closely at
> >>>>> the structure layout for "struct serialize_l214421224411111", "struct
> >>>>> serialize_l4412228" and "struct serialize_l4421224411111" vs the types
> >>>>> passed to the DEFINE_MARKER_TP() in net-extended-trace.c.
> >>>>>
> >>>>>
> >>>> I have compare my kernel tree to lttng-modules-0.19.2, there is no
> >>>> different
> >>>> in ltt-type-serializer.h file, and below is the diff info for
> >>>> net-extended-trace.c:
> >>>>
> >>>> --- net-extended-trace.c.kernel 2010-11-09 15:09:24.000000000 +0800
> >>>> +++ net-extended-trace.c 2010-11-09 15:12:47.000000000 +0800
> >>>> @@ -20,14 +20,15 @@
> >>>>
> >>>> #include<linux/in_route.h>
> >>>> #include<linux/ip.h>
> >>>> -#include<linux/ltt-type-serializer.h>
> >>>> #include<linux/module.h>
> >>>> #include<linux/tcp.h>
> >>>> #include<linux/udp.h>
> >>>> #include<net/route.h>
> >>>> #include<trace/net.h>
> >>>>
> >>>> -void probe_net_dev_xmit_extended(struct sk_buff *skb);
> >>>> +#include "../ltt-type-serializer.h"
> >>>> +
> >>>> +void probe_net_dev_xmit_extended(void *_data, struct sk_buff *skb);
> >>>>
> >>>> DEFINE_MARKER_TP(net, dev_xmit_extended, net_dev_xmit,
> >>>> probe_net_dev_xmit_extended, "skb 0x%lX network_protocol #n2u%hu "
> >>>> @@ -35,7 +36,7 @@
> >>>> "tot_len #n2u%hu ihl #1u%u source #n2u%hu dest #n2u%hu seq
> >>>> #n4u%lu "
> >>>> "ack_seq #n4u%lu doff #1u%u ack #1u%u rst #1u%u syn #1u%u fin
> >>>> #1u%u");
> >>>>
> >>>> -notrace void probe_net_dev_xmit_extended(struct sk_buff *skb)
> >>>> +notrace void probe_net_dev_xmit_extended(void *_data, struct sk_buff *skb)
> >>>> {
> >>>> struct marker *marker;
> >>>> struct serialize_l214421224411111 data;
> >>>> @@ -70,14 +71,14 @@
> >>>> &data, serialize_sizeof(data), sizeof(long));
> >>>> }
> >>>>
> >>>> -void probe_tcpv4_rcv_extended(struct sk_buff *skb);
> >>>> +void probe_tcpv4_rcv_extended(void *_data, struct sk_buff *skb);
> >>>>
> >>>> DEFINE_MARKER_TP(net, tcpv4_rcv_extended, net_tcpv4_rcv,
> >>>> probe_tcpv4_rcv_extended, "skb 0x%lX saddr #n4u%lu daddr #n4u%lu "
> >>>> "tot_len #n2u%hu ihl #1u%u source #n2u%hu dest #n2u%hu seq
> >>>> #n4u%lu "
> >>>> "ack_seq #n4u%lu doff #1u%u ack #1u%u rst #1u%u syn #1u%u fin
> >>>> #1u%u");
> >>>>
> >>>> -notrace void probe_tcpv4_rcv_extended(struct sk_buff *skb)
> >>>> +notrace void probe_tcpv4_rcv_extended(void *_data, struct sk_buff *skb)
> >>>> {
> >>>> struct marker *marker;
> >>>> struct serialize_l4421224411111 data;
> >>>> @@ -104,14 +105,14 @@
> >>>> &data, serialize_sizeof(data), sizeof(long));
> >>>> }
> >>>>
> >>>> -void probe_udpv4_rcv_extended(struct sk_buff *skb);
> >>>> +void probe_udpv4_rcv_extended(void *_data, struct sk_buff *skb);
> >>>>
> >>>> DEFINE_MARKER_TP(net, udpv4_rcv_extended, net_udpv4_rcv,
> >>>> probe_udpv4_rcv_extended, "skb 0x%lX saddr #n4u%lu daddr #n4u%lu "
> >>>> "unicast #1u%u ulen #n2u%hu source #n2u%hu dest #n2u%hu "
> >>>> "data_start #8u%lx");
> >>>>
> >>>> -notrace void probe_udpv4_rcv_extended(struct sk_buff *skb)
> >>>> +notrace void probe_udpv4_rcv_extended(void *_data, struct sk_buff *skb)
> >>>> {
> >>>> struct marker *marker;
> >>>> struct serialize_l4412228 data;
> >>>>
> >>>>
> >>>> I don't think the difference in net-extended-trace.c will make any impact.
> >>>> This problem has troubled me for weeks.
> >>>>
> >>> Can you try to reproduce with the latest LTTng version ? It's very hard
> >>> for us to help you if we don't know the LTTng version you are using,
> >>> especially for a bug I am aware to have fixed recently.
> >>>
> >> Ah, I remember it now. It had something to do with alignment of 64-bit
> >> fields on 32-bit machines. Older LTTng versions did align these fields
> >> on the next 64-bit multiple. New LTTng versions do the same as gcc and
> >> align these on the next 32-bit boundary.
> >>
> >> Check the diff of your include/linux/ltt-core.h ltt_align() function
> >> between a recent lttng and yours.
> >>
> > Could you provide the content of this patch, I think my problem
> > is almost the same. Thank you very much.
>
> Then you may be facing the following problem: your kernel LTTng version
> is too recent for your LTTV version. I had to change both LTTV and LTTng
> at the same time to support this different alignment style.
>
> So your LTTV is very probably incompatible with the LTTng version you
> are using.
Please compare yout "ltt_align" macro in lttv : ltt/ltt-private.h
between a more recent (and also older) lttv and yours.
Thanks,
Mathieu
>
> Mathieu
>
> >
> > Thanks,
> > Xufeng Zhang
> >> Thanks,
> >>
> >> Mathieu
> >>
> >>
> >>> Also, you might simply try to unload the net-extended-trace module (or
> >>> disable its markers) to see if it helps getting a parseable trace.
> >>>
> >>> Thanks,
> >>>
> >>> Mathieu
> >>>
> >>>
> >>>> Thanks,
> >>>> Xufeng Zhang
> >>>>
> >>>>> Thanks,
> >>>>>
> >>>>> Mathieu
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>>>> aborting...
> >>>>>>>> /usr/local/bin/lttv-gui: line 10: 6559 Aborted
> >>>>>>>> $LTTV_CMD.real -m lttvwindow -m guievents -m guifilter -m guicontrolflow
> >>>>>>>> -m resourceview -m guistat
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> And I found another people met the same problem on MIPS32, a patch have
> >>>>>>>> also been provided for MIPS32.
> >>>>>>>> Here is the similar code in ARM trace-clock.c:
> >>>>>>>> linux/arch/arm/mach-omap2/trace-clock.c line 537
> >>>>>>>> void get_trace_clock(void)
> >>>>>>>> {
> >>>>>>>> spin_lock(&trace_clock_lock);
> >>>>>>>> if (trace_clock_refcount++)
> >>>>>>>> goto end;
> >>>>>>>> _start_trace_clock();
> >>>>>>>> end:
> >>>>>>>> spin_unlock(&trace_clock_lock);
> >>>>>>>> }
> >>>>>>>> EXPORT_SYMBOL_GPL(get_trace_clock);
> >>>>>>>>
> >>>>>>>> void put_trace_clock(void)
> >>>>>>>> {
> >>>>>>>> spin_lock(&trace_clock_lock);
> >>>>>>>> WARN_ON(trace_clock_refcount<= 0);
> >>>>>>>> if (trace_clock_refcount != 1)
> >>>>>>>> goto end;
> >>>>>>>> _stop_trace_clock();
> >>>>>>>> end:
> >>>>>>>> trace_clock_refcount--;
> >>>>>>>> spin_unlock(&trace_clock_lock);
> >>>>>>>> }
> >>>>>>>> EXPORT_SYMBOL_GPL(put_trace_clock);
> >>>>>>>>
> >>>>>>>> Could anybody figure out what's wrong with the kernel code? Any
> >>>>>>>> suggestion would be appreciated.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Xufeng Zhang
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> _______________________________________________
> >>>>>>>> ltt-dev mailing list
> >>>>>>>> ltt-dev at lists.casi.polymtl.ca
> >>>>>>>> http://lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>> _______________________________________________
> >>>>>> ltt-dev mailing list
> >>>>>> ltt-dev at lists.casi.polymtl.ca
> >>>>>> http://lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>> --
> >>> Mathieu Desnoyers
> >>> Operating System Efficiency R&D Consultant
> >>> EfficiOS Inc.
> >>> http://www.efficios.com
> >>>
> >>> _______________________________________________
> >>> ltt-dev mailing list
> >>> ltt-dev at lists.casi.polymtl.ca
> >>> http://lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev
> >>>
> >>>
> >>
> >
>
> --
> Mathieu Desnoyers
> Operating System Efficiency R&D Consultant
> EfficiOS Inc.
> http://www.efficios.com
>
> _______________________________________________
> ltt-dev mailing list
> ltt-dev at lists.casi.polymtl.ca
> http://lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev
>
--
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com
More information about the lttng-dev
mailing list