[ltt-dev] "Kernel/LTTV event size differs for event udpv4_rcv_extended: kernel 32, LTTV 28" error when view trace
Mathieu Desnoyers
compudj at krystal.dyndns.org
Wed Nov 10 09:08:21 EST 2010
* xufeng zhang (xufeng.zhang at windriver.com) wrote:
> On 11/09/2010 07:57 PM, Mathieu Desnoyers wrote:
>> * Mathieu Desnoyers (compudj at krystal.dyndns.org) wrote:
>>
>>> * xufeng zhang (xufeng.zhang at windriver.com) wrote:
>>>
>>>> On 11/08/2010 05:55 PM, Mathieu Desnoyers wrote:
>>>>
>>>>> * xufeng zhang (xufeng.zhang at windriver.com) wrote:
>>>>>
>>>>>
>>>>>> On 11/08/2010 09:45 AM, Benjamin Poirier wrote:
>>>>>>
>>>>>>
>>>>>>> On 05/11/10 02:24 AM, xufeng zhang wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Hi,
>>>>>>>> I'm using ltt-control-0.86 and lttv-0.12.31 on ARM to trace the kernel,
>>>>>>>> however,
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> Which version of the kernel tracer patches are you using?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> I have query the developers who merged LTTng into kernel, they said
>>>>>> there is no incompatibility
>>>>>> between LTTV and LTTng, so does there any other reasons which lead to
>>>>>> this error?
>>>>>>
>>>>>> Thanks,
>>>>>> Xufeng Zhang
>>>>>>
>>>>>>
>>>>>>> -Ben
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> I cannot view the trace after I have successfully generated, here is the
>>>>>>>> error
>>>>>>>> when I run 'lttv-gui -t trace1' command:
>>>>>>>> ** Message: statistics viewer : background computation data ready.
>>>>>>>>
>>>>>>>> ** (lttv.real:6559): WARNING **: Cannot find pin_in in schedchange 671
>>>>>>>>
>>>>>>>> ** ERROR **: Kernel/LTTV event size differs for event
>>>>>>>> udpv4_rcv_extended: kernel 32, LTTV 28
>>>>>>>>
>>>>>>>>
>>>>> Please compare the /ltt/probes/net-extended-trace.c file from your
>>>>> distro kernel tree to the:
>>>>>
>>>>> lttng-modules package, file: /probes/net-extended-trace.c
>>>>>
>>>>> Also compare the /ltt/ltt-type-serializer.h (or was it
>>>>> include/linux/ltt-type-serializer.h ?) from your distro kernel with the
>>>>> lttng-modules package:
>>>>>
>>>>> /ltt-type-serializer.h
>>>>>
>>>>> I remember having fixed this problem in the past months. Look closely at
>>>>> the structure layout for "struct serialize_l214421224411111", "struct
>>>>> serialize_l4412228" and "struct serialize_l4421224411111" vs the types
>>>>> passed to the DEFINE_MARKER_TP() in net-extended-trace.c.
>>>>>
>>>>>
>>>> I have compare my kernel tree to lttng-modules-0.19.2, there is no
>>>> different
>>>> in ltt-type-serializer.h file, and below is the diff info for
>>>> net-extended-trace.c:
>>>>
>>>> --- net-extended-trace.c.kernel 2010-11-09 15:09:24.000000000 +0800
>>>> +++ net-extended-trace.c 2010-11-09 15:12:47.000000000 +0800
>>>> @@ -20,14 +20,15 @@
>>>>
>>>> #include<linux/in_route.h>
>>>> #include<linux/ip.h>
>>>> -#include<linux/ltt-type-serializer.h>
>>>> #include<linux/module.h>
>>>> #include<linux/tcp.h>
>>>> #include<linux/udp.h>
>>>> #include<net/route.h>
>>>> #include<trace/net.h>
>>>>
>>>> -void probe_net_dev_xmit_extended(struct sk_buff *skb);
>>>> +#include "../ltt-type-serializer.h"
>>>> +
>>>> +void probe_net_dev_xmit_extended(void *_data, struct sk_buff *skb);
>>>>
>>>> DEFINE_MARKER_TP(net, dev_xmit_extended, net_dev_xmit,
>>>> probe_net_dev_xmit_extended, "skb 0x%lX network_protocol #n2u%hu "
>>>> @@ -35,7 +36,7 @@
>>>> "tot_len #n2u%hu ihl #1u%u source #n2u%hu dest #n2u%hu seq
>>>> #n4u%lu "
>>>> "ack_seq #n4u%lu doff #1u%u ack #1u%u rst #1u%u syn #1u%u fin
>>>> #1u%u");
>>>>
>>>> -notrace void probe_net_dev_xmit_extended(struct sk_buff *skb)
>>>> +notrace void probe_net_dev_xmit_extended(void *_data, struct sk_buff *skb)
>>>> {
>>>> struct marker *marker;
>>>> struct serialize_l214421224411111 data;
>>>> @@ -70,14 +71,14 @@
>>>> &data, serialize_sizeof(data), sizeof(long));
>>>> }
>>>>
>>>> -void probe_tcpv4_rcv_extended(struct sk_buff *skb);
>>>> +void probe_tcpv4_rcv_extended(void *_data, struct sk_buff *skb);
>>>>
>>>> DEFINE_MARKER_TP(net, tcpv4_rcv_extended, net_tcpv4_rcv,
>>>> probe_tcpv4_rcv_extended, "skb 0x%lX saddr #n4u%lu daddr #n4u%lu "
>>>> "tot_len #n2u%hu ihl #1u%u source #n2u%hu dest #n2u%hu seq
>>>> #n4u%lu "
>>>> "ack_seq #n4u%lu doff #1u%u ack #1u%u rst #1u%u syn #1u%u fin
>>>> #1u%u");
>>>>
>>>> -notrace void probe_tcpv4_rcv_extended(struct sk_buff *skb)
>>>> +notrace void probe_tcpv4_rcv_extended(void *_data, struct sk_buff *skb)
>>>> {
>>>> struct marker *marker;
>>>> struct serialize_l4421224411111 data;
>>>> @@ -104,14 +105,14 @@
>>>> &data, serialize_sizeof(data), sizeof(long));
>>>> }
>>>>
>>>> -void probe_udpv4_rcv_extended(struct sk_buff *skb);
>>>> +void probe_udpv4_rcv_extended(void *_data, struct sk_buff *skb);
>>>>
>>>> DEFINE_MARKER_TP(net, udpv4_rcv_extended, net_udpv4_rcv,
>>>> probe_udpv4_rcv_extended, "skb 0x%lX saddr #n4u%lu daddr #n4u%lu "
>>>> "unicast #1u%u ulen #n2u%hu source #n2u%hu dest #n2u%hu "
>>>> "data_start #8u%lx");
>>>>
>>>> -notrace void probe_udpv4_rcv_extended(struct sk_buff *skb)
>>>> +notrace void probe_udpv4_rcv_extended(void *_data, struct sk_buff *skb)
>>>> {
>>>> struct marker *marker;
>>>> struct serialize_l4412228 data;
>>>>
>>>>
>>>> I don't think the difference in net-extended-trace.c will make any impact.
>>>> This problem has troubled me for weeks.
>>>>
>>> Can you try to reproduce with the latest LTTng version ? It's very hard
>>> for us to help you if we don't know the LTTng version you are using,
>>> especially for a bug I am aware to have fixed recently.
>>>
>> Ah, I remember it now. It had something to do with alignment of 64-bit
>> fields on 32-bit machines. Older LTTng versions did align these fields
>> on the next 64-bit multiple. New LTTng versions do the same as gcc and
>> align these on the next 32-bit boundary.
>>
>> Check the diff of your include/linux/ltt-core.h ltt_align() function
>> between a recent lttng and yours.
>>
> Could you provide the content of this patch, I think my problem
> is almost the same. Thank you very much.
Then you may be facing the following problem: your kernel LTTng version
is too recent for your LTTV version. I had to change both LTTV and LTTng
at the same time to support this different alignment style.
So your LTTV is very probably incompatible with the LTTng version you
are using.
Mathieu
>
> Thanks,
> Xufeng Zhang
>> Thanks,
>>
>> Mathieu
>>
>>
>>> Also, you might simply try to unload the net-extended-trace module (or
>>> disable its markers) to see if it helps getting a parseable trace.
>>>
>>> Thanks,
>>>
>>> Mathieu
>>>
>>>
>>>> Thanks,
>>>> Xufeng Zhang
>>>>
>>>>> Thanks,
>>>>>
>>>>> Mathieu
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>>> aborting...
>>>>>>>> /usr/local/bin/lttv-gui: line 10: 6559 Aborted
>>>>>>>> $LTTV_CMD.real -m lttvwindow -m guievents -m guifilter -m guicontrolflow
>>>>>>>> -m resourceview -m guistat
>>>>>>>>
>>>>>>>>
>>>>>>>> And I found another people met the same problem on MIPS32, a patch have
>>>>>>>> also been provided for MIPS32.
>>>>>>>> Here is the similar code in ARM trace-clock.c:
>>>>>>>> linux/arch/arm/mach-omap2/trace-clock.c line 537
>>>>>>>> void get_trace_clock(void)
>>>>>>>> {
>>>>>>>> spin_lock(&trace_clock_lock);
>>>>>>>> if (trace_clock_refcount++)
>>>>>>>> goto end;
>>>>>>>> _start_trace_clock();
>>>>>>>> end:
>>>>>>>> spin_unlock(&trace_clock_lock);
>>>>>>>> }
>>>>>>>> EXPORT_SYMBOL_GPL(get_trace_clock);
>>>>>>>>
>>>>>>>> void put_trace_clock(void)
>>>>>>>> {
>>>>>>>> spin_lock(&trace_clock_lock);
>>>>>>>> WARN_ON(trace_clock_refcount<= 0);
>>>>>>>> if (trace_clock_refcount != 1)
>>>>>>>> goto end;
>>>>>>>> _stop_trace_clock();
>>>>>>>> end:
>>>>>>>> trace_clock_refcount--;
>>>>>>>> spin_unlock(&trace_clock_lock);
>>>>>>>> }
>>>>>>>> EXPORT_SYMBOL_GPL(put_trace_clock);
>>>>>>>>
>>>>>>>> Could anybody figure out what's wrong with the kernel code? Any
>>>>>>>> suggestion would be appreciated.
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Xufeng Zhang
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> ltt-dev mailing list
>>>>>>>> ltt-dev at lists.casi.polymtl.ca
>>>>>>>> http://lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> _______________________________________________
>>>>>> ltt-dev mailing list
>>>>>> ltt-dev at lists.casi.polymtl.ca
>>>>>> http://lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>> --
>>> Mathieu Desnoyers
>>> Operating System Efficiency R&D Consultant
>>> EfficiOS Inc.
>>> http://www.efficios.com
>>>
>>> _______________________________________________
>>> ltt-dev mailing list
>>> ltt-dev at lists.casi.polymtl.ca
>>> http://lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev
>>>
>>>
>>
>
--
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com
More information about the lttng-dev
mailing list