[lttng-dev] Inappropriate ioctl for device error

[Neil Bryan]

Hi Jonathan,

Please see below:

-----Original Message-----
From: Jonathan Rajotte Julien [mailto:Jonathan.rajotte-julien at efficios.com] 
Sent: Monday, March 16, 2015 5:21 PM
To: Neil Bryan
Cc: lttng-dev at lists.lttng.org
Subject: Re: [lttng-dev] Inappropriate ioctl for device error

Hey Neil,

On 2015-03-16 07:22 AM, Neil Bryan wrote:
> What distribution are you on?
>
> The kernel-version is:
>
> Linux socfpga_cyclone5 3.10.31-ltsi-05082-g018fe6c #1 SMP Fri Mar 13 
> 16:38:52 GMT 2015 armv7l GNU/Linux

> We noticed that you had multiple lttng-modules recipes, 

This was not my doing, if you look here:

https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/recipes-kernel/lttng?h=daisy

You can see that the Yocto Poky Daisy repository confusingly contains bitbake recipes for several versions:

lttng-modules_2.3.3.bb
lttng-modules_2.4.2.bb
lttng-tools_2.3.1.bb
lttng-tools_2.4.0.bb
lttng-ust_2.3.1.bb
lttng-ust_2.4.0.bb

I enable LTTng to be built and included in a core-image-minimal build by adding the following to the conf/local.conf file:

CORE_IMAGE_EXTRA_INSTALL = "ethtool lttng-ust lttng-modules lttng-tools openssh kmod"

Note to others:
ethtool -> adds ethtool to the minimal image
openssh -> allows ssh/scp on target.
kmod    -> adds modinfo to the RFS.

I shall remove lttng-*_2.3.*.bb recipes and try again. I am not sure how Yocto Daisy decides which versions of lttng-modules to build,
but if the recipes are not there, then it can't build that version!

Here is the output from modinfo for each LTTng *.ko file:

root at socfpga_cyclone5:/lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules# find -name '*.ko' | xargs modinfo
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./lttng-ring-buffer-client-mmap-discard.ko
description:    LTTng Ring Buffer Client Discard Mode
author:         Mathieu Desnoyers
license:        GPL and additional rights
description:    LTTng ring buffer discard-mmap client
author:         Mathieu Desnoyers
license:        GPL and additional rights
depends:        lttng-tracer,lttng-lib-ring-buffer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./lttng-ring-buffer-client-overwrite.ko
description:    LTTng Ring Buffer Client Overwrite Mode
author:         Mathieu Desnoyers
license:        GPL and additional rights
description:    LTTng ring buffer overwrite client
author:         Mathieu Desnoyers
license:        GPL and additional rights
depends:        lttng-tracer,lttng-lib-ring-buffer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./lttng-tracer.ko
description:    LTTng Events
author:         Mathieu Desnoyers <mathieu.desnoyers at efficios.com>
license:        GPL and additional rights
description:    Linux Trace Toolkit PID Context
author:         Mathieu Desnoyers
license:        GPL and additional rights
description:    Linux Trace Toolkit Perf Support
author:         Mathieu Desnoyers
license:        GPL and additional rights
description:    Linux Trace Toolkit Priority Context
author:         Mathieu Desnoyers
license:        GPL and additional rights
description:    Linux Trace Toolkit Nice Context
author:         Mathieu Desnoyers
license:        GPL and additional rights
description:    Linux Trace Toolkit vPID Context
author:         Mathieu Desnoyers
license:        GPL and additional rights
description:    Linux Trace Toolkit TID Context
author:         Mathieu Desnoyers
license:        GPL and additional rights
description:    Linux Trace Toolkit vTID Context
author:         Mathieu Desnoyers
license:        GPL and additional rights
description:    Linux Trace Toolkit PPID Context
author:         Mathieu Desnoyers
license:        GPL and additional rights
description:    Linux Trace Toolkit vPPID Context
author:         Mathieu Desnoyers
license:        GPL and additional rights
description:    Linux Trace Toolkit Perf Support
author:         Mathieu Desnoyers
license:        GPL and additional rights
description:    Linux Trace Toolkit Perf Support
author:         Mathieu Desnoyers
license:        GPL and additional rights
depends:        lttng-lib-ring-buffer,lttng-statedump,lttng-ftrace
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./lttng-ring-buffer-metadata-mmap-client.ko
description:    LTTng Ring Buffer Metadata Client
author:         Mathieu Desnoyers
license:        GPL and additional rights
description:    LTTng ring buffer metadata-mmap client
author:         Mathieu Desnoyers
license:        GPL and additional rights
depends:        lttng-tracer,lttng-lib-ring-buffer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./probes/lttng-probe-napi.ko
description:    LTTng napi probes
author:         Wade Farnsworth <wade_farnsworth at mentor.com>
license:        GPL and additional rights
depends:        lttng-tracer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./probes/lttng-probe-scsi.ko
description:    LTTng scsi probes
author:         Wade Farnsworth <wade_farnsworth at mentor.com> and Andrew Gabbasov <andrew_gabbasov at mentor.com>
license:        GPL and additional rights
depends:        lttng-tracer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./probes/lttng-probe-timer.ko
description:    LTTng timer probes
author:         Mathieu Desnoyers <mathieu.desnoyers at efficios.com>
license:        GPL and additional rights
depends:        lttng-tracer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./probes/lttng-probe-skb.ko
description:    LTTng skb probes
author:         Wade Farnsworth <wade_farnsworth at mentor.com> and Andrew Gabbasov <andrew_gabbasov at mentor.com>
license:        GPL and additional rights
depends:        lttng-tracer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./probes/lttng-probe-udp.ko
description:    LTTng udp probes
author:         Wade Farnsworth <wade_farnsworth at mentor.com>
license:        GPL and additional rights
depends:        lttng-tracer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./probes/lttng-probe-compaction.ko
description:    LTTng compaction probes
author:         Andrew Gabbasov <andrew_gabbasov at mentor.com>
license:        GPL and additional rights
depends:        lttng-tracer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./probes/lttng-probe-ext4.ko
description:    LTTng ext4 probes
author:         Andrew Gabbasov <andrew_gabbasov at mentor.com>
license:        GPL and additional rights
depends:        lttng-tracer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./probes/lttng-probe-workqueue.ko
description:    LTTng workqueue probes
author:         Andrew Gabbasov <andrew_gabbasov at mentor.com>
license:        GPL and additional rights
depends:        lttng-tracer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./probes/lttng-probe-vmscan.ko
description:    LTTng vmscan probes
author:         Wade Farnsworth <wade_farnsworth at mentor.com>, Paul Woegerer <paul_woegerer at mentor.com>, and Andrew Gabbasov <andrew_gabbasov at mentor.com>
license:        GPL and additional rights
depends:        lttng-tracer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./probes/lttng-probe-signal.ko
description:    LTTng signal probes
author:         Mathieu Desnoyers <mathieu.desnoyers at efficios.com>
license:        GPL and additional rights
depends:        lttng-tracer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./probes/lttng-probe-gpio.ko
description:    LTTng gpio probes
author:         Wade Farnsworth <wade_farnsworth at mentor.com>
license:        GPL and additional rights
depends:        lttng-tracer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./probes/lttng-probe-kmem.ko
description:    LTTng kmem probes
author:         Wade Farnsworth <wade_farnsworth at mentor.com> and Andrew Gabbasov <andrew_gabbasov at mentor.com>
license:        GPL and additional rights
depends:        lttng-tracer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./probes/lttng-probe-sock.ko
description:    LTTng sock probes
author:         Wade Farnsworth <wade_farnsworth at mentor.com>
license:        GPL and additional rights
depends:        lttng-tracer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./probes/lttng-probe-printk.ko
description:    LTTng printk probes
author:         Andrew Gabbasov <andrew_gabbasov at mentor.com>
license:        GPL and additional rights
depends:        lttng-tracer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./probes/lttng-probe-jbd.ko
description:    LTTng jbd probes
author:         Wade Farnsworth <wade_farnsworth at mentor.com>, Paul Woegerer <paul_woegerer at mentor.com>, and Andrew Gabbasov <andrew_gabbasov at mentor.com>
license:        GPL and additional rights
depends:        lttng-tracer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./probes/lttng-probe-power.ko
description:    LTTng power probes
author:         Wade Farnsworth <wade_farnsworth at mentor.com> and Andrew Gabbasov <andrew_gabbasov at mentor.com>
license:        GPL and additional rights
depends:        lttng-tracer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./probes/lttng-probe-sunrpc.ko
description:    LTTng sunrpc probes
author:         Andrew Gabbasov <andrew_gabbasov at mentor.com>
license:        GPL and additional rights
depends:        lttng-tracer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./probes/lttng-probe-ext3.ko
description:    LTTng ext3 probes
author:         Wade Farnsworth <wade_farnsworth at mentor.com>, Paul Woegerer <paul_woegerer at mentor.com>, and Andrew Gabbasov <andrew_gabbasov at mentor.com>
license:        GPL and additional rights
depends:        lttng-tracer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./probes/lttng-probe-net.ko
description:    LTTng net probes
author:         Wade Farnsworth <wade_farnsworth at mentor.com>
license:        GPL and additional rights
depends:        lttng-tracer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./probes/lttng-probe-jbd2.ko
description:    LTTng jbd2 probes
author:         Wade Farnsworth <wade_farnsworth at mentor.com> and Andrew Gabbasov <andrew_gabbasov at mentor.com>
license:        GPL and additional rights
depends:        lttng-tracer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./probes/lttng-probe-irq.ko
description:    LTTng irq probes
author:         Mathieu Desnoyers <mathieu.desnoyers at efficios.com>
license:        GPL and additional rights
depends:        lttng-tracer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./probes/lttng-ftrace.ko
description:    Linux Trace Toolkit Ftrace Support
author:         Mathieu Desnoyers
license:        GPL and additional rights
depends:
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./probes/lttng-probe-regmap.ko
description:    LTTng regmap probes
author:         Andrew Gabbasov <andrew_gabbasov at mentor.com>
license:        GPL and additional rights
depends:        lttng-tracer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./probes/lttng-probe-random.ko
description:    LTTng random probes
author:         Andrew Gabbasov <andrew_gabbasov at mentor.com>
license:        GPL and additional rights
depends:        lttng-tracer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./probes/lttng-probe-block.ko
description:    LTTng block probes
author:         Mathieu Desnoyers <mathieu.desnoyers at efficios.com>
license:        GPL and additional rights
depends:        lttng-tracer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./probes/lttng-probe-module.ko
description:    LTTng module probes
author:         Wade Farnsworth <wade_farnsworth at mentor.com> and Andrew Gabbasov <andrew_gabbasov at mentor.com>
license:        GPL and additional rights
depends:        lttng-tracer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./probes/lttng-probe-sched.ko
description:    LTTng sched probes
author:         Mathieu Desnoyers <mathieu.desnoyers at efficios.com>
license:        GPL and additional rights
depends:        lttng-tracer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./probes/lttng-probe-rcu.ko
description:    LTTng rcu probes
author:         Andrew Gabbasov <andrew_gabbasov at mentor.com>
license:        GPL and additional rights
depends:        lttng-tracer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./probes/lttng-probe-statedump.ko
description:    LTTng statedump probes
author:         Mathieu Desnoyers <mathieu.desnoyers at efficios.com>
license:        GPL and additional rights
depends:        lttng-tracer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./probes/lttng-types.ko
description:    LTTng types
author:         Mathieu Desnoyers <mathieu.desnoyers at efficios.com>
license:        GPL and additional rights
depends:
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./lttng-ring-buffer-client-discard.ko
description:    LTTng Ring Buffer Client Discard Mode
author:         Mathieu Desnoyers
license:        GPL and additional rights
description:    LTTng ring buffer discard client
author:         Mathieu Desnoyers
license:        GPL and additional rights
depends:        lttng-tracer,lttng-lib-ring-buffer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./lib/lttng-lib-ring-buffer.ko
description:    Ring Buffer Library VFS
author:         Mathieu Desnoyers
license:        GPL and additional rights
depends:
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./lttng-ring-buffer-client-mmap-overwrite.ko
description:    LTTng Ring Buffer Client Overwrite Mode
author:         Mathieu Desnoyers
license:        GPL and additional rights
description:    LTTng ring buffer overwrite-mmap client
author:         Mathieu Desnoyers
license:        GPL and additional rights
depends:        lttng-tracer,lttng-lib-ring-buffer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./lttng-statedump.ko
description:    Linux Trace Toolkit Next Generation Statedump
author:         Jean-Hugues Deschenes
license:        GPL and additional rights
depends:
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8
filename:       /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules/./lttng-ring-buffer-metadata-client.ko
description:    LTTng Ring Buffer Metadata Client
author:         Mathieu Desnoyers
license:        GPL and additional rights
description:    LTTng ring buffer metadata client
author:         Mathieu Desnoyers
license:        GPL and additional rights
depends:        lttng-tracer,lttng-lib-ring-buffer
vermagic:       3.10.31-ltsi-05082-g018fe6c SMP mod_unload ARMv7 p2v8

could you do a modinfo on one of the .ko generated on the yocto build?

I tried this command

find /lib/modules/`uname -r`/ | grep lttng | head -n 1 | xargs modinfo

And got this result:

root at socfpga_cyclone5:~# find /lib/modules/`uname -r`/ | grep lttng | head -n 1 | xargs modinfo
modinfo: ERROR: Module /lib/modules/3.10.31-ltsi-05082-g018fe6c/kernel/lttng-modules not found.

The following gives this:

root at socfpga_cyclone5:~# uname -r
3.10.31-ltsi-05082-g018fe6c

Which matches up with the directory under /lib/modules so I am happy that the modules are compatible.

It is best to have only one version installed to minimize version incompatibility between LTTng component.

>
> Lttng version is:
>
> root at socfpga_cyclone5:~# lttng --version lttng (LTTng Trace Control) 
> 2.4.0 - Époque Opaque
>
> I am using this version because it is the version included as a recipe in the Yocto Daisy distribution for the Altera SoCFPGA.
Is it possible to share it with us (url/file) ?

> As I am five hours or so in front of you guys, I shall try and get a 
> build working with the latest version of LTTng (2.6.0), but I am not a 
> bitbake/yocto expert

Neither am I :P. Looks like you can find up-to-date recipe here [1] . 
You could also try their mailing list for help.

[1] http://packages.yoctoproject.org/rrs/recipes/1.8/M4/



> And each time I try to change something, I end up with a whole raft of new problems to solve.

Let us know what happen.

Cheers!

Jonathan

> Thanks again,
>
> Neil.
>
> On 2015-03-13 12:40 PM, Neil Bryan wrote:
>> Hello,
>>
>> When I create a session and try to capture some kernel traces I see 
>> the following (first output from lttng-sessiond and then from the 
>> tracing session):
>>
>>
>> I am using the following version of LTTng:
>>
>> nbryan at meteor:/data/nbryan/Meteor/YoctoDaisy/poky/meta/recipes-kernel
>> /
>> lttng$
>> ls -1
>>
>> babeltrace_1.1.1.bb
>>
>> lttng-modules
>>
>> lttng-modules_2.3.3.bb
>>
>> lttng-modules_2.4.2.bb
>>
>> lttng-tools
>>
>> lttng-tools_2.3.1.bb
>>
>> lttng-tools_2.4.0.bb
>>
>> lttng-ust
>>
>> lttng-ust_2.3.1.bb
>>
>> lttng-ust_2.4.0.bb
>>
>> Thank you for any clues,
>>
>>
>> Regards,
>>
>> Neil.
>>
>>
>>
>> _______________________________________________
>> lttng-dev mailing list
>> lttng-dev at lists.lttng.org
>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev