[lttng-dev] loading modules with modprobe
Kienan Stewart
kstewart at efficios.com
Thu Jul 18 10:47:06 EDT 2024
Hi François,
The dmesg excerpts are spaced by a number of minutes each, are they from
different attempts to build and load the modules?
On 7/17/24 8:44 PM, François Belias wrote:
> Hello Kienan,
>
> Thanks to your advice, I managed to load some modules. However, I
> encountered errors like these when running `dmesg | grep lttng`:
>
> ```
> [ 218.043672] module lttng_kretprobes: .gnu.linkonce.this_module
> section size must match the kernel's built struct module size at run time
This sounds like the version of the module is built against the wrong
kernel headers or source.
> [ 650.560883] lttng_kretprobes: module verification failed: signature
> and/or required key missing - tainting kernel
Is your system running with secure boot enabled? If so, you either need
to disable secure boot or sign the built modules with a MOK key that is
correctly enrolled.
See the following documentation:
- https://lttng.org/docs/v2.13/#doc-linux-kernel-sig
- https://www.kernel.org/doc/html/latest/admin-guide/module-signing.html
- https://github.com/dell/dkms
-
https://gitlab.com/linux-kernel/stable/-/blob/master/kernel/module/main.c#L2064
Whatever you did for this test is pretty closed to working. The previous
errors with the section size check happen earlier in the module load
process. This check seems to be one of the last, c.f.
https://elixir.bootlin.com/linux/v6.10/source/kernel/module/main.c#L2908
> [ 1468.329988] module lttng_ring_buffer_client_discard:
> .gnu.linkonce.this_module section size must match the kernel's built
> struct module size at run time
> ```
>
> I'm unsure of their meaning. After loading my module, how can I be
> certain that my tracepoints are active? I read online that people use
> `sudo lttng list -k` for this purpose, but when I tried it, I got the
> error: "Error: Unable to list kernel events: Kernel tracer not
> available". Is there another way?
If you're still getting "kernel tracer not available" then either
required kernel modules haven't been loaded or you started lttng-session
with `--no-kernel` option. `lttng-sessiond -vvv`, dmesg, and/or
journalctl should have information on which modules failed to load.
thanks,
kienan
>
> Thank you for your assistance.
>
> Le mer. 17 juill. 2024, à 12 h 08, Kienan Stewart <kstewart at efficios.com
> <mailto:kstewart at efficios.com>> a écrit :
>
> Hi François,
>
> On 7/17/24 11:47 AM, François Belias wrote:
> > Hello, Kienan
> >
> >
> > *Right. Which kernel are you actually running at this time? You
> can check
> > using: `uname -a`*
> > Linux philippe 6.5.0-41-generic #41~22.04.2-Ubuntu SMP
> PREEMPT_DYNAMIC
> > Mon Jun 3 11:32:55 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
> >
> > *Since you've got a few different images and headers installed, it is
> > possible to check that same info the linux-source package: `dpkg -l |
> > grep linux-source`.*
> > ii linux-source 5.15.0.116.116
> > all Linux kernel source with
> > Ubuntu patches
> > ii linux-source-5.15.0 5.15.0-116.126
> > all Linux kernel source for version
> > 5.15.0 with Ubuntu patches
> > ii linux-source-6.5.0 6.5.0-44.44~22.04.1
> > all Linux kernel source for version
>
> Your currently running `6.5.0-41-generic #41~22.04.2-Ubuntu`
>
> You have source installed for `6.5.0-44.44~22.04.1`. From your previous
> e-mail, there is also a linux image installed:
>
> ```
> linux-image-6.5.0-44-generic 6.5.0-44.44~22.04.1
> ```
>
> Assuming you are using grub, could you verify that one of the options
> for booting in `/boot/grub/grub.cfg` is linux 6.5.0-44.44. Look for the
> lines similar to the following inside a menu entry block:
>
> (Note my example is 5.15.0-116, but you will be lookng for 6.5.0-44 or
> 6.5.0-44.44)
>
> ```
> linux /boot/vmlinuz-5.15.0-116-generic root=....
> initrd /boot/initrd.img-5.15.0-116-generic
> ```
>
> If you have a grub entry for linux 6.5.0-44.44, could you reboot to run
> on the linux 6.5.0-44.44.
>
> After rebooting verify with `uname -a` that your kernel version is
> 6.5.0-44.44.
>
> Make sure you have unpacked the linux sources:
>
> ```
> # Approximating the version numbers, since I don't have 6.5.0 installed
> cd /usr/src/linux-source-6.5.0
> bunzip2 linux-source-6.5.0.tar.bz2
> tar -xf linux-source-6.5.0.tar
> ```
>
> Then retry the build with
> `KERNELDIR=/usr/src/linux-source-6.5.0/linux-source-6.5.0/`
>
> I haven't personally tried the full steps on an Ubuntu installation so
> you may still need to tweak a couple of things, but I think this should
> get you on the right path.
>
> Really the important thing is to make sure that the kernel source code
> matches the kernel you are running.
>
> > 6.5.0 with Ubuntu patches
> > *Could you run either `apt-cache policy` or `apt policy` and
> `apt-mark
> > showhold` in order to know which sources you have configured.*
> > For this part, /*apt-mark showhold*/ gave me nothing I don't know
> what
> > is means.
>
> `apt-mark showhold` will show any packages that are on hold - it is a
> way of marking packages that shouldn't be updated.
>
> As you have the linux-source for 6.5.0-44.44~22.04.1, I don't think it
> will be necessary to have the output of `apt-cache policy`.
>
> thanks,
> kienan
>
> >
> > I am looking at the links, but nothing seems to match my kernel
> version.
> >
> > Thanks for your assistance.
> >
> > Le mer. 17 juill. 2024, à 11 h 46, François Belias
> > <beliasossim at gmail.com <mailto:beliasossim at gmail.com>
> <mailto:beliasossim at gmail.com <mailto:beliasossim at gmail.com>>> a écrit :
> >
> > Hello, Kienan
> >
> >
> > *Right. Which kernel are you actually running at this time?
> You can
> > check
> > using: `uname -a`*
> > Linux philippe 6.5.0-41-generic #41~22.04.2-Ubuntu SMP
> > PREEMPT_DYNAMIC Mon Jun 3 11:32:55 UTC 2 x86_64 x86_64
> x86_64 GNU/Linux
> >
> > *Since you've got a few different images and headers
> installed, it is
> > possible to check that same info the linux-source package:
> `dpkg -l |
> > grep linux-source`.*
> > ii linux-source
> 5.15.0.116.116
> > all Linux kernel
> source
> > with Ubuntu patches
> > ii linux-source-5.15.0 5.15.0-116.126
> > all Linux kernel source for
> > version 5.15.0 with Ubuntu patches
> > ii linux-source-6.5.0 6.5.0-44.44~22.04.1
> > all Linux kernel source for
> > version 6.5.0 with Ubuntu patches
> > *Could you run either `apt-cache policy` or `apt policy` and
> `apt-mark
> > showhold` in order to know which sources you have configured.*
> > For this part, /*apt-mark showhold*/ gave me nothing I don't know
> > what is means.
> >
> > I am looking at the links, but nothing seems to match my kernel
> > version.
> >
> > Thanks for your assistance.
> >
> > Le mer. 17 juill. 2024, à 10 h 55, Kienan Stewart
> > <kstewart at efficios.com <mailto:kstewart at efficios.com>
> <mailto:kstewart at efficios.com <mailto:kstewart at efficios.com>>> a écrit :
> >
> > Hi François,
> >
> > On 7/16/24 4:34 PM, François Belias wrote:
> > > Hello Kienan,
> > >
> > > When I proceed with the following steps:
> > >
> > > ```
> > > # Warning: this removes all untracked content from the
> > lttng-modules
> > > directory
> > > cd /path/to/lttng-modules-source
> > > git clean -dxf
> > >
> > > make
> > > sudo make modules_install
> > > sudo depmod -a
> > > ```
> > >
> > > I encounter a warning in
> > > `/home/philippe/lttng-modules/src/probes/Kbuild:79`
> stating
> > "File
> > > ./arch/x86/kvm/lapic.h not found. Probe 'kvm'
> x86-specific is
> > disabled.
> > > Use full kernel source tree to enable it." Although
> > everything compiles
> > > fine and I'm able to load modules, when I execute
> `sudo lttng
> > list -k`,
> > > I don't see the tracepoints `kvm_entry` and
> `kvm_exit`, which
> > I believe
> > > should be available. Could this be due to not using my
> full
> > kernel tree
> > > during compilation?
> >
> >
> > Exactly. Those probes depend on having the full kernel
> source as
> > they
> > require private headers which aren't shipped as part of the
> > `linux-headers` package.
> >
> > >
> > > Here are the answers to your questions:
> > >
> > > - *Where did `/usr/src/linux-source-6.5.0` come from?*
> > > I initially attempted `sudo apt-get update && sudo
> apt-get
> > install
> > > linux-source`, but it gave me this answer
> */linux-source is
> > already the
> > > newest version (5.15.0.116.116)/*`, which doesn't match my
> > current
> > > kernel version. So, I downloaded it directly from
> kernel.org <http://kernel.org>
> > <http://kernel.org <http://kernel.org>>
> > > <http://kernel.org <http://kernel.org>
> <http://kernel.org <http://kernel.org>>> using this command:
> > `wget
> > >
> > https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.5.tar.xz`
> <https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.5.tar.xz>
> >
> <https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.5.tar.xz
> <https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.5.tar.xz>>
> > >
> >
> <https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.5.tar.xz`
> <https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.5.tar.xz>
> >
> <https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.5.tar.xz
> <https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.5.tar.xz>>>.
> > >
> >
> > Right. Which kernel are you actually running at this
> time? You
> > can check
> > using: `uname -a`
> >
> > In any case, Linux upstream 6.5 is not the same as
> Ubuntus's 6.5.
> >
> > > - *Which Linux distribution and version are you using?*
> > > Running `lsb_release -a` gives me:
> > > ```
> > > No LSB modules are available.
> > > Distributor ID: Ubuntu
> > > Description: Ubuntu 22.04.4 LTS
> > > Release: 22.04
> > > Codename: jammy
> > > ```
> > >
> > > - *Assuming your distribution has kernel packages,
> which do
> > you have
> > > installed, and do you have the corresponding kernel
> headers
> > package
> > > installed?*
> > >
> > > Running `dpkg -l | grep linux-image` shows these
> installed
> > packages:
> > > ```
> > > ii linux-image-5.15.0-116-generic
> 5.15.0-116.126
> > > amd64 Signed kernel image generic
> > > ii linux-image-6.5.0-28-generic
> > 6.5.0-28.29~22.04.1
> > > amd64 Signed kernel image generic
> > > ii linux-image-6.5.0-41-generic
> > 6.5.0-41.41~22.04.2
> > > amd64 Signed kernel image generic
> > > ii linux-image-6.5.0-44-generic
> > 6.5.0-44.44~22.04.1
> > > amd64 Signed kernel image generic
> > > ii linux-image-generic
> 5.15.0.116.116
> > > amd64 Generic Linux kernel image
> > > ii linux-image-generic-hwe-22.04
> > 6.5.0.44.44~22.04.1
> > > amd64 Generic Linux kernel image
> > > ```
> > >
> > > And `dpkg -l | grep linux-headers` shows these
> installed
> > header packages:
> > > ```
> > > ii linux-headers-5.15.0-116
> > 5.15.0-116.126 all
> > > Header files related to Linux kernel version 5.15.0
> > > ii linux-headers-5.15.0-116-generic
> 5.15.0-116.126
> > > amd64 Linux kernel headers for version 5.15.0 on
> 64 bit
> > x86 SMP
> > > ii linux-headers-6.5.0-28-generic
> > 6.5.0-28.29~22.04.1
> > > amd64 Linux kernel headers for version 6.5.0 on 64 bit
> > x86 SMP
> > > ii linux-headers-6.5.0-41-generic
> > 6.5.0-41.41~22.04.2
> > > amd64 Linux kernel headers for version 6.5.0 on 64 bit
> > x86 SMP
> > > ii linux-headers-6.5.0-44-generic
> > 6.5.0-44.44~22.04.1
> > > amd64 Linux kernel headers for version 6.5.0 on 64 bit
> > x86 SMP
> > > ii linux-headers-generic
> 5.15.0.116.116
> > > amd64 Generic Linux kernel headers
> > > ii linux-headers-generic-hwe-22.04
> > 6.5.0.44.44~22.04.1
> > > amd64 Generic Linux kernel headers
> > > ```
> > >
> >
> > Since you've got a few different images and headers
> installed,
> > it is
> > possible to check that same info the linux-source
> package: `dpkg
> > -l |
> > grep linux-source`.
> >
> > Maybe one thing tripping this process up is that you
> don't have
> > an apt
> > source that provides linux-source that matches the kernel you
> > are using.
> > Could you run either `apt-cache policy` or `apt policy` and
> > `apt-mark
> > showhold` in order to know which sources you have configured.
> >
> > There are also alternative sources for the full source tree:
> >
> > - You could download the specific package from
> > packages.ubuntu.com <http://packages.ubuntu.com>
> <http://packages.ubuntu.com <http://packages.ubuntu.com>>. See
> > https://packages.ubuntu.com <https://packages.ubuntu.com>
> <https://packages.ubuntu.com <https://packages.ubuntu.com>> and
> > use `apt install ./downloaded_package.deb`
> >
> > - You could clone the kernel source for Ubuntu and
> checkout the
> > appropriate tag. See
> >
> https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/jammy/refs/tags <https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/jammy/refs/tags> <https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/jammy/refs/tags <https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/jammy/refs/tags>>
> >
> > thanks,
> > kienan
> >
>
More information about the lttng-dev
mailing list