[lttng-dev] loading modules with modprobe
François Belias
beliasossim at gmail.com
Thu Jul 18 15:46:50 EDT 2024
Hello Kienan,
my issue has been fixed. thanks for your help
Le jeu. 18 juill. 2024, à 10 h 58, Kienan Stewart <kstewart at efficios.com> a
écrit :
> A couple of addendums inline below
>
> On 7/18/24 10:47 AM, Kienan Stewart via lttng-dev wrote:
> > Hi François,
> >
> > The dmesg excerpts are spaced by a number of minutes each, are they from
> > different attempts to build and load the modules?
> >
> > On 7/17/24 8:44 PM, François Belias wrote:
> >> Hello Kienan,
> >>
> >> Thanks to your advice, I managed to load some modules. However, I
> >> encountered errors like these when running `dmesg | grep lttng`:
> >>
> >> ```
> >> [ 218.043672] module lttng_kretprobes: .gnu.linkonce.this_module
> >> section size must match the kernel's built struct module size at run
> time
> >
> > This sounds like the version of the module is built against the wrong
> > kernel headers or source.
>
> Also, are you using lttng-modules master or did you swap back to
> stable-2.13?
>
> If you're still on master and have the most recent changes
> lttng_kretprobes isn't required anymore. You will also need lttng-tools
> from master with the corresponding changes so that it doesn't try to
> load lttng_kretprobes.
>
> >
> >> [ 650.560883] lttng_kretprobes: module verification failed: signature
> >> and/or required key missing - tainting kernel
> >
> > Is your system running with secure boot enabled? If so, you either need
> > to disable secure boot or sign the built modules with a MOK key that is
> > correctly enrolled.
> >
> > See the following documentation:
> >
> > - https://lttng.org/docs/v2.13/#doc-linux-kernel-sig
> > - https://www.kernel.org/doc/html/latest/admin-guide/module-signing.html
> > - https://github.com/dell/dkms
> > -
> >
> https://gitlab.com/linux-kernel/stable/-/blob/master/kernel/module/main.c#L2064
> >
> > Whatever you did for this test is pretty closed to working. The previous
> > errors with the section size check happen earlier in the module load
> > process. This check seems to be one of the last, c.f.
> > https://elixir.bootlin.com/linux/v6.10/source/kernel/module/main.c#L2908
> >
> >> [ 1468.329988] module lttng_ring_buffer_client_discard:
> >> .gnu.linkonce.this_module section size must match the kernel's built
> >> struct module size at run time
> >> ```
> >>
> >> I'm unsure of their meaning. After loading my module, how can I be
> >> certain that my tracepoints are active? I read online that people use
> >> `sudo lttng list -k` for this purpose, but when I tried it, I got the
> >> error: "Error: Unable to list kernel events: Kernel tracer not
> >> available". Is there another way?
> >
> > If you're still getting "kernel tracer not available" then either
> > required kernel modules haven't been loaded or you started lttng-session
> > with `--no-kernel` option. `lttng-sessiond -vvv`, dmesg, and/or
> > journalctl should have information on which modules failed to load.
> >
>
> I know you're doing `sudo lttng list -k`, but the basic checks still apply:
>
> - is the lttng-sessiond running as the root user
> - (when running without sudo) does the tracing group exist, and is your
> user a member of that group
>
> > kienan
> >
> >>
> >> Thank you for your assistance.
> >>
> >> Le mer. 17 juill. 2024, à 12 h 08, Kienan Stewart
> >> <kstewart at efficios.com <mailto:kstewart at efficios.com>> a écrit :
> >>
> >> Hi François,
> >>
> >> On 7/17/24 11:47 AM, François Belias wrote:
> >> > Hello, Kienan
> >> >
> >> >
> >> > *Right. Which kernel are you actually running at this time? You
> >> can check
> >> > using: `uname -a`*
> >> > Linux philippe 6.5.0-41-generic #41~22.04.2-Ubuntu SMP
> >> PREEMPT_DYNAMIC
> >> > Mon Jun 3 11:32:55 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
> >> >
> >> > *Since you've got a few different images and headers installed,
> >> it is
> >> > possible to check that same info the linux-source package:
> >> `dpkg -l |
> >> > grep linux-source`.*
> >> > ii linux-source 5.15.0.116.116
> >> > all Linux kernel source
> >> with
> >> > Ubuntu patches
> >> > ii linux-source-5.15.0 5.15.0-116.126
> >> > all Linux kernel source for
> >> version
> >> > 5.15.0 with Ubuntu patches
> >> > ii linux-source-6.5.0 6.5.0-44.44~22.04.1
> >> > all Linux kernel source for
> >> version
> >>
> >> Your currently running `6.5.0-41-generic #41~22.04.2-Ubuntu`
> >>
> >> You have source installed for `6.5.0-44.44~22.04.1`. From your
> >> previous
> >> e-mail, there is also a linux image installed:
> >>
> >> ```
> >> linux-image-6.5.0-44-generic 6.5.0-44.44~22.04.1
> >> ```
> >>
> >> Assuming you are using grub, could you verify that one of the
> options
> >> for booting in `/boot/grub/grub.cfg` is linux 6.5.0-44.44. Look
> >> for the
> >> lines similar to the following inside a menu entry block:
> >>
> >> (Note my example is 5.15.0-116, but you will be lookng for
> >> 6.5.0-44 or
> >> 6.5.0-44.44)
> >>
> >> ```
> >> linux /boot/vmlinuz-5.15.0-116-generic root=....
> >> initrd /boot/initrd.img-5.15.0-116-generic
> >> ```
> >>
> >> If you have a grub entry for linux 6.5.0-44.44, could you reboot
> >> to run
> >> on the linux 6.5.0-44.44.
> >>
> >> After rebooting verify with `uname -a` that your kernel version is
> >> 6.5.0-44.44.
> >>
> >> Make sure you have unpacked the linux sources:
> >>
> >> ```
> >> # Approximating the version numbers, since I don't have 6.5.0
> >> installed
> >> cd /usr/src/linux-source-6.5.0
> >> bunzip2 linux-source-6.5.0.tar.bz2
> >> tar -xf linux-source-6.5.0.tar
> >> ```
> >>
> >> Then retry the build with
> >> `KERNELDIR=/usr/src/linux-source-6.5.0/linux-source-6.5.0/`
> >>
> >> I haven't personally tried the full steps on an Ubuntu
> >> installation so
> >> you may still need to tweak a couple of things, but I think this
> >> should
> >> get you on the right path.
> >>
> >> Really the important thing is to make sure that the kernel source
> >> code
> >> matches the kernel you are running.
> >>
> >> > 6.5.0 with Ubuntu patches
> >> > *Could you run either `apt-cache policy` or `apt policy` and
> >> `apt-mark
> >> > showhold` in order to know which sources you have configured.*
> >> > For this part, /*apt-mark showhold*/ gave me nothing I don't know
> >> what
> >> > is means.
> >>
> >> `apt-mark showhold` will show any packages that are on hold - it is
> a
> >> way of marking packages that shouldn't be updated.
> >>
> >> As you have the linux-source for 6.5.0-44.44~22.04.1, I don't
> >> think it
> >> will be necessary to have the output of `apt-cache policy`.
> >>
> >> thanks,
> >> kienan
> >>
> >> >
> >> > I am looking at the links, but nothing seems to match my kernel
> >> version.
> >> >
> >> > Thanks for your assistance.
> >> >
> >> > Le mer. 17 juill. 2024, à 11 h 46, François Belias
> >> > <beliasossim at gmail.com <mailto:beliasossim at gmail.com>
> >> <mailto:beliasossim at gmail.com <mailto:beliasossim at gmail.com>>> a
> >> écrit :
> >> >
> >> > Hello, Kienan
> >> >
> >> >
> >> > *Right. Which kernel are you actually running at this time?
> >> You can
> >> > check
> >> > using: `uname -a`*
> >> > Linux philippe 6.5.0-41-generic #41~22.04.2-Ubuntu SMP
> >> > PREEMPT_DYNAMIC Mon Jun 3 11:32:55 UTC 2 x86_64 x86_64
> >> x86_64 GNU/Linux
> >> >
> >> > *Since you've got a few different images and headers
> >> installed, it is
> >> > possible to check that same info the linux-source package:
> >> `dpkg -l |
> >> > grep linux-source`.*
> >> > ii linux-source 5.15.0.116.116
> >> > all Linux kernel
> >> source
> >> > with Ubuntu patches
> >> > ii linux-source-5.15.0 5.15.0-116.126
> >> > all Linux kernel
> >> source for
> >> > version 5.15.0 with Ubuntu patches
> >> > ii linux-source-6.5.0
> >> 6.5.0-44.44~22.04.1
> >> > all Linux kernel source
> >> for
> >> > version 6.5.0 with Ubuntu patches
> >> > *Could you run either `apt-cache policy` or `apt policy` and
> >> `apt-mark
> >> > showhold` in order to know which sources you have
> configured.*
> >> > For this part, /*apt-mark showhold*/ gave me nothing I
> >> don't know
> >> > what is means.
> >> >
> >> > I am looking at the links, but nothing seems to match my
> >> kernel
> >> > version.
> >> >
> >> > Thanks for your assistance.
> >> >
> >> > Le mer. 17 juill. 2024, à 10 h 55, Kienan Stewart
> >> > <kstewart at efficios.com <mailto:kstewart at efficios.com>
> >> <mailto:kstewart at efficios.com <mailto:kstewart at efficios.com>>> a
> >> écrit :
> >> >
> >> > Hi François,
> >> >
> >> > On 7/16/24 4:34 PM, François Belias wrote:
> >> > > Hello Kienan,
> >> > >
> >> > > When I proceed with the following steps:
> >> > >
> >> > > ```
> >> > > # Warning: this removes all untracked content from the
> >> > lttng-modules
> >> > > directory
> >> > > cd /path/to/lttng-modules-source
> >> > > git clean -dxf
> >> > >
> >> > > make
> >> > > sudo make modules_install
> >> > > sudo depmod -a
> >> > > ```
> >> > >
> >> > > I encounter a warning in
> >> > > `/home/philippe/lttng-modules/src/probes/Kbuild:79`
> >> stating
> >> > "File
> >> > > ./arch/x86/kvm/lapic.h not found. Probe 'kvm'
> >> x86-specific is
> >> > disabled.
> >> > > Use full kernel source tree to enable it." Although
> >> > everything compiles
> >> > > fine and I'm able to load modules, when I execute
> >> `sudo lttng
> >> > list -k`,
> >> > > I don't see the tracepoints `kvm_entry` and
> >> `kvm_exit`, which
> >> > I believe
> >> > > should be available. Could this be due to not using my
> >> full
> >> > kernel tree
> >> > > during compilation?
> >> >
> >> >
> >> > Exactly. Those probes depend on having the full kernel
> >> source as
> >> > they
> >> > require private headers which aren't shipped as part of
> >> the
> >> > `linux-headers` package.
> >> >
> >> > >
> >> > > Here are the answers to your questions:
> >> > >
> >> > > - *Where did `/usr/src/linux-source-6.5.0` come from?*
> >> > > I initially attempted `sudo apt-get update && sudo
> >> apt-get
> >> > install
> >> > > linux-source`, but it gave me this answer
> >> */linux-source is
> >> > already the
> >> > > newest version (5.15.0.116.116)/*`, which doesn't
> >> match my
> >> > current
> >> > > kernel version. So, I downloaded it directly from
> >> kernel.org <http://kernel.org>
> >> > <http://kernel.org <http://kernel.org>>
> >> > > <http://kernel.org <http://kernel.org>
> >> <http://kernel.org <http://kernel.org>>> using this command:
> >> > `wget
> >> > >
> >> > https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.5.tar.xz`
> <https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.5.tar.xz>
> >> <https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.5.tar.xz>
> >> >
> >> <https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.5.tar.xz
> >> <https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.5.tar.xz>>
> >> > >
> >> >
> >> <https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.5.tar.xz`
> <https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.5.tar.xz>
> >> <https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.5.tar.xz>
> >> >
> >> <https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.5.tar.xz
> >> <https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.5.tar.xz>>>.
> >> > >
> >> >
> >> > Right. Which kernel are you actually running at this
> >> time? You
> >> > can check
> >> > using: `uname -a`
> >> >
> >> > In any case, Linux upstream 6.5 is not the same as
> >> Ubuntus's 6.5.
> >> >
> >> > > - *Which Linux distribution and version are you
> using?*
> >> > > Running `lsb_release -a` gives me:
> >> > > ```
> >> > > No LSB modules are available.
> >> > > Distributor ID: Ubuntu
> >> > > Description: Ubuntu 22.04.4 LTS
> >> > > Release: 22.04
> >> > > Codename: jammy
> >> > > ```
> >> > >
> >> > > - *Assuming your distribution has kernel packages,
> >> which do
> >> > you have
> >> > > installed, and do you have the corresponding kernel
> >> headers
> >> > package
> >> > > installed?*
> >> > >
> >> > > Running `dpkg -l | grep linux-image` shows these
> >> installed
> >> > packages:
> >> > > ```
> >> > > ii linux-image-5.15.0-116-generic
> >> 5.15.0-116.126
> >> > > amd64 Signed kernel image generic
> >> > > ii linux-image-6.5.0-28-generic
> >> > 6.5.0-28.29~22.04.1
> >> > > amd64 Signed kernel image generic
> >> > > ii linux-image-6.5.0-41-generic
> >> > 6.5.0-41.41~22.04.2
> >> > > amd64 Signed kernel image generic
> >> > > ii linux-image-6.5.0-44-generic
> >> > 6.5.0-44.44~22.04.1
> >> > > amd64 Signed kernel image generic
> >> > > ii linux-image-generic 5.15.0.116.116
> >> > > amd64 Generic Linux kernel image
> >> > > ii linux-image-generic-hwe-22.04
> >> > 6.5.0.44.44~22.04.1
> >> > > amd64 Generic Linux kernel image
> >> > > ```
> >> > >
> >> > > And `dpkg -l | grep linux-headers` shows these
> >> installed
> >> > header packages:
> >> > > ```
> >> > > ii linux-headers-5.15.0-116
> >> > 5.15.0-116.126 all
> >> > > Header files related to Linux kernel version
> >> 5.15.0
> >> > > ii linux-headers-5.15.0-116-generic
> >> 5.15.0-116.126
> >> > > amd64 Linux kernel headers for version 5.15.0 on
> >> 64 bit
> >> > x86 SMP
> >> > > ii linux-headers-6.5.0-28-generic
> >> > 6.5.0-28.29~22.04.1
> >> > > amd64 Linux kernel headers for version 6.5.0 on
> >> 64 bit
> >> > x86 SMP
> >> > > ii linux-headers-6.5.0-41-generic
> >> > 6.5.0-41.41~22.04.2
> >> > > amd64 Linux kernel headers for version 6.5.0 on
> >> 64 bit
> >> > x86 SMP
> >> > > ii linux-headers-6.5.0-44-generic
> >> > 6.5.0-44.44~22.04.1
> >> > > amd64 Linux kernel headers for version 6.5.0 on
> >> 64 bit
> >> > x86 SMP
> >> > > ii linux-headers-generic 5.15.0.116.116
> >> > > amd64 Generic Linux kernel headers
> >> > > ii linux-headers-generic-hwe-22.04
> >> > 6.5.0.44.44~22.04.1
> >> > > amd64 Generic Linux kernel headers
> >> > > ```
> >> > >
> >> >
> >> > Since you've got a few different images and headers
> >> installed,
> >> > it is
> >> > possible to check that same info the linux-source
> >> package: `dpkg
> >> > -l |
> >> > grep linux-source`.
> >> >
> >> > Maybe one thing tripping this process up is that you
> >> don't have
> >> > an apt
> >> > source that provides linux-source that matches the
> >> kernel you
> >> > are using.
> >> > Could you run either `apt-cache policy` or `apt policy`
> >> and
> >> > `apt-mark
> >> > showhold` in order to know which sources you have
> >> configured.
> >> >
> >> > There are also alternative sources for the full source
> >> tree:
> >> >
> >> > - You could download the specific package from
> >> > packages.ubuntu.com <http://packages.ubuntu.com>
> >> <http://packages.ubuntu.com <http://packages.ubuntu.com>>. See
> >> > https://packages.ubuntu.com <https://packages.ubuntu.com>
> >> <https://packages.ubuntu.com <https://packages.ubuntu.com>> and
> >> > use `apt install ./downloaded_package.deb`
> >> >
> >> > - You could clone the kernel source for Ubuntu and
> >> checkout the
> >> > appropriate tag. See
> >> >
> >>
> >>
> https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/jammy/refs/tags
> <
> https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/jammy/refs/tags>
> <
> https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/jammy/refs/tags
> <
> https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/jammy/refs/tags
> >>
> >> >
> >> > thanks,
> >> > kienan
> >> >
> >>
> > _______________________________________________
> > lttng-dev mailing list
> > lttng-dev at lists.lttng.org
> > https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.lttng.org/pipermail/lttng-dev/attachments/20240718/4f9ef84c/attachment-0001.htm>
More information about the lttng-dev
mailing list