<div dir="ltr">Hello Kienan,<br><br>When I proceed with the following steps:<br><br>```<br># Warning: this removes all untracked content from the lttng-modules directory<br>cd /path/to/lttng-modules-source<br>git clean -dxf<br><br>make<br>sudo make modules_install<br>sudo depmod -a<br>```<br><br>I encounter a warning in `/home/philippe/lttng-modules/src/probes/Kbuild:79` stating "File ./arch/x86/kvm/lapic.h not found. Probe 'kvm' x86-specific is disabled. Use full kernel source tree to enable it." Although everything compiles fine and I'm able to load modules, when I execute `sudo lttng list -k`, I don't see the tracepoints `kvm_entry` and `kvm_exit`, which I believe should be available. Could this be due to not using my full kernel tree during compilation?<br><br>Here are the answers to your questions:<br><br>- <b>Where did `/usr/src/linux-source-6.5.0` come from?</b><br> I initially attempted `sudo apt-get update && sudo apt-get install linux-source`, but it gave me this answer <b><i>linux-source is already the newest version (5.15.0.116.116)</i></b>`, which doesn't match my current kernel version. So, I downloaded it directly from <a href="http://kernel.org">kernel.org</a> using this command: `wget <a href="https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.5.tar.xz`">https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.5.tar.xz`</a>.<br><br>- <b>Which Linux distribution and version are you using?</b><br> Running `lsb_release -a` gives me:<br> ```<br> No LSB modules are available.<br> Distributor ID: Ubuntu<br> Description: Ubuntu 22.04.4 LTS<br> Release: 22.04<br> Codename: jammy<br> ```<br><br>- <b>Assuming your distribution has kernel packages, which do you have installed, and do you have the corresponding kernel headers package installed?</b><br><br> Running `dpkg -l | grep linux-image` shows these installed packages:<br> ```<br> ii linux-image-5.15.0-116-generic 5.15.0-116.126 amd64 Signed kernel image generic<br> ii linux-image-6.5.0-28-generic 6.5.0-28.29~22.04.1 amd64 Signed kernel image generic<br> ii linux-image-6.5.0-41-generic 6.5.0-41.41~22.04.2 amd64 Signed kernel image generic<br> ii linux-image-6.5.0-44-generic 6.5.0-44.44~22.04.1 amd64 Signed kernel image generic<br> ii linux-image-generic 5.15.0.116.116 amd64 Generic Linux kernel image<br> ii linux-image-generic-hwe-22.04 6.5.0.44.44~22.04.1 amd64 Generic Linux kernel image<br> ```<br><br> And `dpkg -l | grep linux-headers` shows these installed header packages:<br> ```<br> ii linux-headers-5.15.0-116 5.15.0-116.126 all Header files related to Linux kernel version 5.15.0<br> ii linux-headers-5.15.0-116-generic 5.15.0-116.126 amd64 Linux kernel headers for version 5.15.0 on 64 bit x86 SMP<br> ii linux-headers-6.5.0-28-generic 6.5.0-28.29~22.04.1 amd64 Linux kernel headers for version 6.5.0 on 64 bit x86 SMP<br> ii linux-headers-6.5.0-41-generic 6.5.0-41.41~22.04.2 amd64 Linux kernel headers for version 6.5.0 on 64 bit x86 SMP<br> ii linux-headers-6.5.0-44-generic 6.5.0-44.44~22.04.1 amd64 Linux kernel headers for version 6.5.0 on 64 bit x86 SMP<br> ii linux-headers-generic 5.15.0.116.116 amd64 Generic Linux kernel headers<br> ii linux-headers-generic-hwe-22.04 6.5.0.44.44~22.04.1 amd64 Generic Linux kernel headers<br> ```<br><br>Thanks.<br><br><div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Le mar. 16 juill. 2024, à 15 h 41, Kienan Stewart <<a href="mailto:kstewart@efficios.com">kstewart@efficios.com</a>> a écrit :<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi François,<br>
<br>
On 7/16/24 3:01 PM, François Belias wrote:<br>
> Hello Kienan,<br>
> <br>
> I've continued investigating my issue and discovered something new that <br>
> might shed some light. Here's the rundown:<br>
> <br>
> 1. To ensure my add-ons weren't causing problems, deleted all old <br>
> modules and I followed the documentation on the LTTng website <br>
> (<a href="https://lttng.org/docs/v2.13/#doc-building-from-source" rel="noreferrer" target="_blank">https://lttng.org/docs/v2.13/#doc-building-from-source</a> <br>
> <<a href="https://lttng.org/docs/v2.13/#doc-building-from-source" rel="noreferrer" target="_blank">https://lttng.org/docs/v2.13/#doc-building-from-source</a>>) and downloaded <br>
> `lttng-modules-2.13.13`.<br>
> <br>
> 2. Inside the downloaded folder, I successfully executed `make <br>
> KERNELDIR=/usr/src/linux-source-6.5.0` without any issues.<br>
> <br>
> 3. Subsequently, I ran `make KERNELDIR=/usr/src/linux-source-6.5.0 <br>
> modules_install`, which also completed successfully.<br>
> <br>
> 4. After that, I updated the module dependencies using `sudo depmod -a`, <br>
> which went fine.<br>
> <br>
> However, when I tried to load a module, such as `sudo modprobe <br>
> lttng-probe-kvm`, it reported that the module couldn't be found at <br>
> `/lib/modules/6.5.0-41-generic/`. Upon checking the logs, I confirmed <br>
> that step 3 had installed the modules in `/lib/modules/6.5.0/`.<br>
> <br>
> To address this, I modified step 3 to simply `make modules_install`, <br>
> which then installed the modules correctly at <br>
> `/lib/modules/6.5.0-41-generic/` and ran again step 4. But now, when <br>
> attempting to load a module, I'm encountering the error `modprobe: <br>
> ERROR: could not insert 'lttng_probe_kvm': Exec format error`.<br>
> <br>
> Do you have any insights or ideas on what might be causing this issue?<br>
> <br>
<br>
It sounds like /usr/src/linux-source-6.5.0 is not correct source for the <br>
kernel that you are currently running.<br>
<br>
When running `make modules_install` in your modified step 3 (without <br>
KERNELDIR set), the modules were installed for your current kernel <br>
directory as auto-detected by the build script.<br>
<br>
What happens if you do the following<br>
<br>
```<br>
unset KERNELDIR<br>
<br>
# Warning: this removes all untracked content from the lttng-modules <br>
directory<br>
cd /path/to/lttng-modules-source<br>
git clean -dxf<br>
<br>
make<br>
sudo make modules_install<br>
sudo depmod -a<br>
```<br>
<br>
KERNELDIR is set based on the current running kernel: `uname -r`. C.f. <br>
<a href="https://github.com/lttng/lttng-modules/blob/434e86eea846b33cad57f1d8105089905f541853/Makefile#L16C1-L16C50" rel="noreferrer" target="_blank">https://github.com/lttng/lttng-modules/blob/434e86eea846b33cad57f1d8105089905f541853/Makefile#L16C1-L16C50</a><br>
<br>
A couple of questions for you:<br>
<br>
- Where did /usr/src/linux-source-6.5.0 come from?<br>
- Which linux distribution + version are you using?<br>
- Assuming your distribution has kernel packages, which do you have <br>
installed, and do you have the corresponding kernel headers package <br>
installed?<br>
<br>
thanks,<br>
kienan<br>
<br>
> Thanks again for your assistance.<br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> Le mar. 16 juill. 2024, à 09 h 05, Kienan Stewart <<a href="mailto:kstewart@efficios.com" target="_blank">kstewart@efficios.com</a> <br>
> <mailto:<a href="mailto:kstewart@efficios.com" target="_blank">kstewart@efficios.com</a>>> a écrit :<br>
> <br>
> Hi François,<br>
> <br>
> On 7/15/24 6:10 PM, François Belias via lttng-dev wrote:<br>
> > Hello,<br>
> ><br>
> > I'm currently running Linux kernel version 6.5.0-41-generic and<br>
> > attempting to install LTTng from source. Here are the steps I<br>
> followed:<br>
> ><br>
> > - I cloned the LTTng git repository to my computer from<br>
> ><br>
> [<a href="https://github.com/lttng/lttng-modules.git%5D(https://github.com/lttng/lttng-modules.git)" rel="noreferrer" target="_blank">https://github.com/lttng/lttng-modules.git](https://github.com/lttng/lttng-modules.git)</a> <<a href="https://github.com/lttng/lttng-modules.git%5D(https://github.com/lttng/lttng-modules.git)" rel="noreferrer" target="_blank">https://github.com/lttng/lttng-modules.git%5D(https://github.com/lttng/lttng-modules.git)</a>> <<a href="https://github.com/lttng/lttng-modules.git%5D(https://github.com/lttng/lttng-modules.git)" rel="noreferrer" target="_blank">https://github.com/lttng/lttng-modules.git](https://github.com/lttng/lttng-modules.git)</a> <<a href="https://github.com/lttng/lttng-modules.git%5D(https://github.com/lttng/lttng-modules.git)" rel="noreferrer" target="_blank">https://github.com/lttng/lttng-modules.git%5D(https://github.com/lttng/lttng-modules.git)</a>>>.<br>
> > - Downloaded the kernel source tree corresponding to my kernel<br>
> version<br>
> > to enable compilation.<br>
> > - Executed `make KERNELDIR=/path/to/my/linux/source`, which was<br>
> > completed successfully.<br>
> > - Ran `make KERNELDIR=/path/to/my/linux/source modules_install`,<br>
> also<br>
> > without any issues.<br>
> > - Finally, I executed `sudo depmod -a`, which was completed<br>
> successfully.<br>
> ><br>
> > However, when I attempt to load a module with commands like `sudo<br>
> > modprobe lttng-ketprobes` or "sudo modprobe lttng-tracer", I<br>
> encounter<br>
> > the following errors:<br>
> ><br>
> > - For `lttng-ketprobes`: `modprobe: FATAL: Module lttng-ketprobes<br>
> not<br>
> > found in directory /lib/modules/6.5.0-41-generic`.<br>
> > - For `lttng-tracer`: `modprobe: ERROR: could not insert<br>
> 'lttng_tracer':<br>
> > Exec format error`.<br>
> <br>
> If you rebuilt modules on the master branch as of yesterday you may<br>
> need<br>
> to clean the LTTng modules install directory, rebuild, and reinstall.<br>
> <br>
> This is due to kprobes, kretprobes, and uprobes modules being included<br>
> in the tracer core module. C.f.<br>
> <a href="https://github.com/lttng/lttng-modules/commit/0c2c6606eb66cb5222ab7a2a4f8329fd5dfdef70" rel="noreferrer" target="_blank">https://github.com/lttng/lttng-modules/commit/0c2c6606eb66cb5222ab7a2a4f8329fd5dfdef70</a> <<a href="https://github.com/lttng/lttng-modules/commit/0c2c6606eb66cb5222ab7a2a4f8329fd5dfdef70" rel="noreferrer" target="_blank">https://github.com/lttng/lttng-modules/commit/0c2c6606eb66cb5222ab7a2a4f8329fd5dfdef70</a>><br>
> <br>
> You may want to stick with running lttng-modules 2.13.x which will not<br>
> have breaking changes merged in.<br>
> <br>
> The exec format error also could be due to modules that are installed<br>
> but built against a different version of the Linux kernel.<br>
> <br>
> A few things to check<br>
> <br>
> - If your kernel has been upgraded recently, ensure the new kernel is<br>
> the one actually running<br>
> - If the modules haven't been rebuilt since the upgrade, they should be<br>
> rebuilt and reinstalled<br>
> - As I recall, you are playing around with some addon modules - do any<br>
> of those modules define dependencies on the kprobes, kretprobes, or<br>
> uprobes modules which now are no longer produced by builds of the<br>
> master<br>
> branch<br>
> <br>
> Is there further information in dmesg or journalctl regarding the<br>
> module<br>
> loading?<br>
> <br>
> hope this helps.<br>
> <br>
> thanks,<br>
> kienan<br>
> <br>
> ><br>
> > I've confirmed that the modules exist but can't load them. I'm<br>
> unsure<br>
> > how to resolve these issues. Could someone please assist me?<br>
> ><br>
> > Thank you.<br>
> ><br>
> > _______________________________________________<br>
> > lttng-dev mailing list<br>
> > <a href="mailto:lttng-dev@lists.lttng.org" target="_blank">lttng-dev@lists.lttng.org</a> <mailto:<a href="mailto:lttng-dev@lists.lttng.org" target="_blank">lttng-dev@lists.lttng.org</a>><br>
> > <a href="https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev" rel="noreferrer" target="_blank">https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev</a><br>
> <<a href="https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev" rel="noreferrer" target="_blank">https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev</a>><br>
> <br>
</blockquote></div>