[lttng-dev] Odd behaviour of lttng 2.0 kernel event listing

[David Goulet]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Daniel,

On 12-01-18 02:12 PM, Thibault, Daniel wrote:
> 
>    Well, the good news is that the new head (lttng-tools-2.0-pre16+-76150f6 2012-01-18 17:21) compiles correctly (no warnings, no errors in bootstrap, configure, make and install logs).  The bad news is that it still misbehaves.  Here's my session log, edited for brevity:
> 

Hmmmm... this is really unusual. I'll try to test this issue on a 2.6.38 kernel
because apart from that, I have no idea what can cause this. We can't reproduce
it here... Some of my dev. is on Lucid 10.04 and Debian testing so I doubt that
the Ubuntu version makes difference here. I'll let you know for sure.

> $ sudo lttng list -k
> Spawning a session daemon
> Kernel events:
> -------------
>       sched_kthread_stop (type: tracepoint)
>       sched_kthread_stop_ret (type: tracepoint)
>       [...]
>       kvm_ioapic_set_irq (type: tracepoint)
>       kvm_msi_set_irq (type: tracepoint)
>        (type: tracepoint) [disabled]
>        (type: tracepoint) [disabled]
>       [... 27 more instances ...]
>        (type: tracepoint) [disabled]
>        (type: tracepoint) [disabled]
> 
> $ sudo lttng list -k
> Kernel events:
> -------------
>       sched_kthread_stop (type: tracepoint)
>       sched_kthread_stop_ret (type: tracepoint)
>       [...]
>       kvm_ioapic_set_irq (type: tracepoint)
>       kvm_msi_set_irq (type: tracepoint)
>       kvm_ack_irq (type: tracepoint)
>       kvm_mmio (type: tracepoint)
>       [...]
>       block_bio_remap (type: tracepoint)
>       block_rq_remap (type: tracepoint)
> 
>    When I kill the lttng-sessiond daemon, all 17 lttng-modules are unloaded.
> 
>    I tried doing 'sudo modprobe lttng_probe_block' before calling 'sudo lttng list -k' to see if that wouldn't un-anonymise the block_* tracepoints (which I think are serviced by the lttng_probe_block module).  The module loaded along with a handful of other modules it depends on (ltt_relay, lttng_kretprobes, lttng_kprobes, lttng_ftrace, lib_ring_buffer) but 'sudo lttng list -k' yielded precisely the same output.
> 
> -----
> 
>    I'd like to take this opportunity to point out a problem when running lttng from an unelevated account:
> 
> $ lttng list -k
> Spawning a session daemon
> FATAL: Error inserting lttng_ftrace (/lib/modules/2.6.38-020638-generic/extra/probes/lttng-ftrace.ko): Operation not permitted
> FATAL: Error inserting lttng_kprobes (/lib/modules/2.6.38-020638-generic/extra/probes/lttng-kprobes.ko): Operation not permitted
> [... 14 more FATAL errors interleaved with 30 WARNINGs ...]
> WARNING: Error inserting ltt_relay (/lib/modules/2.6.38-020638-generic/extra/ltt-relay.ko): Operation not permitted
> FATAL: Error inserting lttng_probe_sched (/lib/modules/2.6.38-020638-generic/extra/probes/lttng-probe-sched.ko): Operation not permitted
> Error: Unable to list kernel events
> Error: Kernel tracer not available
> 
>    If the error is FATAL, why does lttng keep going?  The final error message could be improved to read something like "Error: Kernel tracer not available or not accessible at your privilege level", in order to at least hint at what the source of the problem is.
> 

I did fix that with latest commit :

Author: David Goulet <dgoulet at efficios.com>
Date:   Thu Jan 19 13:43:57 2012 -0500

    Don't init kernel tracer if not root (UID=0)

    No point of trying to init kernel tracer if the session daemon is not
    root. At this time, only UID=0 can load kernel modules and interact with
    the lttng kernel tracer.

    Reported-by: Daniel Thibault <daniel.thibault at drdc-rddc.gc.ca>
    Signed-off-by: David Goulet <dgoulet at efficios.com>

Thanks of the report!

Note that after this commit, I'll release lttng-tools pre17 and completely
feature freeze it until February 16th 2012. We are going into stable mode aiming
at the stable packages for Ubuntu next LTS.

Cheers!
David

> Daniel U. Thibault
> R & D pour la défense Canada - Valcartier (RDDC Valcartier) / Defence R&D Canada - Valcartier (DRDC Valcartier)
> Système de systèmes (SdS) / System of Systems (SoS)
> Solutions informatiques et expérimentations (SIE) / Computing Solutions and Experimentations (CSE)
> 2459 Boul. Pie XI Nord
> Québec, QC  G3J 1X5
> CANADA
> Vox : (418) 844-4000 x4245
> Fax : (418) 844-4538
> NAC: 918V QSDJ
> Gouvernement du Canada / Government of Canada
> <http://www.valcartier.drdc-rddc.gc.ca/>
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iQEcBAEBAgAGBQJPGGcaAAoJEELoaioR9I028EgH/217mx6gMoPm07ylxWwUBbg1
J5FfpdIGCFiuspT38zUXhlM3+J+W4oMY8giONGTlVuBkdK34PWNflXf0QyBfg3XW
i1ZmBBLGg/3vVbjGA74MGrYDVIUUr7Cpm+cycM2XD9k7xhe3RWbPdzP7JY8RWmzH
IqKDUxSvPHp9GnbNaRHdZ1G6nF89w6xXIHO5gaMJP4087AhGvDo8m52nCHBXIEJA
+uaB49NhvoOES5LPTxOJbaYPEpLgzEdGGjx1uriDX3BGGnfPRMmAS27Nf+4iiusI
H7B6Oye104hTZFqR4a/15T0v+FKR3dlGZvJga2MBHqiNoDRX5CuIohHBG0YDsvk=
=lD76
-----END PGP SIGNATURE-----