[lttng-dev] Odd behaviour of lttng 2.0 kernel event listing

[Thibault, Daniel]

De : David Goulet [mailto:david.goulet at polymtl.ca] 
>
> I just pushed a fix with the auto spawned session daemon but not related 
> to your problem.
>
> Is it possible for you to test with the latest git head of lttng-tools 
> because since the commit 9beed4c there is a large number of fixes!
>
> The session daemon sends a signal to the parent (here lttng client) when 
> it is ready to process commands. So, the module are loaded for sure at 
> this time.
>
> I'm unable to reproduce your problem... so what I can recommend is to 
> report *any* compilation error with the latest git head you have and 
> I'll see if it's related to your problem. You might want to check also 
> if all lttng-modules are unloaded when NO session daemon is running (# 
> lsmod | grep ltt).
>
> Also, with lttng-tools commit bcfa8a05018b339b9b5609fe975a3a7b29cc0c6e, 
> using --help/-h with any lttng commands does NOT spawned the session 
> daemon anymore.
>
> Please let me know if you are still having the problem with those 
> recommendations.

   Well, the good news is that the new head (lttng-tools-2.0-pre16+-76150f6 2012-01-18 17:21) compiles correctly (no warnings, no errors in bootstrap, configure, make and install logs).  The bad news is that it still misbehaves.  Here's my session log, edited for brevity:

$ sudo lttng list -k
Spawning a session daemon
Kernel events:
-------------
      sched_kthread_stop (type: tracepoint)
      sched_kthread_stop_ret (type: tracepoint)
      [...]
      kvm_ioapic_set_irq (type: tracepoint)
      kvm_msi_set_irq (type: tracepoint)
       (type: tracepoint) [disabled]
       (type: tracepoint) [disabled]
      [... 27 more instances ...]
       (type: tracepoint) [disabled]
       (type: tracepoint) [disabled]

$ sudo lttng list -k
Kernel events:
-------------
      sched_kthread_stop (type: tracepoint)
      sched_kthread_stop_ret (type: tracepoint)
      [...]
      kvm_ioapic_set_irq (type: tracepoint)
      kvm_msi_set_irq (type: tracepoint)
      kvm_ack_irq (type: tracepoint)
      kvm_mmio (type: tracepoint)
      [...]
      block_bio_remap (type: tracepoint)
      block_rq_remap (type: tracepoint)

   When I kill the lttng-sessiond daemon, all 17 lttng-modules are unloaded.

   I tried doing 'sudo modprobe lttng_probe_block' before calling 'sudo lttng list -k' to see if that wouldn't un-anonymise the block_* tracepoints (which I think are serviced by the lttng_probe_block module).  The module loaded along with a handful of other modules it depends on (ltt_relay, lttng_kretprobes, lttng_kprobes, lttng_ftrace, lib_ring_buffer) but 'sudo lttng list -k' yielded precisely the same output.

-----

   I'd like to take this opportunity to point out a problem when running lttng from an unelevated account:

$ lttng list -k
Spawning a session daemon
FATAL: Error inserting lttng_ftrace (/lib/modules/2.6.38-020638-generic/extra/probes/lttng-ftrace.ko): Operation not permitted
FATAL: Error inserting lttng_kprobes (/lib/modules/2.6.38-020638-generic/extra/probes/lttng-kprobes.ko): Operation not permitted
[... 14 more FATAL errors interleaved with 30 WARNINGs ...]
WARNING: Error inserting ltt_relay (/lib/modules/2.6.38-020638-generic/extra/ltt-relay.ko): Operation not permitted
FATAL: Error inserting lttng_probe_sched (/lib/modules/2.6.38-020638-generic/extra/probes/lttng-probe-sched.ko): Operation not permitted
Error: Unable to list kernel events
Error: Kernel tracer not available

   If the error is FATAL, why does lttng keep going?  The final error message could be improved to read something like "Error: Kernel tracer not available or not accessible at your privilege level", in order to at least hint at what the source of the problem is.

Daniel U. Thibault
R & D pour la défense Canada - Valcartier (RDDC Valcartier) / Defence R&D Canada - Valcartier (DRDC Valcartier)
Système de systèmes (SdS) / System of Systems (SoS)
Solutions informatiques et expérimentations (SIE) / Computing Solutions and Experimentations (CSE)
2459 Boul. Pie XI Nord
Québec, QC  G3J 1X5
CANADA
Vox : (418) 844-4000 x4245
Fax : (418) 844-4538
NAC: 918V QSDJ
Gouvernement du Canada / Government of Canada
<http://www.valcartier.drdc-rddc.gc.ca/>