[lttng-dev] Regarding LTTng Support for ARM NO-HZ

[Hongbo Zhang]

On 21 March 2014 04:21, Mathieu Desnoyers <mathieu.desnoyers at efficios.com>wrote:

> ------------------------------
>
> *From: *"Hongbo Zhang" <hongbo.zhang at linaro.org>
> *To: *"mathieu desnoyers" <mathieu.desnoyers at efficios.com>,
> lttng-dev at lists.lttng.org
> *Sent: *Tuesday, March 18, 2014 4:07:19 AM
> *Subject: *Regarding LTTng Support for ARM NO-HZ
>
>
> Hi Mathieu Desnoyers and all LTTng developers,
>
>
> Hi,
>
> This is Hongbo Zhang, working for Linaro. We have to enable the NO-HZ
> feature for some special case, and this leads to the question whether the
> LTTng can work without kernel ticks on ARM or not.
>
> I found out some clue that LTTng only supports x86 NO-HZ in an old link:
>
> http://git.lttng.org/?p=lttv.git;a=blob_plain;f=doc/developer/lttng-lttv-roadmap.html
>
>
> This is an old LTTng 0.x link, and does not apply to LTTng 2.x.
>
> I think the the LTTng doesn't support ARM NO-HZ, right?
>
>
> While lib ring buffer within lttng-modules has configuration modes to allow
> minimizing the impact on NO_HZ systems, the current lttng-modules 2.x
> is not setup to use those configuration options, so lttng-modules will
> wakeup
> cpus periodically to flush buffers (when kernel tracing is active).
>
> And I am going to add the ARM NO-HZ support to it now, so could you please
> give me some instructions for quick start?
>
>
> You will probably want to start by looking at the "read timer" option of
> channels, and see how it is handled within lttng-modules, and how it
> impacts
> the lib ring buffer client configuration with lttng-modules.
>
> what you probably want is lttng-modules:
>
> lib/ringbuffer/config.h
>
> RING_BUFFER_WAKEUP_BY_TIMER
> RING_BUFFER_WAKEUP_BY_WRITER
> (and compare the various wakeup modes)
>
> However, if you use wakeup by writer, this has impacts on the
> instrumentation
> coverage (e.g. you then can trigger issues when instrumenting some core
> scheduler
> code paths, NMIs, etc). So wakeup by timer is really the mode that allows
> being
> called from the entire kernel code base without risking having
> side-effects due to
> interaction of the wakeup mechanism with the rest of the kernel.
>
>
> 1. Which part of LTTng relies on kernel ticks? currently I only find the
> ring-bugger by "grep NO*HZ -i -r" command.
>
> ring_buffer_tick_nohz_callback() was a callback meant to be called by the
> in-kernel code that manages no hz. This was implemented when lib ring
> buffer
> was an in-kernel library (a patchset to the Linux kernel) rather than
> sitting within
> lttng-modules.
>
> When it was a in-kernel patchset, we could modify the nohz notifier code
> from
> the Linux kernel to notify the ring buffer that it enters or exits nohz
> mode. Unfortunately,
> there is no such notifier in the kernel today (and Thomas Gleixner refused
> to add one).
>
> So we need to add this notifier to the Linux kernel if we want to make the
> "timer-based"
> mode of the lib ring buffer work well on nohz kernels (meaning: don't wake
> up the CPUs
> that entered idle state needlessly).
>
> A possible work-around to try is to change the per-cpu timers within lib
> ring buffer into
> deferrable timers. This should be attempted too.
>
> Another possible solution would be to implement a global system wide timer
> rather than
> a per-cpu timer. This would lessen the impact of the timers on idle CPUs,
> at the expense
> of scalability. The loss of scalability is the reason why I did not
> implement this solution.
>
> 2. How does this part(ring-buffer and maybe others) relies on kernel
> ticks? can you give some explanations or do you have any document of
> it/them?
>
> I will look through the codes, but I think some clear explanations can
> make me go to the right way and speed up before reading so many codes
> without any dedicated target.
>
>
> lttng-modules lib ring buffer is the only part of lttng-modules that
> depends on timers,
> and therefore is the only part that should interact with NO HZ.
>
> Hopefully the above explanations shed some light on the topic.
>
> Please let me know if you have more questions,
>
>
Thank you Mathieu very much, I will look into the codes and raise questions
about further details/solutions if(likely) I have.

But quick questions for now:
I see there was an attempt to unify ring buffer for lttng, ftrace and perf,
but nowadays they are still using their own ring buffer respectively, am I
right? no further effort to unify it any more?

And by the way, do you have any idea about ftrace and perf, do they relies
on kernel ticks?
Thanks.


Thanks,
>
> Mathieu
>
>
> Thanks.
>
>
>
>
> --
> Mathieu Desnoyers
> EfficiOS Inc.
> http://www.efficios.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lttng.org/pipermail/lttng-dev/attachments/20140321/4b90fa3b/attachment.html>