[ltt-dev] lttv unable to execute textDump on MIPS multicore target

[Mathieu Desnoyers]

* David Daney (ddaney at caviumnetworks.com) wrote:
> On 02/14/2011 06:39 AM, Mathieu Desnoyers wrote:
>> [ Adding David Daney to CC list, as he may have info on Octeon CPUs with
>>    non-synchronized TSCs ]
>>
>> * Naresh Bhat (nareshgbhat at gmail.com) wrote:
>>> Hi Mathieu,
>>>
>>> I have backported the patch and now TSC is enabled in my kernel and also I
>>> am able to boot the kernel.  But one thing I have observed is between single
>>> core and multiple core boot logs is that  as following
>>>
>
> I missed the patch.
>
> However, I would note that at the time I submitted my  MIPS patches to  
> Mathieu, LTTng seemed to be working well on multi-CPU Octeon systems.
>
>
> [...]
>>
>> timex.h:
>>
>> static inline void write_tsc(u32 val1, u32 val2)
>> {
>> #ifndef CONFIG_CPU_CAVIUM_OCTEON
>>          write_c0_count(val1);
>> #else
>> 	write_c0_cvmcount(((unsigned long) val2<<  32UL) | (unsigned long) val1);
>> #endif
>>          /* Arrange for an interrupt in a short while */
>>          write_c0_compare(read_c0_count() + DELAY_INTERRUPT);
>> }
>>
>
> This is not correct.  The main clocksource as well as sched_clock()  
> which is used by ftrace and the scheduler are all driven by c0_cvmcount.  
>  We take great care to properly synchronize this counter across all CPUs 
> when they are brought on-line.  Anything that sets this counter can only 
> make the synchronization worse, not better.

That's weird. Then I wonder why LTTng TSC synchronicity checker
complains about non-sync'd TSCs. The code is in kernel/time/tsc-sync.c.
It uses get_cycles() on each pair of cores. It gets run when a flag is
raised in memory for both cores of the pair.......

Ahhhh! I think I found it!

Naresh, can you try the following patch ?


MIPS: octeon fix get_cycles

Make sure get_cycles(), used by kernel/time/tsc-sync.c TSC synchronicity
checker, works fine on octeon by using the full 64-bits.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers at efficios.com>
---
 arch/mips/Kconfig             |   11 +++++++++--
 arch/mips/include/asm/timex.h |   27 ++++++++++++++++++++++++++-
 2 files changed, 35 insertions(+), 3 deletions(-)

Index: linux-2.6-lttng/arch/mips/Kconfig
===================================================================
--- linux-2.6-lttng.orig/arch/mips/Kconfig
+++ linux-2.6-lttng/arch/mips/Kconfig
@@ -1949,9 +1949,16 @@ config CPU_R4400_WORKAROUNDS
 config HAVE_GET_CYCLES_32
 	def_bool y
 	depends on !CPU_R4400_WORKAROUNDS
+	depends on !CPU_CAVIUM_OCTEON
 	select HAVE_TRACE_CLOCK
-	select HAVE_TRACE_CLOCK_32_TO_64 if (!CPU_CAVIUM_OCTEON)
-	select HAVE_UNSYNCHRONIZED_TSC if (!CPU_CAVIUM_OCTEON)
+	select HAVE_TRACE_CLOCK_32_TO_64
+	select HAVE_UNSYNCHRONIZED_TSC
+
+config HAVE_GET_CYCLES
+	def_bool y
+	depends on CPU_CAVIUM_OCTEON
+	select HAVE_TRACE_CLOCK
+	select HAVE_UNSYNCHRONIZED_TSC
 
 #
 # - Highmem only makes sense for the 32-bit kernel.
Index: linux-2.6-lttng/arch/mips/include/asm/timex.h
===================================================================
--- linux-2.6-lttng.orig/arch/mips/include/asm/timex.h
+++ linux-2.6-lttng/arch/mips/include/asm/timex.h
@@ -42,9 +42,32 @@ extern unsigned int mips_hpt_frequency;
  * will result in the timer interrupt getting lost.
  */
 
+#ifdef CONFIG_HAVE_GET_CYCLES
+typedef unsigned int cycles_t;
+
+static inline cycles_t get_cycles(void)
+{
+	return read_c0_cvmcount();
+}
+
+static inline void get_cycles_barrier(void)
+{
+}
+
+static inline cycles_t get_cycles_rate(void)
+{
+	return mips_hpt_frequency;
+}
+
+extern int test_tsc_synchronization(void);
+extern int _tsc_is_sync;
+static inline int tsc_is_sync(void)
+{
+	return _tsc_is_sync;
+}
+#elif defined(CONFIG_HAVE_GET_CYCLES_32)
 typedef unsigned int cycles_t;
 
-#ifdef CONFIG_HAVE_GET_CYCLES_32
 static inline cycles_t get_cycles(void)
 {
 	return read_c0_count();
@@ -66,6 +89,8 @@ static inline int tsc_is_sync(void)
 	return _tsc_is_sync;
 }
 #else
+typedef unsigned int cycles_t;
+
 static inline cycles_t get_cycles(void)
 {
 	return 0;




-- 
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com