[lttng-dev] Profiling LTTng tracepoint latency on different arm platforms

[Mousa, Anas]

Hello,



I've recently profiled the latency of LTTng tracepoints on arm platforms,

using the follow sample program:



----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

static inline uint64_t get_time_nsec(void)

{

        struct timespec ts;



        if (caa_unlikely(clock_gettime(CLOCK_MONOTONIC, &ts))) {

                ts.tv_sec = 0;

                ts.tv_nsec = 0;

        }

        return ((uint64_t) ts.tv_sec * 1000000000ULL) + ts.tv_nsec;

}






int main(int argc, char *argv[])

{

    unsigned int i;

    int tp_num = 0;

    uint64_t total_time = 0;

    uint64_t now, nowz;



    if (argc > 1) {

        sscanf (argv[1],"%d",&tp_num);

    }

    for (i = 0; i < tp_num; i++) {

        now = get_time_nsec();

        lttng_ust_tracepoint(hello_world, my_first_tracepoint,

                             i, "some_str");

        nowz = get_time_nsec();

        total_time += (nowz - now);

    }



    if (tp_num) {

        printf("---------------------------Average TP time is %"PRIu64"---------------------------\n", total_time / tp_num);

    }

}

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------






I observed a big average latency variance on different platforms when tracing a high number (many thousands to millions) of tracepoints:



  *   [platform 1] with CPU info running a linux kernel based on Buildroot (4.19.273 aarch64 GNU/Linux):



BogoMIPS        : 187.50

Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid

CPU implementer   : 0x41

CPU architecture: 8

CPU variant       : 0x0

CPU part  : 0xd08

CPU revision      : 3



     *    Saw an average latency of 2-3usec


  *   [platform 2] with CPU info running a linux kernel based on Amazon Linux (4.14.294-220.533.amzn2.aarch64 aarch64 GNU/Linux):



BogoMIPS        : 243.75

Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp ssbs

CPU implementer   : 0x41

CPU architecture: 8

CPU variant       : 0x3

CPU part  : 0xd0c

CPU revision      : 1



     *   Saw an average latency of ~0.5usec







Are there any suggestions to root cause the high latency and potentially improve it on platform 1?



Thanks and best regards,

Anas.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.lttng.org/pipermail/lttng-dev/attachments/20230620/066b1a95/attachment-0001.htm>