[lttng-dev] Feature request: dynamically load from prefix

[Norbert Lange]

Am Di., 5. Okt. 2021 um 20:28 Uhr schrieb Mathieu Desnoyers
<mathieu.desnoyers at efficios.com>:
>
> ----- On Jul 15, 2021, at 7:21 AM, lttng-dev lttng-dev at lists.lttng.org wrote:
>
> > Hello,
> >
> > The production rootfs should be untouched, ideally read-only,
> > for development/tests a subdirectory can be mounted (eg. /usr/local).
> > Idea is that the contents of that directory alone (and at most some
> > env variables)
> > should allow enabling development features.
> >
> > For lttng I would have wanted to add a library
> > '/usr/local/lib/libmyservice-tracepoints.so' with runpath
> > '/usr/local/lib' that would activate lttng tracing,
> > pulling in lttng libraries (ust, ust-tracepoint) from /usr/local/lib.
> >
> > There is a caveat though, unless 'libmyservice-tracepoints.so'' is
> > preloaded, the code in lttng/tracepoint.h will run constructor functions
> > to register the tracepoint probes, trying to dlopen the lttng-ust-tracepoint
> > library and fail at that because this is not in the library search paths.
> >
> > At a later time,  'libmyservice-tracepoints.so'' will be loaded, and
> > lttng-ust-tracepoint (along with lttng-ust) can be resolved. but the
> > tracepoints are not registered.
> >
> > So I guess what I would need is to either retrigger the registration
> > of tracepoints
> > (likely complicated with static and weak symbols potentially causing a mess), or
> > redirect the dlopen function.
> > Useful would be either try to find the library in /usr/local/lib or
> > use '/usr/local/lib/libmyservice-tracepoints.so''
> > instead of  lttng-ust-tracepoint (both have (dis)advantages).
> >
> > At any rate, I would welcome some customization macro.
>
> I'm currently working on improvements to the reporting of such scenario.
>
> See:
>
> https://review.lttng.org/c/lttng-ust/+/6480 tracepoints: print debug message when lttng-ust-tracepoint.so is not found
> https://review.lttng.org/c/lttng-ust/+/6484 tracepoints: increase dlopen failure message level from debug to critical
>
> With this in place, it should be easier for a lttng end-user to figure out that
> liblttng-ust-tracepoint.so cannot be found by dlopen() because it is not in the
> system's library search path.
>
> From that point, what is wrong with requesting the user to add the path towards
> liblttng-ust-tracepoint.so to the environment variable LD_LIBRARY_PATH when running
> the application ?

Not feasible if this is a read-only rootfs, service configuration cant
be modified.
the idea is to have some trickery to allow /usr/local mounted to a NFS
or small rw filesystem,
then drop the files (libmyservice-tracepoints.so and
liblttng-ust-tracepoint.so and services) there.
then, the app can be instructed to load
/usr/local/lib/libmyservice-tracepoints.so

/usr/local is not referenced anywhere in the rootfs and should not
affect anything,
if for ex I drop another libc.so.6 there it should not affect anything
normally running.
Ideally liblttng-ust-tracepoint.so could be loaded later and not
pulled in by the tracepoint stubs,
but I understand that this is complicated.

Any good reason not to provide some customization point like
LTTNG_TRACEPOINT_PROBE_DLOPEN
for the stuff you locally include into your build?
Would meet me half-way ;)

Norbert



>
> Thanks,
>
> Mathieu
>
> >
> > For illustration the current hack-around is following
> >
> > Norbert Lange
> >
> > #define TRACEPOINT_DEFINE
> > #define TRACEPOINT_PROBE_DYNAMIC_LINKAGE
> >
> > #include <dlfcn.h>
> >
> > static inline void *s_remap_dlopen(const char *localfilename, int
> > flags, unsigned prelen) {
> >    void *md = (dlopen)(localfilename + prelen, flags);
> >    return md ? md : (dlopen)(localfilename, flags);
> > }
> >
> > # ideally this would be LTTNG_TRACEPOINT_PROBE_DLOPEN instead of the dlopen mess
> > #define dlopen(x, y) s_remap_dlopen("/usr/local/lib/" x, y,
> > (unsigned)sizeof("/usr/local/lib/") - 1U)
> >
> > #include "trace_mytracepoints.h"
> > _______________________________________________
> > lttng-dev mailing list
> > lttng-dev at lists.lttng.org
> > https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>
> --
> Mathieu Desnoyers
> EfficiOS Inc.
> http://www.efficios.com