[lttng-dev] Fun with glibc

[Thibault, Daniel]

   Let's assume I'm trying to instrument glibc.  Making glibc is tricky by itself, but the following works under Ubuntu 14.04 and glibc 2.15 (obtained from ftp.gnu.org/gnu/glibc/). I have deployed glibc-2.15.tar.gz to a local ./glibc-2.15 folder and created a sibling ./glibc-build folder:

glibc-build$ ../glibc-2.15/configure CFLAGS="-O2 -U_FORTIFY_SOURCE -fno-stack-protector" --prefix=/home/<user>/<path>/usr-local --with-pkgversion=user-glibc &> configure.log
glibc-build$ make &> make.log

   Now I take a "simple" glibc service, such as printf (glibc-2.15/stdio-common/printf.c), and try to instrument it.  I modify printf.c like this:

#include <libioP.h>
#include <stdarg.h>
#include <stdio.h>

//THREE NEW LINES
#define TRACEPOINT_DEFINE
#define TRACEPOINT_PROBE_DYNAMIC_LINKAGE
#include "printf-tp.h"

#undef printf

/* Write formatted output to stdout from the format string FORMAT.  */
/* VARARGS1 */
int
__printf (const char *format, ...)
{
  va_list arg;
  int done;

//ONE NEW LINE
  tracepoint(libc_tracer, printf);
  va_start (arg, format);
  done = vfprintf (stdout, format, arg);
  va_end (arg);

  return done;
}

   The tracepoint is a simple one.

glibc-2.15/stdio-common/printf-tp.c :

#define TRACEPOINT_CREATE_PROBES
#include "printf-tp.h"

glibc-2.15/stdio-common/printf-tp.h :

<usual preface>
TRACEPOINT_EVENT(
	libc_tracer,
	printf,
	TP_ARGS(),
	TP_FIELDS()
)
TRACEPOINT_LOGLEVEL(
	libc_tracer,
	printf,
	TRACE_INFO)
<usual postface>

   This is then compiled manually in place:

glibc-2.15/stdio-common$ gcc -I. -fPIC -c -o libprintf-tp.o printf-tp.c
glibc-2.15/stdio-common$ gcc -shared -Wl,-soname,libprintf-tp.so.1 -Wl,-no-as-needed -o libprintf-tp.so.1.0 -L/usr/local/lib -llttng-ust libprintf-tp.o
glibc-2.15/stdio-common$ ln -sf libprintf-tp.so.1.0 libprintf-tp.so.1
glibc-2.15/stdio-common$ ln -sf libprintf-tp.so.1 libprintf-tp.so

   Pretty standard so far (using LTTng 2.3.0).

   The goal is to run apps from the instrumented glibc (libc.so.6) by LD_PRELOADing it (or installing it as the standard glibc) that become LTTng-active (provide user-space events) if the tracepoint provider shared object is also preloaded.  This is similar to the standard dynamically-loaded statically-unaware approach to app instrumentation, where the instrumented app runs fine without the tracepoint provider shared object, and becomes LTTng-active only if the latter is preloaded.

   The glibc make doesn't like this, however.  It fails with a bunch of undefined references to dlsym, dlopen, dlclose (in __tracepoint__init_urcu_sym, __tracepoints__init, __tracepoints__ptrs_init, __tracepoints__destroy, __tracepoints__ptrs_destroy) when it tries to use libc_pic.os in the final(?) construction of libc.so.  (I can provide the make log, of course)

   Anybody else tried to do something like this?  What is the least intrusive way of getting an LTTng-instrumented libc.so to compile?

Daniel U. Thibault
Protection des systèmes et contremesures (PSC) | Systems Protection & Countermeasures (SPC)
Cyber sécurité pour les missions essentielles (CME) | Mission Critical Cyber Security (MCCS)
RDDC - Centre de recherches de Valcartier | DRDC - Valcartier Research Centre
2459 route de la Bravoure
Québec QC  G3J 1X5
CANADA
Vox : (418) 844-4000 x4245
Fax : (418) 844-4538
NAC : 918V QSDJ <http://www.travelgis.com/map.asp?addr=918V%20QSDJ>
Gouvernement du Canada | Government of Canada
<http://www.valcartier.drdc-rddc.gc.ca/>
 LinkedIn: <http://ca.linkedin.com/in/daniel-u-thibault/>
 Research Gate: <https://www.researchgate.net/profile/Daniel_Thibault/>