[lttng-dev] Babeltrace2 - compilation error with intel18

[Simon Marchi]

On 2020-03-20 6:32 p.m., Rocky Dunlap via lttng-dev wrote:
> Simon,
> 
> Thanks for this - so after I updated to gcc 9.2, the shared libraries command ran correctly and the build finished. All the tests pass and I was able to read in a CTF trace from a python script.  
> 
> The issue was that the system gcc was gcc 4.X.X, so too old.  So, I had to manually add the newer gcc to the path alongside the intel compiler.  I can try to option of setting LDSHARED, but at this point that seems even more confusing than just using the system gcc, which apparently has no trouble linking intel object files into a shared library.  Maybe there are good reasons why this is the case and this is why distutils does not use the right compiler?

Glad you got it working.

> Ideally the build would just "work" with ./configure, make, make install since these kinds of compilation issues can quickly get over people's heads and cause them to abandon the library altogether.  I wonder if there is a way for the configure/make to set the LDSHARED automatically so that is is transparent to the user.  Certainly we are not the first to run into this issue with distutils.

I don't know if there's a good reason for them not to use $CC
for linking.  What's funny is that on Darwin/macOS they do it:

https://github.com/python/cpython/blob/6723e933c4d90a408cf3818362a0e4de6d84c932/Lib/distutils/sysconfig.py#L201-L209

Note that I don't know if Intel's Python distribution is using the
same distutils package as CPython.  But since you hit the same bug,
it must be quite close, if not the same.

There's a bug here about re-using CC for LDSHARED on Linux:

  https://bugs.python.org/issue24935

There are patches here

  https://github.com/python/cpython/pull/14950
  https://github.com/python/cpython/pull/7914

There wasn't much progress, but at least nobody spoke against them.

But even with that fixed there are other problems.  As I mentioned in my
previous message, in some cases, the CFLAGS value built into Python (which
means there's no easy way for us to override it) contains some flags
incompatible with clang (this could happen with icc too).  When trying to
build with CC=clang-9 on Arch Linux, I get:

ccache clang ... -fno-semantic-interposition ... -c bt2/native_bt.c -o build/temp.linux-x86_64-3.8/bt2/native_bt.o
clang-9: error: unknown argument: '-fno-semantic-interposition'

Arch Linux got this idea of building with `-fno-semantic-interposition` from Fedora:

  https://fedoraproject.org/wiki/Changes/PythonNoSemanticInterpositionSpeedup

So I guess we will face the same issue on Red Hat by the end of the decade.

So since distutils really wants to compile the Python native modules using all the same
flags as the Python interpreter was built with, I presume that they really assume that
you'll be using the exact same toolchain to build your module as was used to build the
interpreter.  Maybe we could just not pass CC/CFLAGS when building the Python module,
so it will simply be built with the same compiler/linker as Python was built with, and
we'll avoid all these problems...

Simon