How do I initialize LLVM's external symbolizer? - runtime-error

When compiling with -fsanitize=memory I get WARNING: Trying to symbolize code, but external symbolizer is not initialized! when running the program. How do I initialize the external symbolizer?

I solved my own problem using MSAN_SYMBOLIZER_PATH=$(which llvm-symbolizer-3.4) ./a.out. The problem is that Ubuntu postfixes the version number but the binary doesn't know that. Of course you need to use MSAN instead of ASAN when using the memory sanitizer.

You are supposed to be able to set the ASAN_FILTER environment variable to point at a symbolizer, but I could not get it to work. However, you can redirect stderr into a symbolizer after the fact. You'll still get the warnings about the uninitialized symbolizer, but the filenames and line numbers will be correct.
You can use asan_symbolizer.py as the external symbolizer. After downloading it from that link (to /tmp, for example), invoke your program like so (in bash, for this example):
./myprogram 2>&1 | /tmp/asan_symbolize.py | c++filt

On my Ubuntu system, the issue is that LLVM's tools are installed under /usr/bin with version suffixes (like llvm-symbolizer-4.0), and the sanitizer tools are looking for them without version suffixes.
LLVM also installs its binaries to, e.g., /usr/lib/llvm-4.0/bin; the tools under /usr/bin are actually just symlinks. So an easy solution is to add the appropriate /usr/lib/llvm-*/bin directory to your path when working with sanitizers.

I received such warning when I run program debug version (compiled with -fsanitize=address) on Linux machine that didn't contain clang installation. The problem disappeared after I installed clang from devtoolset.

Related

Why am I getting a zsh: exec format error?

I am trying to run the metal executable from my zsh terminal in order to meta-analyze GWAS data. I have the executable in the correct directory and have checked that it is not 0MB due to truncation.
Reproducible:
Download the Linux file from http://csg.sph.umich.edu/abecasis/metal/download/
In terminal:
PATH TO EXECUTABLE ./metal
zsh: exec format error: ./metal
You probably install wrong OS of go, for example, you might install go for MacOS in Linux
I solve this problem by installing go for correct OS
The executable has been pre-compiled on a certain distribution with libraries at a certain places at certain versions.
If you have a different distribution, libraries versions, it won't work and you better compile from the source.
Basically what you have to do is to download and extract the sources, go in the folder and execute make. (You will have probably to install make first.)
I think that's not your job at all so maybe you can find a geeky person to help you, because you may stumble upon problems, libraries to install, old versions not supported anymore, new versions not supported yet...
It happened to me when I emptied an executable by mistake.
~>true > a
~>wc -c a
0 a
~>./a
zsh: exec format error: ./a

Compiling C++ code for R (CRAN) packages on Solaris

I am a little bit confused on how to efficiently prepare the R package, so that it will be compatible across all needed system platforms. This is needed so that the new version of package will be accepted by CRAN. The main difficulty comes from compiling external C++ shared library, and optionally CUDA version if the compiler is available. To support this flow I've created specific Makefile, unfortunately using GNU-extensions. It works fine on Linux, OSX and when executed manually via gmake on Solaris. Relevant part is here:
# Checking whether nvcc compiler is available
NVCC_TEST = $(shell basename $(shell which nvcc 2> /dev/null)"")
ifeq ($(NVCC_TEST),nvcc)
ALL_LIBS += libcucubes_gpu.so
ALL_OBJS += $(GPU_OBJS)
ALL_FLAGS += $(GPU_FLAGS)
else
ALL_OBJS += gpu_fallback.o
endif
Turns out that, when running R CMD INSTALL (...) on Solaris, the installation fails on something like this:
make: Fatal error in reader: Makefile, line 39: Unexpected end of line seen
ERROR: compilation failed for package 'libcucubes'
As it turns out, it is caused by the fact that Solaris' version of make is executed instead of GNU-compatible gmake (I've tested it works fine), even though it is available. My question is whether there is any simple way to force usage of gmake here, for the R package build. In general I know I could use autotools to solve compatibility issues during installation, but it seems to bring too much complexity for that simple case. Any advices will be really appreciated, thanks!
If you can't get your build process to use gmake instead of Solaris's pure POSIX make, you can use this hack:
Make a dedicated directory for this hack: mkdir $HOME/make_hack
Softlink gmake asmakein that directory: ln -s /path/to/gmake $HOME/make_hack/make
Set your PATH: PATH=$HOME/make_hack:$PATH
Now, run your build process using that PATH, and it should use gmake. Hopefully it just uses make from its PATH envval and not some hardcoded full path.
Yeah, it's a hack. But it's probably a lot easier than modifying the build process to use gmake instead of make.
From Writing R Extensions:
If you really must require GNU make, declare it in the DESCRIPTION
file by
SystemRequirements: GNU make
and ensure that you use the value of environment variable MAKE (and
not just make) in your scripts.
configure scripts are the preferred solution though. BTW, in general a Makevars file is also preferred over a full Makefile.

Shared library present with different name

I compiled my program with gcc using openssl crypto library.
I moved the executable to an other system, and after installing openssl tried to run it. I got the following error:
error while loading shared libraries: libcrypto.so.1.0.0: cannot open shared object file: No such file or directory
What I understand from the following the library is present, but has another name:
locate libcrypto
/usr/lib64/.libcrypto.so.1.0.2f.hmac
/usr/lib64/.libcrypto.so.10.hmac
/usr/lib64/libcrypto.so
/usr/lib64/libcrypto.so.1.0.2f
/usr/lib64/libcrypto.so.10
/usr/lib64/pkgconfig/libcrypto.pc
Can I somehow tell the binary to use one of the present libraries? Or is there a way to install the one which is required?
The compilation was done on Ubuntu 15.10 64 bit, tried to run on Fedora 4.2.3-300.fc23.x86_64.
Can I somehow tell the binary to use one of the present libraries?
No, you can't: there is a reason these libraries have a different name: they are not ABI-compatible. If you managed to somehow tell the binary to use the other library, the result will be a crash if you are lucky, or a silent corruption if you are not.
(BTW, you can try this by creating a symbolic link: ln -s libcrypto.so.1.0.2f libcrypto.so.1.0.0, but you've been warned not to do this).
is there a way to install the one which is required?
Sure: you should be able to copy libcrypto.so.1.0.0 from Ubuntu machine to Fedora one, assuming you can't find a Fedora package that provides it.

Compiling haskell module Network on win32/cygwin

I am trying to compile Network.HTTP (http://hackage.haskell.org/package/network) on win32/cygwin. However, it does fail with following message:
Setup.hs: Missing dependency on a foreign library:
* Missing (or bad) header file: HsNet.h
This problem can usually be solved by installing the system package that
provides this library (you may need the "-dev" version). If the library is
already installed but in a non-standard location then you can use the flags
--extra-include-dirs= and --extra-lib-dirs= to specify where it is.
If the header file does exist, it may contain errors that are caught by the C
compiler at the preprocessing stage. In this case you can re-run configure
with the verbosity flag -v3 to see the error messages.
Unfortuntely it does not give more clues. The HsNet.h includes sys/uio.h which, actually should not be included, and should be configurered correctly.
Don't use cygwin, instead follow Johan Tibells way
Installing MSYS
Install the latest Haskell Platform. Use the default settings.
Download version 1.0.11 of MSYS. You'll need the following files:
MSYS-1.0.11.exe
msysDTK-1.0.1.exe
msysCORE-1.0.11-bin.tar.gz
The files are all hosted on haskell.org as they're quite hard to find in the official MinGW/MSYS repo.
Run MSYS-1.0.11.exe followed by msysDTK-1.0.1.exe. The former asks you if you want to run a normalization step. You can skip that.
Unpack msysCORE-1.0.11-bin.tar.gz into C:\msys\1.0. Note that you can't do that using an MSYS shell, because you can't overwrite the files in use, so make a copy of C:\msys\1.0, unpack it there, and then rename the copy back to C:\msys\1.0.
Add C:\Program Files\Haskell Platform\VERSION\mingw\bin to your PATH. This is neccesary if you ever want to build packages that use a configure script, like network, as configure scripts need access to a C compiler.
These steps are what Tibell uses to compile the Network package for win and I have used this myself successfully several times on most of the haskell platform releases.
It is possible to build network on win32/cygwin. And the above steps, though useful (by Jonke) may not be necessary.
While doing the configuration step, specify
runghc Setup.hs configure --configure-option="--build=mingw32"
So that the library is configured for mingw32, else you will get link or "undefined references" if you try to link or use network library.
This combined with #Yogesh Sajanikar's answer made it work for me (on win64/cygwin):
Make sure the gcc on your path is NOT the Mingw/Cygwin one, but the
C:\ghc\ghc-6.12.1\mingw\bin\gcc.exe
(Run
export PATH="/cygdrive/.../ghc-7.8.2/mingw/bin:$PATH"
before running cabal install network in the Cygwin shell)

cygwin warning when building R package

I'm trying to build my own package using Windows. I installed everything I need, and running R CMD build mypackage and R CMD INSTALL mypackage seem to run fine. when I run the build command, though, I get a warning from cygwin:
cygwin warning:
MS-DOS style path detected: C:/Documents and Settings/e_sander/My Documents/mypackage_1.0.tar.gz
Preferred POSIX equivalent is: /cygdrive/c/Documents and Settings/e_sander/My Documents/mypackage_1.0.tar.gz
CYGWIN environment variable option "nodosfilewarning"turns off this warning. Consult the user's guide for more details about POSIX paths:
http://cygwin.com/cygwin-ug-net/using.html#using-pathnames
I did go to the recommended website, but I don't know much about cygwin or linux so I'm not sure there's anything I need to do. I realize that using the MS-DOS style path is deprecated and not recommended in cygwin, but I'm not sure how to change that, since I'm running Windows and that's the path I need. I also haven't noticed any problems with my package, at least when I install it to my computer (and although I haven't used the tarball, I've opened it and everything looks fine). So here's what I'm trying to figure out:
Does leaving the path as is affect my package in any way?
If so, how could it adversely affect my package?
How do I change the path to make cygwin happy?
This is just a warning and it tells you how to disable it. It doesn't affect anything. If you want it to go away, run this from your shell:
export CYGWIN="nodosfilewarning"
Or you could mount C: to /c/ (see man mount).
Does leaving the path as is affect my package in any way?
- No, it is just a warning.
If so, how could it adversely affect my package?
- N/A
How do I change the path to make cygwin happy?
- Set the environment variable as the output states. There are multiple ways to do this; I chose to go with a solution that handles the issue across any invoked environment that parses or inherits from the windows environment by using the "Rapid Environment Editor" program to add a User Variable named CYGWIN with a value nodosfilewarning. But if you wanted you could add it through the control panel using Add environment variables for your account.

Resources