BLAS-LAPACK-ATLAS-R installation with multithreading? - r

I am trying to setup ATLAS, BLAS, LAPACK, and R on an OpenSuSE 12.2 server with an 8 core AMD athlon FX-8320. I have turned off the CPU frequency scaling using cpufreq and a script cfu from paste.opensuse.org/92162247 made by James McDaniel.
So following the instructions, here http://math-atlas.sourceforge.net/atlas_install/node45.html I didn't need to install gcc (that I know yet) I have gcc (SUSE Linux) 4.7.1 20120723 installed, g++, and gfortran the same version as gcc.
The instructions are to make a bogus configuration of ATLAS to determine the compiler flags to use. ATLAS is configured in the instructions as '../configure -b 64 -D c -DPentiumCPS=2200 -Fa alg -fPIC'. I modify this to what I think will work for my setup ' ../configure -b 64 -t 8 -Fa alg -fpic -A x86_64 -D c -DpentiumCPS=3500 -Si latune 1 --prefix=/usr/local/lib64 --with-netlib-lapack-tarfile=/opt/maths/lapack-3.4.2.tgz' I get these errors.
OS configured as Linux (1)
Assembly configured as GAS_x8664 (2)
Vector ISA Extension configured as AVXMAC (3,504)
ERROR: enum fam=6, chip=8, mach=0
make[3]: *** [atlas_run] Error 44
make[2]: *** [IRunArchInfo_x86] Error 2
/bin/sh: line 1: 13849 Segmentation fault ./xarchinfo_linux -a > config0.out
make[3]: *** [atlas_run] Error 139
make[2]: *** [IRunArchInfo_linux] Error 2
Architecture configured as UNKNOWNx86 (35)
ERROR: enum fam=6, chip=8, mach=0
make[3]: *** [atlas_run] Error 44
make[2]: *** [IRunArchInfo_x86] Error 2
Clock rate configured as 3500Mhz
Parallel make command configured as '$(MAKE) -j 8'
ERROR: enum fam=6, chip=8, mach=0
make[3]: *** [atlas_run] Error 44
make[2]: *** [IRunArchInfo_x86] Error 2
Cannot detect CPU throttling.
Atlas makes a Make.inc and if I fgrep F77 and F77 flags I get 'gfortran-4.7' AND '-O -mavx -fpic -m64'. Maybe the -mavx flag handles the AVXMAC error? Do I need -mavx? I will proceed with my instructions...

(Not a real answer, but difficult to read as a comment):
A few more considerations on Dirk's recommendation to use OpenBLAS.
I also use OpenBLAS (on Ubuntu and CentOS systems).
One consideration that is important for me on the 8 and 12 core machines is that OpenBLAS allows dynamic setting of the number of threads to use, which ATLAS doesn't. In Order to set the number of threads dynamicall from within R, I use Simon Fuller's OpenBLASThreads.
OpenBLAS was pretty straightforward to build on both systems.
Binary .deb packages are available, but last time I checked them, they had the maximum possible number of threads set to 2. However, this is going to change. Till now, I also build the library from source on Ubuntu machines with > 2 cores.

Related

How to configure sqlite3's tcltests?

For research purposes, I am trying to run the sqlite3 full test suite.
I am running under Linux for this test.
I believe I have installed the correct libraries
sudo apt-get install tcl-dev tk-dev
And when I run the configure command, it finds tcl
checking whether to use an in-ram database for temporary tables... no
checking if executables have the .exe suffix... unknown
checking for Tcl configuration... found /usr/lib/tclConfig.sh
checking for existence of /usr/lib/tclConfig.sh... loading
checking for library containing readline... no
checking for library containing tgetent... no
checking for readline in -lreadline... no
checking readline.h usability... no
Then when I run make test it ends with:
Time: orderby8.test 5080 ms
Time: orderby9.test 291 ms
Makefile:1201: recipe for target 'tcltest' failed
make: *** [tcltest] Killed
It is hard to tell because your question misses important details:
What *nix exactly?
More importantly, which Tcl versions are installed by apt?
How do you obtain the sqlite3 sources incl. test harnesses?
I just tested fine the following:
Ubuntu Xenial
apt-get install tcl tcl-dev (turns out to be 8.6.7)
wget https://www.sqlite.org/2018/sqlite-src-3250200.zip
unzip sqlite-src-3250200.zip
cd sqlite-src-3250200
./configure && make test
... and it succeeds to complete the make test run:
...
Time: orderby7.test 6 ms
Time: orderby8.test 115 ms
Time: orderby9.test 6 ms
Time: oserror.test 49 ms
...
Time: zipfile2.test 10 ms
SQLite 2018-09-25 19:08:10 fb90e7189ae6d62e77ba3a308ca5d683f90bbe633cf681865365b8e92792d1c7
0 errors out of 147094 tests on builda Linux 64-bit little-endian
All memory allocations freed - no leaks
Maximum memory usage: 9278848 bytes
Current memory usage: 0 bytes
Number of malloc() : -1 calls
It seems that oserror.test crashes whatever Tcl in your case.

AddressSanitizer: SEGV in _dl_get_tls_static_info

I'm trying to get ASAN working with one program, but anything I did led to ASAN:DEADLYSIGNAL, so I tried to narrow it down and instrument a small test program with only a few compiler options, just to see if it will work at all:
$ cat > test.c <<EOF
int main(void) { return 0; }
EOF
Here's the command line:
$ gcc -g -O0 -fno-omit-frame-pointer -fsanitize=address test.c && ./a.out
ASAN:DEADLYSIGNAL
=================================================================
==5711==ERROR: AddressSanitizer: SEGV on unknown address 0xb7f11e70 (pc 0xb7f11e84 bp 0xb7ab6320 sp 0xbf92368c T16777215)
==5711==The signal is caused by a WRITE memory access.
#0 0xb7f11e83 in _dl_get_tls_static_info (/lib/ld-linux.so.2+0x11e83)
#1 0xb7a24ff9 (/usr/lib/i386-linux-gnu/libasan.so.4+0x101ff9)
#2 0xb7a10f15 (/usr/lib/i386-linux-gnu/libasan.so.4+0xedf15)
#3 0xb7f0f91a (/lib/ld-linux.so.2+0xf91a)
#4 0xb7f00cb9 (/lib/ld-linux.so.2+0xcb9)
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV (/lib/ld-linux.so.2+0x11e83) in _dl_get_tls_static_info
==5711==ABORTING
(The same error I got for the "real" program.)
With -static-libasan the stack trace is just a bit more descriptive:
$ gcc -g -O0 -fno-omit-frame-pointer -fsanitize=address -static-libasan test.c && ./a.out
ASAN:DEADLYSIGNAL
=================================================================
==5719==ERROR: AddressSanitizer: SEGV on unknown address 0xb7fc6e70 (pc 0xb7fc6e84 bp 0x005f91a0 sp 0xbfe77c2c T16777215)
==5719==The signal is caused by a WRITE memory access.
#0 0xb7fc6e83 in _dl_get_tls_static_info (/lib/ld-linux.so.2+0x11e83)
#1 0x560a49 in __sanitizer::InitTlsSize() (/home/gkirilov/test/a.out+0xe9a49)
#2 0x552e05 in __asan::AsanInitInternal() (/home/gkirilov/test/a.out+0xdbe05)
#3 0xb7fc491a (/lib/ld-linux.so.2+0xf91a)
#4 0xb7fb5cb9 (/lib/ld-linux.so.2+0xcb9)
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV (/lib/ld-linux.so.2+0x11e83) in _dl_get_tls_static_info
==5719==ABORTING
Here are some details about the system:
$ uname -a
Linux drinkpad 4.14.0-3-686-pae #1 SMP Debian 4.14.17-1 (2018-02-14) i686 GNU/Linux
$ gcc --version
gcc (Debian 7.3.0-11) 7.3.0
$ /lib/i386-linux-gnu/libc-2.27.so
Compiled by GNU CC version 7.3.0.
libc ABIs: UNIQUE IFUNC
I also tried clang-6 and gcc-8 (which brings ASAN v5 with it), again, with static and shared libasan, and I got the same messages.
Is my platform not supported? But then, I installed all tools from the official Debian repositories.
The home page of ASAN says that it has only been tested on 64-bit Ubuntu, but I think it is outdated, as the table just above it lists both Linux x86 and x86_64.
Am I doing anything wrong or is my platform not supported?
You are facing a known Asan bug (here's Glibc thread
and GCC bug). TLDR is that it fails on Glibc 2.25+ because Glibc changed some internal interfaces that Asan relied upon. Sadly this bug hasn't got enough attention yet.

Problems installing emacs ESS using make

I am trying to install emacs ESS using GNU make.
i follow the installation instructions for ESS found in the following
link:
steps 2 and 3 :
http://ess.r-project.org/Manual/ess.html#Installation
for each step i get errors when using make:
below are the instructions and the error messages for each step:
First Instruction:
Optionally, compile elisp files and build the documentation with:
cd /path/to/ESS/
make
Without this step, info, pdf and html documentation and reference card will not be available.
My Action:
C:\Program Files\ESS\ess-17.11>make
System Messages:
cd etc; make all
make[1]: Entering directory `/cygdrive/c/Program Files/ESS/ess-17.11/etc'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/cygdrive/c/Program Files/ESS/ess-17.11/etc'
cd lisp; make all
make[1]: Entering directory `/cygdrive/c/Program Files/ESS/ess-17.11/lisp'
emacs -batch -no-site-file -no-init-file -l ./ess-comp.el -f batch-byte-compile ess-custom.el
emacs: not found
make[1]: [ess-custom.elc] Error 127
make[1]: Leaving directory `/cygdrive/c/Program Files/ESS/ess-17.11/lisp'
make: [all] Error 2
Second Instruction:
Optionally, install into your local machine with make install. You might need administrative privileges:
make install
The files are installed into /usr/share/emacs directory. For this step to run correctly on macOS, you will need to adjust the PREFIX path in Makeconf. The necessary code and instructions are commented in that file.
My Action:
C:\Program Files\ESS\ess-17.11>make install
System Messages:
cd etc; make install
make[1]: Entering directory `/cygdrive/c/Program Files/ESS/ess-17.11/etc'
mkdir -p /usr/share/emacs/etc/ess/icons
mkdir -p /usr/share/emacs/etc/ess/ESSR/R
cp -p .IS.RELEASE git-ref *.S sas-keys.* ess-sas-sh-command *.jl /usr/share/emacs/etc/ess
cp -p icons/*.xpm /usr/share/emacs/etc/ess/icons
cp -p ESSR/LOADREMOTE ESSR/VERSION /usr/share/emacs/etc/ess/ESSR
cp -p ESSR/R/*.R ESSR/R/.*.R /usr/share/emacs/etc/ess/ESSR/R
chmod +x /usr/share/emacs/etc/ess/ess-sas-sh-command
chmod: not found
make[1]: *** [install] Error 127
make[1]: Leaving directory `/cygdrive/c/Program Files/ESS/ess-17.11/etc'
make: *** [install] Error 2
I'll appreciate you help with installing ESS using make.
Thanks
Rafael
You are trying to execute under Windows commands, which are for Linux environments.
Specifically, your installation fails at the following lines:
emacs -batch -no-site-file -no-init-file -l ./ess-comp.el -f batch-byte-compile ess-custom.el
emacs: not found
and
chmod +x /usr/share/emacs/etc/ess/ess-sas-sh-command
chmod: not found
There is no chmod command under Windows. Here is the cmd equivalent: https://www.cyberciti.biz/tips/windows-change-access-permissions-from-the-command-line.html
The ESS Manual you provided suggests the following approach:
Note for Windows and macOS users: The most straightforward way to
install Emacs on your machine is by downloading the Emacs binary by
Vincent Goulet; either Emacs Modified for macOS or from Emacs Modified
for Windows.
I also suggest that you install the Emacs Modified for Windows, as installation via make under windows will be too difficult/costly - virtually impossible.

How long does compiling minix take?

I downloaded the Minix source code through git:
git clone git://git.minix3.org/minix minixsrc
Then I followed the instructions on Crosscompiling MINIX with build.sh
and ran the command
sh build.sh -mi386 -O ../build tools
Now I have been waiting for about an hour for it to compile and it is continuously showing messages on my terminal like this:
# compile libclangSema/SemaExprMember.lo
c++ -O -O2 -fno-rtti -fno-exceptions -I. -I/home/keeyan/git/minixsrc/tools/llvm-lib/libclangSema/../../llvm/../../external/bsd/llvm/lib/../dist/clang/include -I/home/keeyan/git/minixsrc/tools/llvm-lib/libclangSema/../../llvm/../../external/bsd/llvm/lib/../dist/lld/include -I/home/keeyan/git/minixsrc/tools/llvm-lib/libclangSema/../../llvm/../../external/bsd/llvm/lib/../dist/llvm/include -I/home/keeyan/git/minixsrc/tools/llvm-lib/libclangSema/../../llvm/../../external/bsd/llvm/lib/../dist/mclinker/include -I/home/keeyan/git/minixsrc/tools/llvm-lib/libclangSema/../../llvm/../../external/bsd/llvm/lib/../dist/lldb/include -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -DCLANG_ENABLE_STATIC_ANALYZER -I/home/keeyan/git/minixsrc/../build/tools/llvm-include -I/home/keeyan/git/minixsrc/../build/tools/llvm/config/include -I/home/keeyan/git/minixsrc/../build/tools/llvm/config/tools/clang/include -I/home/keeyan/git/minixsrc/tools/llvm-lib/libclangSema/../../llvm/../../external/bsd/llvm/lib/../config/mclinker -std=c++11 -std=c++11 -c -o SemaExprMember.lo.o /home/keeyan/git/minixsrc/tools/llvm-lib/libclangSema/../../llvm/../../external/bsd/llvm/lib/../dist/clang/lib/Sema/SemaExprMember.cpp
mv SemaExprMember.lo.o SemaExprMember.lo
My question is, how long should this process take? Is something wrong?
System Specs:
OS: Linux Mint 18 64bit
CPU: Intel Core i7-5500U CPU # 2.4GHz x 2
I believe it should take this long to do the job. It took me about 2 hours for the sh build.sh -mi386 -O ../build tools c to finish and it completed successfully and I got the build tools. After that process I ran the sh build.sh -mi386 -O ../build -U distribution command and that also took a number of hours. I ended up with the built distribution of MINIX which is what was expected of the process. It seems after that there is no way to turn this into an ISO though.
I don't know if the build script unsets MAKEFLAGS but you can do:
MAKEFLAGS=-j\ $(nproc) sh build.sh -mi386 -O ../build tools
nproc returns the number of cores in your CPU so it uses all cores and is faster.

MPICH3 not running on multiple mahines: hydra_pmi_poxy error : Exec format error

I set up MPICH3 (mpich-3.1.3) on my notebook(Intel Core i5) and a slave processor running on ARM Cortex15 processor and both running Ubuntu 14.04 OS with ssh keygen setup for free communication.
I have installed mpich3 in the folder which is shared between the cluster through nfs.
I have exported the path from my master server only.
The installation went well and i tried out the following command on my master node alone which runs fine:
mpiexec -n 2 ./cpi
Process 0 of 2 is on MingF
Process 1 of 2 is on MingF
pi is approximately 3.1415926544231341, Error is 0.0000000008333410
wall clock time = 0.000182
But when I try running on my slave and master, then i get this error and it hangs:
mpiexec -f hosts -n 2 ./cpi
bash: /mirror/mpich3/bin/hydra_pmi_proxy: cannot execute binary file: Exec format error
It hangs there until i press 'Ctrl + C' to break out of it.
I am guessing its because of the change in processor type but I may be wrong. could someone help me out?
You cannot run the same executable on such different architectures as x86 and ARM. Compile it separately on both machines and pay attention to the endianness of the ARM machine.

Resources