I looked for the difference between the Linux Environment Modules or Conda Environment, but I am not clear how are these different. Basically, I want to install OpenMPI and Mpich both in my Ubuntu, but they share the several compiler wrappers. To be on safe side, I want to isolate them, so what should I use - Modules or Conda Environment or any other alternative?
Also, if I use Conda Environment, can I limit the compiled software binary to the environment? I am surprised that I have installed some software in the base environment but I am able to access is from another environment.
Environment Modules is designed to enable users with software not available in the default environment. In your situation, you can install both OpenMPI and Mpich in isolated paths, then with modulefiles users can enable the MPI flavor of their choice.
This is what is provided by default when you install openmpi and mpich on a Fedora system for instance:
$ module show mpi/mpich-x86_64
-------------------------------------------------------------------
/etc/modulefiles/mpi/mpich-x86_64:
conflict mpi
setenv MPI_BIN /usr/lib64/mpich/bin
setenv MPI_SYSCONFIG /etc/mpich-x86_64
setenv MPI_FORTRAN_MOD_DIR /usr/lib64/gfortran/modules/mpich
setenv MPI_INCLUDE /usr/include/mpich-x86_64
setenv MPI_LIB /usr/lib64/mpich/lib
setenv MPI_MAN /usr/share/man/mpich-x86_64
setenv MPI_PYTHON_SITEARCH /usr/lib64/python2.7/site-packages/mpich
setenv MPI_PYTHON2_SITEARCH /usr/lib64/python2.7/site-packages/mpich
setenv MPI_PYTHON3_SITEARCH /usr/lib64/python3.6/site-packages/mpich
setenv MPI_COMPILER mpich-x86_64
setenv MPI_SUFFIX _mpich
setenv MPI_HOME /usr/lib64/mpich
prepend-path PATH /usr/lib64/mpich/bin
prepend-path LD_LIBRARY_PATH /usr/lib64/mpich/lib
prepend-path MANPATH /usr/share/man/mpich-x86_64
prepend-path PKG_CONFIG_PATH /usr/lib64/mpich/lib/pkgconfig
-------------------------------------------------------------------
$ module show mpi/openmpi-x86_64
-------------------------------------------------------------------
/usr/share/modulefiles/mpi/openmpi-x86_64:
conflict mpi
prepend-path PATH /usr/lib64/openmpi/bin
prepend-path LD_LIBRARY_PATH /usr/lib64/openmpi/lib
prepend-path PKG_CONFIG_PATH /usr/lib64/openmpi/lib/pkgconfig
prepend-path MANPATH /usr/share/man/openmpi-x86_64
setenv MPI_BIN /usr/lib64/openmpi/bin
setenv MPI_SYSCONFIG /etc/openmpi-x86_64
setenv MPI_FORTRAN_MOD_DIR /usr/lib64/gfortran/modules/openmpi
setenv MPI_INCLUDE /usr/include/openmpi-x86_64
setenv MPI_LIB /usr/lib64/openmpi/lib
setenv MPI_MAN /usr/share/man/openmpi-x86_64
setenv MPI_PYTHON_SITEARCH /usr/lib64/python2.7/site-packages/openmpi
setenv MPI_PYTHON2_SITEARCH /usr/lib64/python2.7/site-packages/openmpi
setenv MPI_PYTHON3_SITEARCH /usr/lib64/python3.6/site-packages/openmpi
setenv MPI_COMPILER openmpi-x86_64
setenv MPI_SUFFIX _openmpi
setenv MPI_HOME /usr/lib64/openmpi
-------------------------------------------------------------------
Related
I want to create environment in SageMaker on AWS with miniconda, and make it available as kernels in Jupyter when I restart the session. But the SageMaker keep failing.
I followed the instructions found in here:
https://aws.amazon.com/premiumsupport/knowledge-center/sagemaker-lifecycle-script-timeout/
basically it says:
"Create a custom, persistent Conda installation on the notebook instance's Amazon Elastic Block Store (Amazon EBS) volume: Run the on-create script in the terminal of an existing notebook instance. This script uses Miniconda to create a separate Conda installation on the EBS volume (/home/ec2-user/SageMaker/). Then, run the on-start script as a lifecycle configuration to make the custom environment available as a kernel in Jupyter. This method is recommended for more technical users, and it is a better long-term solution."
I run this on-create.sh script on the terminal on Jupyter:
on-create.sh:
#!/bin/bash
set -e
sudo -u ec2-user -i <<'EOF'
unset SUDO_UID
# Install a separate conda installation via Miniconda
WORKING_DIR=/home/ec2-user/SageMaker/custom-environments
mkdir -p "$WORKING_DIR"
wget https://repo.anaconda.com/miniconda/Miniconda3-4.6.14-Linux-x86_64.sh -O "$WORKING_DIR/miniconda.sh"
bash "$WORKING_DIR/miniconda.sh" -b -u -p "$WORKING_DIR/miniconda"
rm -rf "$WORKING_DIR/miniconda.sh"
# Create a custom conda environment
source "$WORKING_DIR/miniconda/bin/activate"
KERNEL_NAME="conda-test-env"
PYTHON="3.6"
conda create --yes --name "$KERNEL_NAME" python="$PYTHON"
conda activate "$KERNEL_NAME"
pip install --quiet ipykernel
# Customize these lines as necessary to install the required packages
conda install --yes numpy
pip install --quiet boto3
EOF
and it creates the "conda-test-env" environment as expected.
Then I add the on-start.sh as lifestyle configuration:
#!/bin/bash
set -e
sudo -u ec2-user -i <<'EOF'
unset SUDO_UID
source "/home/ec2-user/SageMaker/custom-environments/miniconda/bin/activate"
conda activate conda-test-env
python -m ipykernel install --user --name "conda-test-env" --display-name "conda-test-env"
# Optionally, uncomment these lines to disable SageMaker-provided Conda functionality.
# echo "c.EnvironmentKernelSpecManager.use_conda_directly = False" >> /home/ec2-user/.jupyter/jupyter_notebook_config.py
# rm /home/ec2-user/.condarc
EOF
then I update the instance with the new configuration,
and when I start my notebook instance, after few minutes it fails.
I'll appreciate any help.
I am trying to install emacs ESS using GNU make.
i follow the installation instructions for ESS found in the following
link:
steps 2 and 3 :
http://ess.r-project.org/Manual/ess.html#Installation
for each step i get errors when using make:
below are the instructions and the error messages for each step:
First Instruction:
Optionally, compile elisp files and build the documentation with:
cd /path/to/ESS/
make
Without this step, info, pdf and html documentation and reference card will not be available.
My Action:
C:\Program Files\ESS\ess-17.11>make
System Messages:
cd etc; make all
make[1]: Entering directory `/cygdrive/c/Program Files/ESS/ess-17.11/etc'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/cygdrive/c/Program Files/ESS/ess-17.11/etc'
cd lisp; make all
make[1]: Entering directory `/cygdrive/c/Program Files/ESS/ess-17.11/lisp'
emacs -batch -no-site-file -no-init-file -l ./ess-comp.el -f batch-byte-compile ess-custom.el
emacs: not found
make[1]: [ess-custom.elc] Error 127
make[1]: Leaving directory `/cygdrive/c/Program Files/ESS/ess-17.11/lisp'
make: [all] Error 2
Second Instruction:
Optionally, install into your local machine with make install. You might need administrative privileges:
make install
The files are installed into /usr/share/emacs directory. For this step to run correctly on macOS, you will need to adjust the PREFIX path in Makeconf. The necessary code and instructions are commented in that file.
My Action:
C:\Program Files\ESS\ess-17.11>make install
System Messages:
cd etc; make install
make[1]: Entering directory `/cygdrive/c/Program Files/ESS/ess-17.11/etc'
mkdir -p /usr/share/emacs/etc/ess/icons
mkdir -p /usr/share/emacs/etc/ess/ESSR/R
cp -p .IS.RELEASE git-ref *.S sas-keys.* ess-sas-sh-command *.jl /usr/share/emacs/etc/ess
cp -p icons/*.xpm /usr/share/emacs/etc/ess/icons
cp -p ESSR/LOADREMOTE ESSR/VERSION /usr/share/emacs/etc/ess/ESSR
cp -p ESSR/R/*.R ESSR/R/.*.R /usr/share/emacs/etc/ess/ESSR/R
chmod +x /usr/share/emacs/etc/ess/ess-sas-sh-command
chmod: not found
make[1]: *** [install] Error 127
make[1]: Leaving directory `/cygdrive/c/Program Files/ESS/ess-17.11/etc'
make: *** [install] Error 2
I'll appreciate you help with installing ESS using make.
Thanks
Rafael
You are trying to execute under Windows commands, which are for Linux environments.
Specifically, your installation fails at the following lines:
emacs -batch -no-site-file -no-init-file -l ./ess-comp.el -f batch-byte-compile ess-custom.el
emacs: not found
and
chmod +x /usr/share/emacs/etc/ess/ess-sas-sh-command
chmod: not found
There is no chmod command under Windows. Here is the cmd equivalent: https://www.cyberciti.biz/tips/windows-change-access-permissions-from-the-command-line.html
The ESS Manual you provided suggests the following approach:
Note for Windows and macOS users: The most straightforward way to
install Emacs on your machine is by downloading the Emacs binary by
Vincent Goulet; either Emacs Modified for macOS or from Emacs Modified
for Windows.
I also suggest that you install the Emacs Modified for Windows, as installation via make under windows will be too difficult/costly - virtually impossible.
Compiling R from the R-devel svn branch, I do
export CC='clang'
export CXX='clang++'
sudo ./configure
but the configuration script still tries to use gcc as the compiler. Why?
Because sudo reads environment variables of the root user, but export saves shell variables to your user environment. (within that terminal session only)
To fix this, you need to configure with sudo -E ./configure,
which reads environment variables from your user account (= login name) when executing ./configure with heightened privileges. Also have a look at the sudo -H flag (within man sudo).
Or you can first sudo su into the root account and export CC='clang' from within that root shell.
(the root shell prompt might begin with a # rather than a $, and be missing other config niceties—eg colourisation—from /home/user/.bashrc)
When I tried to make a link inside the rules file it didn't allow me.How can I make it?
Here is what I did.
#!/usr/bin/make -f
icon = $(CURDIR)/frontpage.png
script = $(CURDIR)/guilotinga.py
launcher = $(CURDIR)/internation.desktop
DEST1 = $(CURDIR)/debian/internation/usr/share/internation
DEST2 = $(CURDIR)/debian/internation/usr/share/applications
build: build-stamp
build-stamp:
dh_testdir
touch build-stamp
clean:
dh_testdir
dh_testroot
rm -f build-stamp
dh_clean
install: build clean $(icon) $(script) $(launcher)
dh_testdir
dh_testroot
dh_prep
dh_installdirs
mkdir -m 755 -p $(DEST1)
mkdir -m 755 -p $(DEST2)
install -m 666 $(icon) $(DEST1)
install -m 777 $(script) $(DEST1)
install -m 777 $(launcher) $(DEST2)
ln -s usr/share/internation/guilotinga.py /usr/bin/internation
(That's where I stopped)
The line above is giving error saying I don't have enough privileges.What was my fault?
binary-indep: build install
dh_testdir
dh_testroot
dh_installchangelogs
dh_installdocs
dh_installexamples
dh_installman
dh_link
dh_compress
dh_fixperms
dh_installdeb
dh_gencontrol
dh_md5sums
dh_builddeb
binary-arch: build install
binary: binary-indep binary-arch
.PHONY: build clean binary-indep binary-arch binary install
I'm assuming that what you wanted was to make a symlink to /usr/bin/internation, and have the link show up as /usr/share/internation/guilotinga.py when the package is installed. If so, you're just giving the link command backwards. You want
ln -s /usr/bin/internation usr/share/internation/guilotinga.py
As a further note, though, symlinks in Debian packages should not ever be absolute, unless you're symlinking one toplevel directory to another (see Debian Policy section 10.5).
In your case, you probably don't need to change anything, since you have a call to dh_link in your build script. That tool will automatically fix noncompliant symlinks in the package build area (unless you're running in a super-ancient debhelper compat mode).
But if you want to avoid potentially confusing readers (or yourself), perhaps you ought to do
ln -s ../../bin/internation usr/share/internation/guilotinga.py
I would like my .cshrc file to carry out several commands depending on the host I've logged in to (the .cshrc file is on a disk that is shared by two different hosts).
Let's assume the two hosts are: login1.university.edu and login2.university.edu
I tried this command:
if(hostname == 'login1.university.edu') then
setenv R_LIBS_USER /n/Users/me/R.login1
else
setenv R_LIBS_USER /n/Users/me/R.login2
endif
which doesn't work.
What is the correct syntax?
Surround the hostname command with backtics to execute it and compare the result.
if(`hostname` == 'login1.university.edu') then
setenv R_LIBS_USER /n/Users/me/R.login1
else