adjustments to REBEL module, because not able to process review text - pipeline

Does anyone have any experience with the REBEL module? https://github.com/Babelscape/rebel
I am trying to apply this on an Amazon review dataset, but unfortunately it does not work effectively. Does anyone know how the train dataset can be adjusted?
Edit:
Can anyone indicate what type of code this is? Python/SQL?
#!/bin/bash
# setup conda
source ~/miniconda3/etc/profile.d/conda.sh
# create conda env
read -rp "Enter environment name: " env_name
read -rp "Enter python version (e.g. 3.7) " python_version
conda create -yn "$env_name" python="$python_version"
conda activate "$env_name"
# install torch
read -rp "Enter cuda version (e.g. 10.1 or none to avoid installing cuda support): " cuda_version
if [ "$cuda_version" == "none" ]; then
conda install -y pytorch torchvision cpuonly -c pytorch
else
conda install -y pytorch torchvision cudatoolkit=$cuda_version -c pytorch -c conda-forge
fi
# install python requirements
pip install -r requirements.txt

Related

R fails to load igraph in conda environment

I am trying to run an RShiny app in a WSL2 installation of Ubuntu on Windows. I am no expert in R, but I feel this is a problem due to conda interaction with R. I run the following commands:
conda create -n r_env r-essentials r-base
conda activate r_env
conda install -c conda-forge r-shiny
conda install -c r r-visnetwork
conda install -c conda-forge r-dplyr
conda install -c r r-dt
conda install -c conda-forge r-igraph
conda install -c r r-leaflet
conda install -c conda-forge r-rgdal
conda install -c r r-shinydashboard
conda install -c conda-forge r-shinywidgets
conda install -c conda-forge r-shinycssloaders
conda install -c conda-forge r-igraph
When I run R and type in : library(igraph) i get:
->Error: package or namespace load failed for ‘igraph’ in dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object '/home/carlo/anaconda3/envs/r_env/lib/R/library/igraph/libs/igraph.so':
but I can list it, it's there:
ll /home/carlo/anaconda3/envs/r_env/lib/R/library/igraph/libs/igraph.so
-> -rwxrwxr-x 1 carlo carlo 3816608 Mar 31 15:38 /home/carlo/anaconda3/envs/r_env/lib/R/library/igraph/libs/igraph.so
Did anybody encounter a similar problem?
The igraph library was meant to load correctly
(My Opinion) I would caution against the use of the r channel and the r-essentials package. The Continuum/Anaconda support for R was a good college-try, but is since outmoded and superseded by the broader CRAN support that Conda Forge provides. Users managing R environments will find a better experience ignoring any Continuum/Anaconda documentation and exclusively using Conda Forge for their R environments. (End Opinion)
Mixing channels can lead to symbol reference errors. Furthermore, sequences of ad hoc installations are subpar - instead specify through a YAML.
The following YAML file works just fine on linux-64, osx-64, and win-64 platforms:
so-igraph.yaml
name: so-igraph
channels:
- conda-forge
dependencies:
- r-base=4.1 # adjust to desired version
- r-shiny
- r-visnetwork
- r-dplyr
- r-dt
- r-igraph
- r-leaflet
- r-rgdal
- r-shinydashboard
- r-shinywidgets
- r-shinycssloaders
- r-igraph
Which can be used with
conda env create -n so-igraph -f so-igraph.yaml
conda activate so-igraph
I was having a similar problem with R on AlmaLinux, and it turned out I was missing some libraries on the OS itself, which I thought I had and that were necessary for some R packages. I think they were these, which makes sense, given that we're talking about igraph, a graphing package:
gsl-devel-2.5-1.el8.x86_64
gsl-2.5-1.el8.x86_64
openssl-1.1.1k-6.el8_5.x86_64
geos-devel-3.7.2-1.el8.x86_64
geos-3.7.2-1.el8.x86_64
proj-datumgrid-1.8-6.3.2.4.el8.noarch
proj-6.3.2-4.el8.x86_64
libtiff-devel-4.0.9-21.el8.x86_64
libgeotiff-devel-1.5.1-1.el8.x86_64
libgeotiff-1.5.1-1.el8.x86_64
Installing openblas may work, see https://github.com/conda-forge/r-igraph-feedstock/issues/19

Jupyter kernel uses different Python version than Conda environment

I am not able to link Jupyter kernels to their parent Conda environments. After creating a new kernel linked to Conda environment, I'm getting a different version of Python and its dependencies inside Jupyter lab.
Here are the steps I followed:
Created a conda environment using:
conda create -n nlp python=3.6
conda activate nlp
(nlp) ➜ ~ python --version
Python 3.6.9 :: Anaconda, Inc.
(nlp) ➜ ~ which python
/anaconda3/envs/nlp/bin/python
Inside the environment I created a Jupyter kernel with:
(nlp) ➜ ~ python -m ipykernel install --user --name=nlp
Installed kernelspec nlp in /Users//Library/Jupyter/kernels/nlp
Investigating the created json file for the kernel:
(nlp) ➜ ~ cat /Users/<username>/Library/Jupyter/kernels/nlp/kernel.json
{
"argv": [
"/anaconda3/envs/nlp/bin/python",
"-m",
"ipykernel_launcher",
"-f",
"{connection_file}"
],
"display_name": "nlp",
"language": "python"
}%
It seems to be pointing to the environment version of Python
But when I start Jupyter Lab and select the nlp kernel, I get a different version of Python and some dependencies are missing
!python --version
Python 3.5.6 :: Anaconda, Inc.
!which python
/anaconda3/bin/python
Could you please try the following steps:
conda activate nlp
conda install ipykernel
ipython kernel install --name nlp --user
After these steps please try changing the kernel again in jupyter lab to "nlp".
Thanks.
this behavior is actually normal in Jupyter lab.
If you run
import sys
print(sys.version)
!python --version
in a notebook, the print statement will give you the Python version of the conda env, while the second will give you the Python version of your base env.
The easiest workaround for this is to simply pip install jupyterlab in your conda env and then run jupyter lab in your conda env. Then, there will not be a mismatch in Python versions between the new "base" env and the conda env which will help clear up any DLL problems.
It's probably not best practice, but you do what you gotta do when working with legacy code, ig.
Can you try this :
# in base env
conda install nb_conda_kernels
conda activate nlp
conda install ipykernel
conda install ipywidgets
# install kernelspec
python -m ipykernel install --user --name nlp --display-name "nlp env"
When you run jupyter notebook, you will see 2 nlp kernels. Use the one with "Python [conda:env:nlp]"

Parallel Group setup & mpi4py/OpenMDAO 2.2.X

I am trying to use the parallelization with mpi/openmdao.
I have tried on various ubuntu computers as well as ubuntu bash on windows (a windows 10 feature)
The dependencies work fine independently (i.e. import petsc4py and import mpi4py works fine and I can run the tests of these similar to the links: https://openmdao.readthedocs.io/en/1.7.3/getting-started/mpi_linux.html &
http://mpi4py.scipy.org/docs/usrman/install.html)
But the Paralel Group code in the openmdao 2.2. manual does not work.
For each attempt (varying computers) i seem to get another error most of them seemed like compatibility errors (i.e. I install petsc4py which breaks numpy or mpi4py installation causing proble in the existing openmdao core. )
On some computers I had my own openmpi and petsc installed but conda install command already installs those as far as I see.
Eventually I have tried these steps on a newly started amazon instance
but had similar problems.
sudo apt-get install build-essential
wget http://repo.continuum.io/archive/Anaconda3-5.2.0-Linux-x86_64.sh
bash Anacond*
sudo apt-get install libibnetdisc-dev
sudo apt-get install libblas-dev libatlas-dev liblapack-dev
conda install mpi4py
conda install -c conda-forge petsc4py
if i check ''conda list'' onone of the computers the abbreviated output is ;
mpi 1.0 mpich conda-forge
mpi4py 3.0.0 py36_mpich_1 conda-forge
mpich 3.2.1 1 conda-forge
mpich2 1.4.1p1 0 anaconda
mpmath 1.0.0 py36hfeacd6b_2
msgpack-python 0.5.1 py36h6bb024c_0
multipledispatch 0.4.9 py36h41da3fb_0
mumps 5.0.2 blas_openblas_208 [blas_openblas]
conda-forge
numpy 1.14.3 py36_blas_openblas_200 [blas_openblas] conda-forge
numpydoc 0.7.0 py36h18f165f_0
openblas 0.2.20 8 conda-forge
openmdao 2.2.1 <pip>
openpyxl 2.4.10 py36_0
openssl 1.0.2o 0 conda-forge
petsc 3.9.1 blas_openblas_0 [blas_openblas]
conda-forge
petsc4py 3.9.1 py36_0 conda-forge
pexpect 4.3.1 py36_0
pickleshare 0.7.4 py36h63277f8_0
pillow 5.0.0 py36h3deb7b8_0
pip 10.0.1 <pip
On the same system if try to run
mpirun -n 2 python my_par_model.py
based on the manual code this is what i get
Does anyone have a suggestion where it could be failing or what steps i could follow for ubuntu implementation of anconda/openmdao/petsc/mpi4py and succesful run of paralel openmdao ?
You could take a look at the installation implementation for linux that exists in our .travis.yml file? https://github.com/OpenMDAO/OpenMDAO/blob/master/.travis.yml
This works for installing and testing OpenMDAO from scratch on Trusty Tahr instances on Travis CI. One difference I see at first glance would be our use of pip to install mpi and PETSc into the conda-installed python.
I think MPI compatibility was the main issue. I was not aware that it had to be openmpi and indeed conda install command installs the mpich and possibly causing a problem with openmdao.
I will continue doing more tests but for a working system starting from a brand new installation of ubuntu-16.04.4-desktop-amd64.iso I followed these steps;
(Steps that take time are the openmpi installation and petsc4py pip instalattion.)
1 ) For some dependencies (taken from https://gist.github.com/mrosemeier/088115b2e34f319b913a)
sudo apt-get install libibnetdisc-dev
sudo apt-get install libblas-dev libatlas-dev liblapack-dev
2) Download/Install OpenMPI (mostly taken from http://lsi.ugr.es/jmantas/pdp/ayuda/datos/instalaciones/Install_OpenMPI_en.pdf)
wget https://download.open-mpi.org/release/open-mpi/v3.1/openmpi-3.1.0.tar.gz
tar -xzf openmpi-3.1.0.tar.gz
cd openmpi-*
./configure --prefix="/home/$USER/.openmpi"
make
sudo make install
echo export PATH="$PATH:/home/$USER/.openmpi/bin" >> /home/$USER/.bashrc
echo export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/home/$USER/.openmpi/lib/" >> /home/$USER/.bashrc
3) MINICONDA & Rest (mostly taken from https://github.com/OpenMDAO/OpenMDAO/blob/master/.travis.yml)
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
sh Minicond* # agree to add to the path etc.
conda install --yes python=3.6
conda install --yes numpy==1.14 scipy=0.19.1 nose sphinx mock swig pip;
pip install --upgrade pip;
pip install mpi4py
pip install petsc4py==3.9.1
#petsc4py Gives an error failed building wheel for petsc but then installs petsc itself, afterwards, petsc4py is also installed
sudo apt install git # in the cases git does not exist
# not sure why we need this part but i followed
pip install redbaron;
pip install git+https://github.com/OpenMDAO/testflo.git;
pip install coverage;
pip install git+https://github.com/swryan/coveralls-python#work;
# pyoptsparse and openmdao
git clone https://github.com/mdolab/pyoptsparse.git;
cd pyoptsparse;
python setup.py install;
cd ..;
conda install --yes matplotlib;
git clone http://github.com/OpenMDAO/OpenMDAO
cd OpenMDAO
pip install .
# optional
conda install spyder
4) Check the versions
mpirun --version : Open MPI 3.1.0
python --version : 3.6.5
pip --version :
pip 10.0.1 from /home/user/miniconda3/lib/python3.6/site-packages/pip (python 3.6)
conda list : (note that there is no mpich or similar in the conda list)
openmdao 2.2.1 <pip>
mpi4py 3.0.0 <pip>
petsc 3.9.2 <pip>
petsc4py 3.9.1 <pip>

Anaconda Jupyter R install

Hi I tried to install R for my Jupyter notebook (Anaconda) and used the below command in my Anaconda prompt "conda install -c r r-essentials". But there is an error like below: any suggestions (especially missing package).
(C:\Anaconda3) C:\windows\system32>conda install -c r r-essentials
Using Anaconda Cloud api site https://api.anaconda.org
Fetching package metadata ...Could not connect to
https://repo.continuum.io/pkgs
/pro/noarch/
Could not connect to https://repo.continuum.io/pkgs/free/win-64/
Could not connect to https://repo.continuum.io/pkgs/pro/win-64/
...Could not connect to https://conda.anaconda.org/r/win-64/
.Could not connect to https://conda.anaconda.org/r/noarch/
.Could not connect to https://repo.continuum.io/pkgs/free/noarch/
.
Solving package specifications: .
Error: Package missing in current win-64 channels:
- r-essentials
You can search for packages on anaconda.org with
anaconda search -t conda r-essentials
(C:\Anaconda3) C:\windows\system32>
Thanks
Try this alternative. It worked on my colleague's machine:
conda create -n my-r-env -c r r-essentials
Another approach would be to reinstall Anaconda and execute
conda install -c r r-essentials
Let us know if it worked.

Error setting up environment on Google Cloud Shell to use Cloud ML

Following instructions from https://cloud.google.com/ml/docs/how-tos/getting-set-up:
Created a project and started the Cloud Shell.
Ran below script to Install required tools and dependencies
curl https://raw.githubusercontent.com/GoogleCloudPlatform/cloudml-samples/master/tools/setup_cloud_shell.sh | bash
Expected outcome >>
"Success! Your environment has the required tools and dependencies." when the script finishes successfully.
Actual outcome >>
+ pip install --user --upgrade 'pillow>=3.4.2' --global-option=build_ext --global-option=--disable-jpeg
/usr/local/lib/python2.7/dist-packages/pip-8.1.1- py2.7.egg/pip/commands/install.py:180: UserWarning: Disabling all use of wheels due to the use of --build-options / --g
lobal-options / --install-options.
cmdoptions.check_install_build_global(options)
Collecting pillow>=3.4.2
Using cached Pillow-4.0.0.tar.gz
Collecting olefile (from pillow>=3.4.2)
Using cached olefile-0.43.zip
Installing collected packages: olefile, pillow
Running setup.py install for olefile ... error
Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build- KwQqVS/olefile/setup.py';exec(compile(getattr(tokenize, 'op
en', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" build_ext --disable-jpeg install --record /tmp/pip-eOHYKZ-record/install-record.txt --single-vers
ion-externally-managed --compile --user --prefix=:
usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: -c --help [cmd1 cmd2 ...]
or: -c --help-commands
or: -c cmd --help
error: option --disable-jpeg not recognized
----------------------------------------
Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build- KwQqVS/olefile/setup.py';exec(compile(getattr(tokenize, 'open', open) (__file__).rea
d().replace('\r\n', '\n'), __file__, 'exec'))" build_ext --disable-jpeg install --record /tmp/pip-eOHYKZ-record/install-record.txt --single-version- externally-managed -
-compile --user --prefix=" failed with error code 1 in /tmp/pip-build- KwQqVS/olefile/
I had the same problem and I solved it.
The error came from Pilow.
First you should install Pillow seperately: (with sudo)
sudo pip install --upgrade pillow
Then you run again the command:
curl https://raw.githubusercontent.com/GoogleCloudPlatform/cloudml-samples/master/tools/setup_cloud_shell.sh | bash
Everything will be ok.
The fix has been pushed to Github, the new version did
pip install --user --upgrade pillow,
which doesn't have the '--disable-jpeg' option. It seems that the new version of Pillow fixed the issue mentioned in Fail during installation of Pillow (Python module) in Linux and removed the option.

Resources