R reticulate unable to find installed python library - r

I'm venturing into using Reticulate in R and having trouble installing a package, specifically psycopg2 but I've also tried installing twisted with the same result.
after I load reticulate in R I double check to make sure my package is installed:
> conda_install(envname = "r-reticulate", packages="psycopg2")
Solving environment: ...working... done
# All requested packages already installed.
Looks good. So I set my condaenv to r-reticulate just to be double sure. The docs say I should not have to do this, but it should not hurt:
> use_condaenv( "r-reticulate")
Then I try to import psycopg2:
> psycopg2 <- import('psycopg2')
Error in py_module_import(module, convert = convert) :
ImportError: No module named psycopg2
If I install twisted the same way, I get the same error:
> twisted <- import('twisted')
Error in py_module_import(module, convert = convert) :
ImportError: No module named twisted
So, what am I doing wrong here?
Below is the output of conda_list and py_config... I was expecting py_config to show me the path to the python exe in my r-reticulate environment. However I don't have anything to compare this to, so my expectations may have been wrong.
> conda_list()
name python
1 Anaconda C:\\Users\\jal\\AppData\\Local\\Continuum\\Anaconda\\python.exe
2 r-reticulate C:\\Users\\jal\\AppData\\Local\\Continuum\\Anaconda\\envs\\r-reticulate\\python.exe
> py_config()
python: C:\Users\jal\AppData\Local\CONTIN~1\Anaconda\python.exe
libpython: C:/Users/jal/AppData/Local/CONTIN~1/Anaconda/python27.dll
pythonhome: C:\Users\jal\AppData\Local\CONTIN~1\Anaconda
version: 2.7.15 |Anaconda custom (64-bit)| (default, May 1 2018, 18:37:09) [MSC v.1500 64 bit (AMD64)]
Architecture: 64bit
numpy: C:\Users\jal\AppData\Local\CONTIN~1\Anaconda\lib\site-packages\numpy
numpy_version: 1.14.3
pandas: C:\Users\jal\AppData\Local\CONTIN~1\Anaconda\lib\site-packages\pandas
python versions found:
C:\Users\jal\AppData\Local\CONTIN~1\Anaconda\python.exe
C:\Users\jal\AppData\Local\CONTIN~1\ANACON~1\python.exe

On a whim I downloaded and installed the latest RStudio desktop (1.2.701) and upgraded from 1.1.442. This seems to have solved my issue. The packages load and I now see that when I run py_config I see paths that reflect my environment choice, as I would expect.
> py_config()
python: C:\Users\jal\AppData\Local\Continuum\Anaconda\envs\r-reticulate\python.exe
libpython: C:/Users/jal/AppData/Local/Continuum/Anaconda/envs/r-reticulate/python36.dll
pythonhome: C:\Users\jal\AppData\Local\CONTIN~1\Anaconda\envs\R-RETI~1
version: 3.6.5 |Anaconda, Inc.| (default, Mar 29 2018, 13:32:41) [MSC v.1900 64 bit (AMD64)]
Architecture: 64bit
numpy: C:\Users\jal\AppData\Local\CONTIN~1\Anaconda\envs\R-RETI~1\lib\site-packages\numpy
numpy_version: 1.14.3
psycopg2: C:\Users\jal\AppData\Local\CONTIN~1\Anaconda\envs\R-RETI~1\lib\site-packages\psycopg2
python versions found:
C:\Users\jal\AppData\Local\Continuum\Anaconda\envs\r-reticulate\python.exe
C:\Users\jal\AppData\Local\CONTIN~1\Anaconda\python.exe
C:\Users\jal\AppData\Local\CONTIN~1\ANACON~1\python.exe

I solved my issue by following this way in RStudio:
py_config()
No non-system installation of Python could be found.
Would you like to download and install Miniconda?
Miniconda is an open source environment management system for Python.
See https://docs.conda.io/en/latest/miniconda.html for more details.
Would you like to install Miniconda? [Y/n]: Y
Downloading "https://repo.anaconda.com/miniconda/Miniconda3-latest-Windows-x86_64.exe" ...
trying URL 'https://repo.anaconda.com/miniconda/Miniconda3-latest-Windows-x86_64.exe'
Content type 'application/octet-stream' length 57256056 bytes (54.6 MB)
downloaded 54.6 MB
Installing Miniconda -- please wait a moment ...

I solved my issue by following this way:
Close RStudio.
Open Terminal and activate your desired environment.
Run RStudio by $ open -na Rstudio
Now you have your anac

Related

Error: Python module tensorflow was not found. Rstudio, Windows10 - PATH problem

I am trying to install TensorFlow on my RStudio for Windows10 following the official instructions found in the website https://tensorflow.rstudio.com/installation/
and with the requisite found in the official TensorFlow webs
https://www.tensorflow.org/install?hl=en
I found this is still an unsolved problem considering that nobody could really help in the previous questions like this for example (R, TensorFlow, Anaconda Install on Windows)
So I followed many videos on youtube and read a lot but still nothing. When I run:
library(remotes)
remotes::install_github("rstudio/tensorflow")
library(tensorflow)
install_tensorflow()
tf_config()
Installation of TensorFlow not found.
Python environments searched for 'tensorflow' package:
C:\Users\Simone\anaconda3\envs\r-reticulate\python.exe
C:\Users\Simone\anaconda3\python.exe
You can install TensorFlow using the install_tensorflow() function.
I tried this after tried the way suggested in the official page
install.packages("tensorflow")
library(tensorflow)
install_tensorflow()
library(tensorflow)
tf$constant("Hellow Tensorflow")
but still, same thing. I think it's a PATH problem but I don't know how to solve it.
Consider that when I run:
py_config() #then i get this
python: C:/Users/Simone/anaconda3/envs/r-reticulate/python.exe
libpython: C:/Users/Simone/anaconda3/envs/r-reticulate/python36.dll
pythonhome: C:/Users/Simone/anaconda3/envs/r-reticulate
version: 3.6.12 |Anaconda, Inc.| (default, Sep 9 2020, 00:29:25) [MSC v.1916 64 bit (AMD64)]
Architecture: 64bit
numpy: C:/Users/Simone/anaconda3/envs/r-reticulate/Lib/site-packages/numpy
numpy_version: 1.19.4
tensorflow: C:\Users\Simone\ANACON~2\envs\R-RETI~1\lib\site-packages\tensorflow\__init__.p
python versions found:
C:/Users/Simone/anaconda3/envs/r-reticulate/python.exe
C:/Users/Simone/anaconda3/python.exe
TensorFlow seems to be installed in this PATH C:\Users\Simone\ANACON~2\envs\R-RETI~1\lib\site-packages\tensorflow_init_.p but manually checking this PATH it doesn't appear. Seems like python is installed in one path, and tensorflow in another one
Any idea how I could fix the path problem?
Thanks!
EDIT1:
I also noticed this warning message from install_tensorflow()
WARNING: The script tensorboard.exe is installed in 'C:\Users\Simo\ANACON~2\envs\R-RETI~1\Scripts' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
so that would be the problem I should solve, but don't know how
I have met the same problem of intalling tensorflow in Rstudio, but I found a way to avoid it by installing tensorflow in a new anaconda environment, then activate the envirnoment inside Rstudio. It works in my computer (win 11, R4.2.1).
install anaconda
create a new tensorflow environment named tf-gpu, open anaconda prompt and run:
conda create -n tf-gpu tensorflow-gpu
conda activate tf-gpu
You can check the environment in anaconda prompt by:
conda env list
open Rstudio and run:
library("tensorflow")
use_condaenv("tf-gpu", required = TRUE)
tf$constant("Hello Tensorflow!")

The h5py Python package is required to save and load models in R

I am using Tensorflow 1.15, Keras 2.2 with python 3.7 in r-reticulate env. When I load my model I have an error message:
> model <- load_model_hdf5("my_model_W18_2c2.h5")
Error in load_model_hdf5("my_model_W18_2c2.h5") :
The h5py Python package is required to save and load models
The problem seems to come from the version of H5PY I have. I am not expert in IT and the file I have in the H5PY folder in r-reticulate have the following type H5*.cp36-win_amd64.pyd.
When I updated H5PY version to 2.10.0 Files changed to H5*.cp37-win_amd64.pyd and R-studio crashed with the following error message
Warning! ***HDF5 library version mismatched error
I have no idea how to solve this.
Here my config:
> reticulate::py_config()
python: C:/Users/Mezeix/AppData/Local/r-miniconda/envs/r-reticulate/python.exe
libpython: C:/Users/Mezeix/AppData/Local/r-miniconda/envs/r-reticulate/python37.dll
pythonhome: C:/Users/Mezeix/AppData/Local/r-miniconda/envs/r-reticulate
version: 3.7.9 (default, Aug 31 2020, 17:10:11) [MSC v.1916 64 bit (AMD64)]
Architecture: 64bit
numpy: C:/Users/Mezeix/AppData/Local/r-miniconda/envs/r-reticulate/Lib/site-packages/numpy
numpy_version: 1.19.1
tensorflow: C:\Users\Mezeix\AppData\Local\R-MINI~1\envs\R-RETI~1\lib\site-packages\tensorflow\__init__.p
NOTE: Python version was forced by use_python function
> tf$constant("Hellow Tensorflow")
Tensor("Const_1:0", shape=(), dtype=string)

R Reticulate: 'Unable to locate conda environment' when attempting to use_condaenv(required = TRUE)

I see variations of this question have been asked but none of the answers have resolved my problem. I'm using Windows 10, R version 3.6.2, RStudio version 1.2.5033, reticulate version 1.14
> library(reticulate)
> library(dplyr)
> conda_list()
name python
1 r-miniconda C:\\Users\\caleb\\AppData\\Local\\r-miniconda\\python.exe
2 r-reticulate C:\\Users\\caleb\\AppData\\Local\\r-miniconda\\envs\\r-reticulate\\python.exe
So I try to use the second item:
> conda_list()[[2]][2] %>%
+ use_condaenv(required = TRUE)
But I get the following error:
Error in use_condaenv(., required = TRUE) :
Unable to locate conda environment 'C:/Users/caleb/AppData/Local/r-miniconda/envs/r-reticulate/python.exe'.
Looking for available versions gets me:
> py_discover_config()
python: C:/Users/caleb/AppData/Local/r-miniconda/envs/r-reticulate/python.exe
libpython: C:/Users/caleb/AppData/Local/r-miniconda/envs/r-reticulate/python36.dll
pythonhome: C:/Users/caleb/AppData/Local/r-miniconda/envs/r-reticulate
version: 3.6.10 |Anaconda, Inc.| (default, Jan 7 2020, 15:18:16) [MSC v.1916 64 bit (AMD64)]
Architecture: 64bit
numpy: C:/Users/caleb/AppData/Local/r-miniconda/envs/r-reticulate/Lib/site-packages/numpy
numpy_version: 1.18.1
This is my first attempt use Python from R so I'm confident this is a user-error issue but searching the error message and looking through related stack threads hasn't helped. Does anyone see what I'm doing wrong here?
The use_condaenv function expects an environment name, not the path to the python executable. So just use
use_condaenv("r-reticulate")
rather than trying to call use_condaenv("C:\\Users\\caleb\\AppData\\Local\\r-miniconda\\envs\\r-reticulate\\python.exe")

Can't figure out how to use conda environment after reticulate::use_condaenv(path)

I created a conda environment using the terminal:
conda create --name pathfinder_example_proj_env python=3.6 feather-format=0.4.0 statsmodels=0.9.0
I also created a trivial python script
import feather
import pandas as pd
import statsmodels.api as sm
print("Done")
In an R notebook, I now want to run that script from within the conda environment I created earlier.
I tried:
reticulate::use_condaenv("pathfinder_example_proj_env", required = TRUE)
reticulate::source_python("../python/python_model.py")
But I get the following error:
Error in py_run_file_impl(file, local, convert) : ImportError: No module named feather
When I check the version of python reticulate is using I get:
reticulate::py_config()
python: /usr/bin/python
libpython: /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config/libpython2.7.dylib
pythonhome: /System/Library/Frameworks/Python.framework/Versions/2.7:/System/Library/Frameworks/Python.framework/Versions/2.7
version: 2.7.10 (default, Oct 6 2017, 22:29:07) [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.31)]
numpy: /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy
numpy_version: 1.8.0
python versions found:
/usr/bin/python
/Users/bradcannell/anaconda/bin/python
/Users/bradcannell/.virtualenvs/bradcannell-_MDC9FPE/bin/python
I checked for available versions using py_discover_config()
reticulate::py_discover_config()
python: /usr/bin/python
libpython: /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config/libpython2.7.dylib
pythonhome: /System/Library/Frameworks/Python.framework/Versions/2.7:/System/Library/Frameworks/Python.framework/Versions/2.7
version: 2.7.10 (default, Oct 6 2017, 22:29:07) [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.31)]
numpy: /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy
numpy_version: 1.8.0
python versions found:
/usr/bin/python
/Users/bradcannell/anaconda/bin/python
/Users/bradcannell/.virtualenvs/bradcannell-_MDC9FPE/bin/python
/Users/bradcannell/anaconda/envs/pathfinder_example_proj_env/bin/python
And as you can see, the virtual environment is listed. I'm just not sure how to use it.
I've read all the articles on the reticulate website:
https://rstudio.github.io/reticulate/index.html
I also found a couple of threads on Github:
https://github.com/rstudio/reticulate/issues/1
https://github.com/rstudio/reticulate/issues/292
This works for me;
library(reticulate)
myenvs=conda_list()
envname=myenvs$name[2]
use_condaenv(envname, required = TRUE)
# or
use_condaenv("r-miniconda", required = TRUE)
restart r session is needed sometimes.
I found the solution here: https://community.rstudio.com/t/reticulate-source-python-and-exec-problems/7386/6
After installing the development version of reticulate (devtools::install_github("rstudio/reticulate") reticulate uses the conda environment as expected.
Leaving this post up in case anyone else runs into this issue.
This thing worked:
By setting the value of the RETICULATE_PYTHON environment variable to a Python binary. Note that if you set this environment variable, then the specified version of Python will always be used (i.e. this is prescriptive rather than advisory). To set the value of RETICULATE_PYTHON, insert Sys.setenv(RETICULATE_PYTHON = PATH) into your project’s .Rprofile, where PATH is your preferred Python binary.

R, TensorFlow, Anaconda Install on Windows

I'm tryiing to install tensorflow for R. For this purpose I installed the latest version of Anaconda (4.3.1) to my OS and installed everything as described in:
https://rstudio.github.io/tensorflow/installation.html
To get started, install the tensorflow R package from GitHub as follows:
devtools::install_github("rstudio/tensorflow")
Then, use the install_tensorflow() function to install TensorFlow:
library(tensorflow)
install_tensorflow()
The result of this process is a folder ~\AppData\Local\conda\conda\envs\r-tensorflow which should be used by Anaconda as an Evironment. This environment is not recognized by the current version of Anaconda.
But the output in R is:
Error: Python module tensorflow was not found.
Detected Python configuration:
python: C:\PROGRA~3\ANACON~1\python.exe
libpython: C:/PROGRA~3/ANACON~1/python36.dll
pythonhome: C:\PROGRA~3\ANACON~1
version: 3.6.0 |Anaconda 4.3.1 (64-bit)| (default, Dec 23 2016, 11:57:41) [MSC v.1900 64 bit (AMD64)]
Architecture: 64bit
numpy: C:\PROGRA~3\ANACON~1\lib\site-packages\numpy
numpy_version: 1.11.3
tensorflow: [NOT FOUND]
No comes my guessing:
Anaconda 4.3.1 does use the folder C:\ProgramData\Anaconda3\pkgs for its packages and does not use the folder above at all. This might be the cause why tensorflow is not found.
The described process from https://rstudio.github.io/tensorflow/installation.html does not match the latest version of anaconda.
Can you show me how to get tensorflow to work with the latest versions of R, tensorflow and Anaconda?
I tried to install r-tensorflow to the global Anaconda environment via
conda install -c conda-forge r-tensorflow
It is not recognized by the environment loaded in R. Can I change the environment of tensorflow directly in R?
Ok, I found out that in the case of tensorflow and Anaconda used I need to force Anaconda to use the Tensorflow environment before i can use it. This is not included in the R tensorflow library so far and must be set by the reticulate library:
#set anaconda to tensor flow environment
library(reticulate)
use_condaenv("r-tensorflow")
#Alternative
use_condaenv(condaenv = "r-tensorflow", conda = "YOUR_ANACONDA_PATH")
I posted an issue here:
https://github.com/rstudio/tensorflow/issues/119
My suggestion is to install using the Anaconda Navigator
step 1:
Open Anaconda Navigator and change the Application on from "Root to Tensorflow" channel (top left)
Step 2:
Now install the R available below

Resources