install.keras() in RStudio fails with http connection error - r

I've been trying to install and run keras in RStudio (Windows) in vain.
i installed keras package using normal package "keras"
(didn't use github)
I've installed latest python (3.6) and Anaconda.
then i use
> library(keras)
> install.keras()
and i get this error:
Creating r-tensorflow conda environment for TensorFlow installation...
Fetching package metadata ... CondaHTTPError: HTTP 000 CONNECTION
FAILED for url
https://repo.continuum.io/pkgs/main/win-64/repodata.json.bz2
Elapsed: -
An HTTP error occurred when trying to retrieve this URL. HTTP errors
are often intermittent, and a simple retry will get you on your way.
ConnectTimeout(MaxRetryError("HTTPSConnectionPool(host='repo.continuum.io',
port=443): Max retries exceeded with url:
/pkgs/main/win-64/repodata.json.bz2 (Caused by
ConnectTimeoutError(, 'Connection to repo.continuum.io timed out.
(connect timeout=9.15)'))",),)
Error: Error 1 occurred creating conda environment r-tensorflow In
addition: Warning message: running command
'"C:\PROGRA~3\ANACON~1\Scripts\conda.exe" "create" "--yes" "--name"
"r-tensorflow" "python=3.6"' had status 1
I've looked up everywhere on the web and can't figure out how to install keras and tensorflow properly. Using latest version of R (3.4.2)
Every method fails somewhere.
just to add to misery, i've also tried:
> devtools::install_github("rstudio/keras")
and i get this error:
Installation failed: Timeout was reached: Connection timed out after
10015 milliseconds
I am not behind any authenticated proxies. So, after multiple failure, i just downloaded the zip file from github and manually installed it using the zip file.
i also tried install.packages("keras") and that didn't give me any error either.
when i call the library i don't get any errors (as shown above)
UPDATE: I was able to install and use the package very easily on another computer that doesn't have python/anaconda installed on it already.
UPDATE 2: my proxy does not need authentication and there is no https_proxy either.

OK,, FINALLY found a solution.
Turns out RStudio uses a lot of default proxy settings, so i needed to change all that and set up my own proxy settings.
First step:
Rstudio --> Tools --> Global Options --> packages --> uncheck both "Use secure download method for HTTP" and "Use Internet Explorer librayr/proxy for HTTP"
Second step, in RStudio type:
> file.edit('./.Renviron')
Either an empty file or some file with already existing proxy settings will open. (Mine was empty). Then I included the following two:
http_proxy=http://myusename:password#proxy.server.com:port/
https_proxy=http://myusename:password#proxy.server.com:port/
(a few notes: I didn't have a https_proxy setting but I still needed to use the http_proxy details for my https_proxy setting. This was one of the culprits for my issue. Also, I needed to include the username:password even though my proxy doesn't need secure authentication. Same thing goes with the port. Port number had to be included, otherwise it wouldn't work.
Step 3:
Saved the new changes in .Renviron file and restarted RStudio.
I checked my proxy settings in RStudio after restart by typing:
> Sys.getenv("http_proxy")
> Sys.getenv("https_proxy")
The first few times i did this i realised that the proxy settings were not being changed in RStudio because i was editing the wrong .Renviron file. So, it's best to use file.edit('~/.Renviron') in step 2 to make sure it's the right file.
After all this, when i ran install.keras(), it installed successfully, including installing Tensorflow. Again, initially i had skipped step 1 so keras started being installed but it failed at installing tensorflow.
It was only going through all the steps that i was able to install both keras and tensorflow successfully over a proxy. Hope this helps.

Uninstalling Anaconda3 and installing Anaconda2 (i.e. Python 2.7) did the trick for me: https://www.anaconda.com/download/

Related

Failed to install 'unknown package' from GitHub

I am trying to install the ggpattern package from GitHub (https://www.rdocumentation.org/packages/ggpattern/versions/0.2.0)
I've reinstalled R, followed the all steps according to the site, also tried
remotes::install_github("coolbutuseless/ggpattern", force = TRUE)
But I still get:
Error: Failed to install 'unknown package' from GitHub:
HTTP error 401.
Bad credentials
Rate limit remaining: 19/60
Rate limit reset at: 2022-01-29 18:28:15 UTC
I'm working on R version 4.1.2 (newest according to me) on Windows.
Do you have any idea what is the issue here?
You need to check if you have a personal access token set in your environment. For example, when I have a Git project, I set a personal access token. However, I set this in the project environment, so that it isn't any issues outside of that environment.
To see if there is one assigned:
Sys.getenv("GITHUB_PAT")
If there is one set, write it down (you may need that in the future).
To remove it, so you can install the GitHub package:
Sys.unsetenv("GITHUB_PAT")

Getting: "Failed to fetch metadata" when starting up Jupyter

I am using JupyterLab as my IDE and I have a couple of packages installed. Namely, the 'jupyterlab-dash' and 'juptyerlab-plotly' packages. My issue is when I go to launch 'juptyer lab' from terminal, I notice the following error:
Failed to fetch package metadata for 'jupyterlab-dash': URLError(gaierror(8, 'nodename nor servname provided, or not known'))
I'm not sure why this error is popping up but I've noticed this error only pops up when I have no internet connection (working on a train etc.). Could it be because these packages are trying to call something as I launch and because I don't have an internet connection it raises the error?
In the end I am able to use JupyterLab and the packages as intended (at-least it seems), but I'm curious why this error to 'fetch metadata' appears?
Thanks,

Installing a package from private GitLab server on Windows

I am struggling with installing a package from a GitLab repository on a Windows computer.
I found different hints but still have problems to install my package from GitLab. First of all, I generated a public and private key with puttygen.exe. The files need to be changed afterwards, I had to remove comments and stuff so they look like my the file on my Unix system. So now, both public and private key files have just a single line.
I tried to install my package via devtools::install_git which takes very long and I get the error message
Error: Failed to install 'unknown package' from Git:
Error in 'git2r_remote_ls': Failed to authenticate SSH session: Unable to send userauth-publickey request
And with devtools::install_gitlab I get a different error message and I somehow have the feeling, the link which gets generated doesn't fit to my GitLab server.
Error: Failed to install 'unknown package' from GitLab:
cannot open URL 'https://gitlab.rlp.net/api/v4/projects/madejung%2FMQqueue.git/repository/files/DESCRIPTION/raw?ref=master'
My complete code to test at the moment is
creds <- git2r::cred_ssh_key(publickey="~/.ssh/id_rsa_gitlab.pub",
privatekey="~/.ssh/id_rsa_gitlab")
devtools::install_git(
url='git#gitlab.rlp.net:madejung/MQqueue.git',
quiet=FALSE,
credentials=creds)
devtools::install_gitlab(
repo='madejung/MQqueue.git',
host='gitlab.rlp.net',
quiet=FALSE,
credentials=creds
)
My id_rsa_gitlab.pub file looks like this and is just a single line:
ssh-rsa AAAA....fiwbw== rsa-key-20200121
The id_rsa_gitlab file has just the code:
AAABA.....3WNSIAGE=
Update
On my Mac system it works as expected after installing the libssh2 library via homebrew and and recompiling git2r with install.packages("git2r", type = "source").
So the working code on my machine is:
creds <- git2r::cred_ssh_key(publickey="~/.ssh/id_rsa_gitlab.rlp.net.pub",
privatekey="~/.ssh/id_rsa_gitlab.rlp.net")
devtools::install_git(
url='git#gitlab.rlp.net:madejung/MQqueue.git',
quiet=FALSE,
credentials=creds
)
For some strange reason, the devtools::install_git call needs about a minute to fail in the end. I have no idea where the problem here is.
After struggling for almost a day, I found a solution I can live with...
I first created a PAT (Personal Access Token) in my gitlab account and granted full API access. For some reason the read_only access didn't worked and I am now tired to figure out what the problem is.
After this I had still problems to install my package and for some reason, the wininet setting for downloading doesn't work.
I used the command capabilities("libcurl") to check if libcurl is available on my windows, which was and tried to overwrite wininet to libcurl by using method='libcurl' in the install function. Somehow, this was not enough so I overwrote the options variable download.file.method directly.
options("download.file.method"='libcurl')
devtools::install_gitlab(
repo='madejung/MQqueue',
auth_token='Ho...SOMETHING...xugzb',
host='gitlab.rlp.net',
quiet=FALSE, force=TRUE
)

proxy problems for install_keras with Anaconda and RStudio

I'm trying to install Keras by following the instructions here https://keras.rstudio.com/ when I'm using my office/work laptop.
When I get to this line install_keras() and it fails with this rather long error message shown here in its entirety:
Creating r-tensorflow conda environment for TensorFlow installation...
Fetching package metadata ...
CondaHTTPError: HTTP 000 CONNECTION FAILED for url
<https://repo.continuum.io/pkgs/main/win-64/repodata.json.bz2>
Elapsed: -
An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on
your way.
ProxyError(MaxRetryError("HTTPSConnectionPool(host='repo.continuum.io',
port=443): Max retries exceeded with url: /pkgs/main/win-6
/repodata.json.bz2 (Caused by ProxyError('Cannot connect to proxy.',
NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at
0x000001ACD38FB780>: Failed to establish a new connection: [Errno 11001]
getaddrinfo failed',)))",),)
Error: Error 1 occurred creating conda environment r-tensorflow
In addition: Warning message:
running command '"C:\PROGRA~3\ANACON~1\Scripts\conda.exe" "create" "--
yes" "--name" "r-tensorflow" "python=3.6"' had status 1
and it says something about Anaconda. So I dutifully install the entire Anaconda beast from this Anaconda distribution.
Now when I try install_keras() I get this new error message about a proxy:
Using r-tensorflow conda environment for TensorFlow installation
Determining latest release of TensorFlow...Error in
open.connection(con, "rb") :
Unsupported proxy 'https://proxy-server.mycompanyname.com:8080', libcurl
is built without the HTTPS-proxy support.
I reviewed Jeroen's libcurl suggestion which yields the following responses:
curl::ie_proxy_info()
$AutoDetect
[1] FALSE
$AutoConfigUrl
[1] "http://mcd-server/mcd/proxy.pac"
$Proxy
NULL
$ProxyBypass
NULL
and this
curl::ie_get_proxy_for_url()
[1] "proxy-server:8080"
So I seem to have a good connection to the internet through RStudio but not when using Anaconda's software.
I have verified that I have a good internet connection using this:
httr::BROWSE("https://www.ibm.com")
It appears that libcurl is built without HTTPS-proxy support, apparently.
Any suggestions?
SOLVED!
1) Step 1: allow Anaconda to access the internet by adding proxy info with a new file named .condarc exactly as detailed in this answer. You can verify this works ok by typing conda update conda into the Anaconda Prompt app.
2) Step 2: allow R and RStudio to access the internet by adding these 2 lines into the .Renviron file (in my case this was found in C:\Users\USERNAME\Documents for Windows 10):
http_proxy=http://proxy-server:8080
https_proxy=http://proxy-server:8080
You can now run this to install Keras:
library(keras)
install_keras()

How do I tell the R interpreter how to use the proxy server?

I'm trying to get R (running on Windows) to download some packages from the Internet, but the download fails because I can't get it to correctly use the necessary proxy server. The output text when I try the Windows menu option Packages > Install package(s)... and select a CRAN mirror is:
> utils:::menuInstallPkgs()
--- Please select a CRAN mirror for use in this session ---
Warning: unable to access index for repository http://cran.opensourceresources.org/bin/windows/contrib/2.12
Warning: unable to access index for repository http://www.stats.ox.ac.uk/pub/RWin/bin/windows/contrib/2.12
Error in install.packages(NULL, .libPaths()[1L], dependencies = NA, type = type) :
no packages were specified
In addition: Warning message:
In open.connection(con, "r") :
cannot open: HTTP status was '407 Proxy Authentication Required'
I know the address and port of the proxy, and I also know the address of the automatic configuration script. I don't know what the authentication is called, but when using the proxy (in a browser and some other applications), I enter a username and password in a dialog window that pops up.
To set the proxy, I tried each of the following:
Sys.setenv(http_proxy="http://proxy.example.com:8080")
Sys.setenv("http_proxy"="http://proxy.example.com:8080")
Sys.setenv(HTTP_PROXY="http://proxy.example.com:8080")
Sys.setenv("HTTP_PROXY"="http://proxy.example.com:8080")
For authentication, I similarly tried setting the http_proxy_user environment variable to:
ask
user:passwd
Leaving it untouched
Am I using the right commands in the right way?
You have two options:
Use --internet2 or setInternet2(TRUE) and set the proxy details in the control panel, in Internet Options
Do not use either --internet2 or setInternet2(FALSE), but specify the environment variables
EDIT: One trick is, you cannot change your mind between 1 and 2, after you have tried it in a session, i.e. if you run the command setInternet2(TRUE) and try to use it e.g. install.packages('reshape2'), should this fail, you cannot then call setInternet2(FALSE). You have to restart the R session.
As of R version 3.2.0, the setInternet2 function can set internet connection settings and change them within the same R session. No need to restart.
When using option 2, one way (which is nice and compact) to specify the username and password is http_proxy="http://user:password#proxy.example.com:8080/"
In the past, I have had most luck with option 2
If you want internet2 to be used everytime you use R you could add the following line to the Rprofile.site file which is located in R.x.x\etc\Rprofile.site
utils::setInternet2(TRUE)
I've solved my trouble editing the file .Renviron as documented in Proxy setting for R.
EDITED
The solutions based on the setInternet2 statement do not work with the recent R versions because setInternet2 is declared defunct.
I'm using the 4.2.1 (on Win 11Pro) while I never had any problems in previous versions .
So to solve the problem need to modify some config files in order to fix the proxy issue not only for packages installation but, in general, also to acced to a remote resource (ie. boundary maps in my case).
The question "Proxy setting for R" collect a lot of solutions. I've found that this one has solved both my problems (packages installation and remote resources) explaining step-by-step how to edit the file .Renviron
Other solutions based on the customization of the file Renviron.site for me doesn't work
install.packages("RCurl")
that will solve your problem.

Resources