Why curl package in R is not connecting to the internet? - r

EDIT: Despite having confirmed that this error is ISP-related. I still do not understand why system curl works fine. (error in curl (R) occurs in any site)
EDIT: Currently after a ISP visit. It works fine with cable, but wifi functioning is limited to a ISP router. Error persists with wifi with a personal router or the ISP modem.
Could it be something related to closed ports? (Which would be closed only in the problematic ISP)
I am getting a normal result in system console for:
curl -I https://www.gnu.org/
However the curl package in R gives: (error using the problematic ISP only)
con <- curl("https://gnu.org")
readLines(con)
# Error in readLines(con) :
# Timeout was reached: [gnu.org] Resolving timed out after 10000 milliseconds
The RCurl package works fine, but what I am trying to use requires curl (R)

Related

Hiveserver2: could not start ThriftHttpCliService

I'm attempting to enable SSL on hiveserver2.
I can run in the default binary mode fine. http mode works no problem. As soon as I enable SSL through hive-site.xml, i'm faced with the following error.
ERROR [Thread-28] thrift.ThriftCLIService: Error starting HiveServer2: could not start ThriftHttpCLIService
java.net.BindException: Address already in use
There is nothing using any of the ports, prior to starting hiveserver2. Checked with netstat -tupln
Ports i've configured in hive-site.xml are
hive.server2.webui.port 11002
hive.server2.thrift.http.port 11001
hive.server2.thrift.port 11000
and invoking hiveserver2 via the service /opt/hive/bin/hive --service hiveserver2 &
O/S ubuntu (on kubernetes)
Hive version 3.0.0
Any help greatly appreciate. Google search for problems with ThriftHTTPCliService came up short.
For anyone that come across this post.
I upgraded to Hive 3.1.0, along with the metastore schema.
This fixed the issue, although unsure as to the underlying cause.

How to connect R Studio to Tableau?

I'm trying to connect R Studio to Tableau Desktop to do some data analysis work, but an error has occurred during connection saying: localhost:6311: Connection refused
I'm using MacOS version 10.13.6
Coding on R:
install.packages("Rserve")
library(Rserve)
Rserve()
Try adding the following parameter to your Rserve() line which will hard-code the port:
Rserve(port = 6311)
If that doesn't work, it is worth troubleshooting the port with the following command in terminal (telnet might need to be installed as it is not installed by default):
telnet localhost 6311
the return from the telnet command should be
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Rsrv0103QAP1
More information on the above here
If the command returns a failure, the problem is certainly outside of Tableau.
Some thoughts from there would be to edit the R config file to explicitly accept remote connections.

How to resolve RSelenium error message "Failed to connect to localhost port 4444: Connection refused"?

I am trying to use RSelenium with Dockerto crawl a website. However, I have some issues trying to get RSelenium/Docker to work.
Specifically, I installed Docker on my computer, which appears to be running fine (I see the image of the whale below when I open it).
In R, I then run the following code with no problems and see the expected output.
shell('docker run -d -p 4445:4444 selenium/standalone-chrome')
shell('docker ps')
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
d7de815ce644 selenium/standalone-chrome "/opt/bin/entry_poin…" 13 minutes ago Up 13 minutes 0.0.0.0:4445->4444/tcp zen_mclean
But when I then run the following code, I always receive the following error message:
remDr <- RSelenium::remoteDriver(remoteServerAddr = "localhost",
port = 4444,
browserName = "chrome")
remDr$open()
[1] "Connecting to remote server"
Error in checkError(res) :
Undefined error in httr call. httr output: Failed to connect to localhost port 4444: Connection refused
I am not sure what is going on here (I'm new to scraping). Can anybody help me figure out what to do here?
If it helps, I am running Windows 10.
In docker, you've binded your hosts port 4445 to the selenium-driver port 4444.
Which means if you run R in your host, you need to specify port = 4445
Does that solve it?
I managed to solve the problem by uninstalling Docker Toolbox and VMBox, which I was using, and installing the latest version of Docker from their website instead.

R: Connect to SFTP with RCurl

I'm trying to connect to a SFTP server. It is an encrypted server that uses FIPS mode. I am able to connect and perform file transfer through WinSCP, FileZilla, and through bash sftp commands with no problems.
But, I cannot, access this same SFTP from R using RCurl (R version=3.3.2, RCurl version=1.95-4.10, windows 10). This is what my code looks like and the error message that is being produced:
RCurl::ftpUpload(what="path/to/my/local/file.ext",
to = "sftp://my.eftp.server:portNumber/path/to/my/file.ext",
userpwd = "user:password",
.opts=curlOptions(verbose=TRUE))
The error message is:
* Trying ###.###.###.##...
* Connected to my.eftp.server (###.###.###.##) port ## (#0)
* Failure establishing ssh session
* Closing connection 0
Error in function (type, msg, asError = TRUE) :
Failure establishing ssh session
Any help would be wonderful. I've referenced the following with no luck:
sftp with R - sftp not a protocol with RCurl,
Using RCurl with SFTP
and the RCurl documentation (and other links).
I've also tried:
RCurl::ftpUpload(what="path/to/my/local/file.ext",
to = "sftp://user:password#my.eftp.server:portNumber/path/to/my/file.ext",
.opts=curlOptions(verbose=TRUE))
Also, I've checked my curlVersion()$protocols and scp and sftp protocols are listed (with the others).

Using NGrok with POW

I've almost got NGrok working with POW, but it is resolving to the POW without drilling down to the application running on POW.
I can access my local application running on POW at:
myapp.dev
However when I initialize Ngrok:
/Applications/ngrok -subdomain=myapp myapp.dev:80
As outlined here, I receive the following output from NGrok:
Tunnel Status online
Version 1.6/1.5
Forwarding http://myapp.ngrok.com -> 127.0.0.1:80
Forwarding https://myapp.ngrok.com -> 127.0.0.1:80
Web Interface 127.0.0.1:4040
# Conn 0
Avg Conn Time 0.00ms
Loading http://myapp.ngrok.comresolves to POW, but I get the generic message:
Pow is installed
You’re running version 0.4.1
How can I setup NGrok to resolve to my .dev domain?
Might be a bit late to the party, but here it goes.
It seems like you need to modify your Host header.
For example:
ngrok http -host-header=myapp.dev mayapp.dev:80
This answer only works with ngrok v2.
More info can be found at https://ngrok.com/docs#http
I was using the Powder gem and doing "powder restart", which only works for the local site. You might need to restart pow as described in the manual: http://pow.cx/manual.html#section_3.3

Resources