web scraping with R in country which twitter is blocked - r

I am trying to extract some data with rtweet package in R. The problem is that twitter is blocked in Iran and I have to use a software to pass the filtering. Unfortunately, I am facing this error:
Error in curl::curl_fetch_memory(url, handle = handle) :
Failed to connect to api.twitter.com port 443: Connection refused
I am following regular steps for creating a token:
create_token(
app = "#######",
consumer_key = "#########",
consumer_secret = "############",
access_token = "#############",
access_secret = "############")
Do you have any idea?
Thanks,

Ok, I resolved the problem. Here we go: You should not use proxifiers! The best way is to use a VPN to change the whole path of internet on your system.

Related

Connecting to Azure Data Lake by R using SAS Token

I am trying to connect to Azure DataLake Gen2 via R. I get from Azure DataLake administrator read access to given folder.
I have:
blob_service = 'https://DataLakeName.blob.core.windows.net'
data_lake_storage = 'https://DataLakeName.dfs.core.windows.net'
pathToFolder = 'path/to/folder/with/acess'
blob_sas_token = 'blobksastoken1232133210'
blob_sas_url = 'blob_service/pathToFolder?blob_sas_token'
Accoring to information about library AzureStor I tried:
library(AzureStor)
end_point <- blob_endpoint(endpoint = "https://DataLakeName.blob.core.windows.net/", sas = blob_sas_token)
list_blob_containers(end_point)
Unfortunately I got HTTP status code 500, what means probably an internal server error. Administratior said that token is correct and endpoint also works fine.
Error in process_storage_response(response,
match.arg(http_status_handler), : Internal Server Error (HTTP
500). Failed to complete Storage Services operation. Message:
InternalError Server encountered an internal error. Please try again
after some time.
Any idea what I am doing wrong, or how can I connect to ADL in other way using data that I have?

Error in Oauth2.0 authentication in R using HTTR (failed to connect: connection refused)

I am trying to create a simple app (in R) using GroupMe's API, which utilizes OAuth2.0. The documentation can be found here. However, I'm getting stuck on the first step of authentication/token generation for a user. See below for my code and the response I get:
access_key = ****
client_id = ****
gendpoint <- oauth_endpoint(
authorize = glue("https://oauth.groupme.com/oauth/login_dialog?client_id={client_id}"),
access = glue("http://localhost:1410/?access_token={access_key}")
)
gapp <- oauth_app(
"pingme",
key = client_id,
secret = access_key
)
t <- oauth2.0_token(
endpoint = gendpoint,
app = gapp
)
The above code is sufficient to bring in the login page, which presumably allows me to enter my credentials to obtain a token. However, when I enter my credentials, I get the following error message in R:
Authentication complete.
Error in curl::curl_fetch_memory(url, handle = handle) :
Failed to connect to localhost port 1410: Connection refused
So it looks like my authentication/login credentials worked, but somewhere an error prevented me from actually generating the token.
Could someone help me with this? This is my first time using the OAuth2.0 framework so I'm very confused. Thanks in advance!

How Do I Connect to a Neo4J AuraDB Instance from R Studio

I created a Neo4j AuraDB database where I have dumped the movie recommendation dataset. I am able to start and connect to the instance in the cloud. However, when I try to connect to the instance via its API in R Studio, using the neo4r package and the following code
movieDB <- neo4j_api$new(
url = curl("neo4j+s://0cc45a14.production-orch-0054.neo4j.io:7687"),
user = "neo4j",
password = "password"
)
movieDB$ping()
I get the error message in my console Error: Could not resolve host: 7. Also, when I try to start a graph with the following code
graph <- startGraph(
url = curl("neo4j+s://0cc45a14.production-orch-0054.neo4j.io:7687"),
user = "neo4j",
password = "password"
)
I also get the following error message in my console
Error in function (type, msg, asError = TRUE) :
Could not resolve host: 8
I do not know why the error is happening, as I have previously connected to a Neo4j sandbox instance from within R Studio without any hassle. As always, I will appreciate your helpful suggestions. Thanks!
AuraDB requires a driver which supports the Bolt protocol. I do not know of any R driver that supports the protocol at the moment, they only support HTTP (and HTTP is not available yet in AuraDB).

Retrieve a Users Dashboard in Tumblr with R and TumblR. Oauth Issues

I am trying to use the TumblR package in R to set up the Oauth Authentication to Retrieve a user's dashboard using the second example in
tumblR documentation
However I get the following error, it seems that using twitter others have been able to use a different function to get around this, but I am not finding the same function available for Tumblr.
See twitter package for R authentication: error 401
My code
consumer_key <- OKey
consumer_secret <- SKey
appname <- App_name
tokenURL <- 'http://www.tumblr.com/oauth/request_token'
accessTokenURL <- 'http://www.tumblr.com/oauth/acces_token'
authorizeURL <- 'http://www.tumblr.com/oauth/authorize'
app <- oauth_app(appname , consumer_key, consumer_secret)
endpoint <- oauth_endpoint(tokenURL, authorizeURL, accessTokenURL)
token <- oauth1.0_token(endpoint, app)
The error I am receiving is the following.
Error in init_oauth1.0(self$endpoint, self$app, permission =
self$params$permission, : Unauthorized (HTTP 401)
I am using R version 3.4.0 and Rstudio Version 1.1.463
Reply from the package maintainer.
I'm sorry for the delay in the reply, I'm currently out for work.
The problem you reported depends on the change in the http protocol. The API URLs are now passed to https.
You can use
tokenURL <- 'https://www.tumblr.com/oauth/request_token'
accessTokenURL <- 'https://www.tumblr.com/oauth/acces_token'
authorizeURL <- 'https://www.tumblr.com/oauth/authorize'
I realized it only thanks to your report.
During the Christmas holidays, I plan to arrange the package.
If you have other evidence, let me know.

twitteR package for R authentication: error 401

Hy everyone/anyone and thank you for considering it!
I'm stuck at the final stage of twitteR package authentication. After a successful cred$handshake as I'm trying to end up the authentication with setup_twitter_oauth, the program keeps throwing the following message:
Error in init_oauth1.0(self$endpoint, self$app, permission =
self$params$permission) : client error: (401) Unauthorized
Any idea what this could be and how could I get around it?
Not sure the precise source of your error, and I had used this package about 2 months ago. I ran into errors doing the handshake as well.
To work-around, I avoided the explicit handshake by using setup_twitter_oauth directly.
consumerKey <- "[your Twitter consumer key]"
consumerSecret <- "[your Twitter consumer secret]"
accessToken <- "[your Twitter access token]"
accessTokenSecret <- "[your Twitter access token secret]"
setup_twitter_oauth(consumerKey, consumerSecret, accessToken, accessTokenSecret)
This seemed to work for me. The various OAuth parameters come from Twitter. I wrote up how to do that here: Newborn app using Twitter and R.
The complete R source code is on BitBucket (the stuff at the bottom is probably most relevant): newborn-app.R.

Resources