I am looking to use Bing API to extract data in R. When I am using the following query in my browser
https://api.datamarket.azure.com/Bing/Search/Web?Query=%27analytics%27&$format=json
I am getting the search results in json format. But I have to enter the UserID (Blank) and password (authorization token) for authorization first.
How can I merge the query with the authorization token in R so that I can directly load the search results in my R environment?
You can use authenticate function as a part of httr package:
library(httr)
response = GET(url = 'https://api.datamarket.azure.com/Bing/Search/Web?Query=%27analytics%27&$format=json',
authenticate('','auth token', type = 'basic'))
res = content(response, encoding = 'json')
Related
I'm trying to generate an access token for my Flickr app in httr. My goal is to download images from certain geographical co-ordinates. To my understanding this isn't something the FlickrAPI package can do, so I've decided to use httr.
Running the below code brings up the authorization page but when I submit the authorization I get a localhost error (localhost unexpectedly closed the connection).
What am I forgetting here? I haven't set a callback URL, if that's relevant.
library(httr)
setwd("G:\\Project\\R Code\\")
api_key <- "<TOKEN>"
secret <- "<SECRET>"
flickr.app <- oauth_app("App Name", api_key, secret)
flickr.endpoint <- oauth_endpoint(
request = "https://www.flickr.com/services/oauth/request_token",
authorize = "https://www.flickr.com/services/oauth/authorize",
access = "https://www.flickr.com/services/oauth/access_token"
)
token <- oauth1.0_token(
flickr.endpoint,
flickr.app,
cache = FALSE
)
I have to post some JSON to a restful endpoint and decided to do this in R. I am aware of the httr library. However, I am not sure how to add an access token for Azure active directory. This is some code:
library(httr)
access_token <- "SomeSecretStuffeyJ0eXAiOiJKV"
json <- '{"test_data":"hello_world"}'
uri <- "https://somewhere.net/bla"
result <- POST(
uri
, content_type_json()
, body = json
)
I am not sure if I can do this using httr? Could someone please help out with code and/or suggest alternative library? Thanks!
I want to get data from a tweet.
But these are the errors.
Please help me get data?
by code
install.packages("rtweet")
library(rtweet)
# plotting and pipes - tidyverse!
library(ggplot2)
library(dplyr)
# text mining library
library(tidytext)
# whatever name you assigned to your created app
appname <- "**********"
## api key (example below is not a real key)
key <- "**************"
## api secret (example below is not a real key)
secret <- "***********"
twitter_token <- create_token(
app = appname,
consumer_key = key,
consumer_secret = secret)
rstats_tweets <- search_tweets(q = "#rstats",
n = 500)
Error in check_twitter_oauth (): OAuth authentication error:
This most likely means that you have incorrectly called setup_twitter_oauth () '
Error in curl :: curl_fetch_memory (url, handle = handle):
Timeout was reached: Connection timed out after 10,000 milliseconds
The documentation claims:
API authorization
The first time you make an API request—e.g., search_tweets(), stream_tweets(), get_followers()—a browser window will open.
Log in to your Twitter account.
Agree/authorize the rtweet application.
And that’s it!
However, this did not work for me.
I had to follow the rtweet tutorial how obtaining and using access tokens:
# install from CRAN
install.packages("rtweet")
# load rtweet
library(rtweet)
I also had to install the httpuv package. This is not mentioned in the tutorial but you get a corresponding error message in R.
Error in oauth_listener(authorize_url, is_interactive) :
httpuv package required to capture OAuth credentials.
install.packages("httpuv")
Then Create your Twitter app with the correct Callback URL on apps.twiter.com:
To create a Twitter app, navigate to apps.twitter.com and create a new app by providing a Name,
Description, and Website of your choosing (example screenshot provided
below).
Important In the Callback URL field, make sure to enter the following: http://127.0.0.1:1410
Check yes if you agree and then click “Create your Twitter application”.
## whatever name you assigned to your created app
appname <- "rtweet_token"
## api key (example below is not a real key)
key <- "XYznzPFOFZR2a39FwWKN1Jp41"
## api secret (example below is not a real key)
secret <- "CtkGEWmSevZqJuKl6HHrBxbCybxI1xGLqrD5ynPd9jG0SoHZbD"
## create token named "twitter_token"
twitter_token <- create_token(
app = appname,
consumer_key = key,
consumer_secret = secret)
then, finally, a browser window opened after this code is executed (something like the following):
http://localhost:1410/?oauth_token=asdfasdfasdfaf5naw&oauth_verifier=dffasdfsdfsdfsdfsdfsk9jIQxDG6
showing the message:
Authentication complete. Please close this page and return to R.
and now you should be good to go.
See also:
https://developer.twitter.com/en/docs/basics/authentication/overview/oauth
I am trying to access the QuickBooks online API via R. Here is what I have so far:
library(httr)
library(httpuv)
endPoint <- oauth_endpoint(request = NULL,
authorize = "https://appcenter.intuit.com/connect/oauth2",
access = "https://oauth.platform.intuit.com/oauth2/v1/tokens/bearer")
App <- oauth_app("Untitled",
key = "xxxx",
secret = "xxxxx",
redirect_uri = "http://localhost:1410/")
QBOtoken <- oauth2.0_token(endpoint = endPoint,
app = App,
scope = "com.intuit.quickbooks.accounting",
type = "code",
cache = T)
GET("https://sandbox-quickbooks.api.intuit.com/v3/company/193514718345164/query?query=Select * from Payment", config(token = QBOtoken))
When run the code above through QBOtoken, I go through the whole Oauth2.0 "dance" and get the response:
Waiting for authentication in browser...
Press Esc/Ctrl + C to abort
Authentication complete.
However, when I execute the GET command, it returns:
Error in self$credentials$access_token :
$ operator is invalid for atomic vectors
The .httr-oauth file that is generated is filled with 284 rows of 32 characters. Is that normal?
I have connected to the API via Postman and it generates an access token allowing me to execute queries like the GET request in my code. There is also an Oauth 2.0 playground on Intuit Developer. Somehow, in R, I am not getting the access token and refresh token.
Another concern, is that I am currently trying to connect to a development environment. For QuickBooks Online, the redirect URL can be the localhost in the development environment, but if I want to connect to the production company(my company) data I will need a https redirect URL. Will this be possible through R?
My ultimate goal for this project is for the script to automatically run on a nightly basis, connect to the API, and ETL the data into a Relational Database for reporting/analytics purposes. Any help would be greatly appreciated!
I did get this "functional," but it is by no means good code. Some of this, like storing tokens in a csv, is not best practice, just a band-aid to test the code.
I created a tokens.csv file in the structure:
"RefreshToken","AccessToken"
"<RefreshToken>","<AccessToken>"
Below is my script that retrieved customer data from the QBO api sandbox.
library(httr)
library(httpuv)
library(curl)
library(jsonlite)
library(base64enc)
#Client ID and Client Secret were retrieved from the online explorer
clientID <- "<ClientID>"
clientSecret <- "<ClientSecret>"
scope <- "com.intuit.quickbooks.accounting"
tokens <- read.csv("tokens.csv")
RefreshToken <- as.character(tokens$RefreshToken[1])
AccessToken <- as.character(tokens$AccessToken[1])
authorize <- base64enc::base64encode(charToRaw(paste0(clientID,":",clientSecret)))
oauth_refresh <- httr::POST("https://oauth.platform.intuit.com/oauth2/v1/tokens/bearer",
add_headers('Content-Type'= "application/x-www-form-urlencoded",
'Accept'= 'application/json',
'Authorization'= paste0('Basic ',authorize)
),
body = list('grant_type'='refresh_token',
'refresh_token'=RefreshToken),
encode = "form")
oaJSON <- fromJSON(content(oauth_refresh, as = "text"))
RefreshToken <- oaJSON[["refresh_token"]][1]
AccessToken <- oaJSON[["access_token"]][1]
tokens <- as.data.frame(list('RefreshToken'=RefreshToken,'AccessToken'=AccessToken))
write.csv(tokens,file = "tokens.csv", row.names = F)
datas <- httr::GET("https://sandbox-quickbooks.api.intuit.com/v3/company/<ID>/query?query=SELECT%20%2a%20FROM%20Customer",
accept_json(),
add_headers('Authorization'= paste0("Bearer ",AccessToken))
)
#datas$status_code
j_son <- content(datas, as = "text")
customers <- fromJSON(j_son)
customer_df <- customers$QueryResponse$Customer
Hopefully this gets the ball rolling correctly for you. Let me know if you have any feedback!
The only way I've found to get myself authenticated with the twitter API is the following:
library(twitteR)
setup_twitter_oauth(consumer_key = "a",
consumer_secret = "b",
access_token = "c",
access_secret = "d")
After running this, I can use all functions in twitteR just fine. However, I would also like to use the streamR package, which needs the token as an OAuth object:
filterStream("tweets.json", track = c("Obama", "Biden"), timeout = 20, oauth=my_oauth)
From what I gather, the setup_twitter_oauth function above is a wrapper around some httr functions to get my authorization token. This token is cached in my working directory as a file called ".httr-oauth". My question is: how do I load this file into R, such that I get an OAuth object that I can use with streamR?
Use readRDS()
readRDS('.httr-oauth')
$xxxx0x000xxxx00000x0xx0x000000xx
request: https://api.twitter.com/oauth/request_token
authorize: https://api.twitter.com/oauth/authenticate
access: https://api.twitter.com/oauth/access_token
twitter
key: xxxxxxxxxx0xxxxxxxxxxxxxx
secret:
oauth_token, oauth_token_secret, user_id, screen_name
Access the environment in the list through $long-alphanumeric-hash and within that access $credentials and $oauth_token/$oauth_token_secret
This is a hack and doesn't directly retrieve the OAuth object from setup_twitter_oauth , but it works (adapted from http://www.datablog.sytpp.net/2014/04/scraping-twitter-with-r-a-how-to/).
Do the below once after you've setup your consumer_key and consumer_secret
twitCred <- OAuthFactory$new(consumerKey=consumer_key,
consumerSecret=consumer_secret,
requestURL="https://api.twitter.com/oauth/request_token",
accessURL="https://api.twitter.com/oauth/access_token",
authURL="http://api.twitter.com/oauth/authorize")
twitCred$handshake()
save(twitCred, file="credentials.RData")
When mixing TwitteR and streamR, use twitCred as the OAuth for streamR calls
twitCred<- NULL
load("credentials.RData")
Sample test streamR call to retrieve Tweets relating to football
foo<- filterStream(file.name="",track =c("Football","NFL"),oauth=twitCred,timeout=30)