I'm trying to access the US Census Geocoder batch address API, found here: http://geocoding.geo.census.gov/geocoder/
I've also gone through the documentation for the API here: http://geocoding.geo.census.gov/geocoder/Geocoding_Services_API.pdf
I'm trying to use the httr package in R to post a batch csv file of formatted addresses using this format: Unique ID, Street address, City, State, ZIP
I've tried the single address request version using getURL from RCurl and that works fine, but postForm doesn't seem to submit the file in the right way. The code I'm using right now seems to post the request correctly, but I'm not getting any geocoded data back.
curlhandle <- handle("http://geocoding.geo.census.gov/geocoder/geographies/addressbatch",
cookies = TRUE)
# using fileUpload from RCurl instead of upload_file from httr
upload3 <- fileUpload(contents = address100, contentType = "file")
test <- POST(url="http://geocoding.geo.census.gov/geocoder/geographies/addressbatch",
body = list(addressFile = upload3,
benchmark = "Public_AR_Census2010",
vintage="Census2010_Census2010"),
encode = "multipart",
handle = curlhandle,
followLocation = TRUE,
verbose = TRUE)
Is my request missing something? I'm not sure if I should be using writefunction & writedata in this case. Any help would be appreciated!
You seem to have a weird mix of RCurl and httr. Here's how I'd write the request:
req <- POST(
"http://geocoding.geo.census.gov/geocoder/geographies/addressbatch",
body = list(
addressFile = upload_file(address100),
benchmark = "Public_AR_Census2010",
vintage = "Census2010_Census2010"
),
encode = "multipart",
verbose())
stop_for_status(req)
content(req)
(also "file" is not a valid mime type)
Related
Hopefully a quick question, I'm trying to connect to the KuCoin API, not super relevant as I think this is more an issue with how I'm using the POST function and how it sends JSON along
Here is my function that is supposed to place an order:
API.Order <- function(pair,buysell,price,size) {
path = "/api/v1/orders"
now = as.integer(Sys.time()) * 1000
json <- list(
clientOid = as.character(now),
side = buysell,
symbol=pair,
type="limit",
price=price,
size=size
)
json=toJSON(json, auto_unbox = TRUE)
str_to_sign = (paste0(as.character(now), 'POST', path, json))
signature = as.character(base64Encode(hmac(api_secret,str_to_sign,"sha256", raw=TRUE)))
passphrase=as.character(base64Encode(hmac(api_secret,api_passphrase,"sha256", raw=TRUE)))
response=content(POST(url=url,
path=path,
body=json,
encode="json",
config = add_headers("KC-API-SIGN"=signature,
"KC-API-TIMESTAMP"=as.character(now),
"KC-API-KEY"=api_key,
"KC-API-PASSPHRASE"=passphrase,
"KC-API-KEY-VERSION"="2")
),
"text",encoding = "UTF-8")
response
data.table(fromJSON(response)$data)
}
API.Order(pair,"sell",1.42,1.0)
And everything works, except I get the following response:
"{\"code\":\"415000\",\"msg\":\"Unsupported Media Type\"}"
Which is puzzling to me. Everything else checks out (the signature and other auth headers), and I set the encode to "json" in the POST.. I also can put it as standard "application/json" and neither works. I've been staring at this for hours now and I can't see what (likely very little) thing I got wrong?
Thanks
I am trying to call the stats of a list of players from the call of duty API. This API requires firstly the login in website https://profile.callofduty.com/cod/login. Once logged in, the user can see the stats of a player using the call-of-duty API. For example, the stats of the streamer savyultras90 from Warzone can be seen through the following link: https://my.callofduty.com/api/papi-client/stats/cod/v1/title/mw/platform/psn/gamer/savyultras90/profile/type/wz.
If I log in from the website and try to see the stats of a player and the related json, I am able to do via browser. However, this doesn't seem straightforward in R.
I try to log in using the GET function from httr package as follows
respo <- GET('https://profile.callofduty.com/cod/login', authenticate('USER', 'PWD'))
But when I try to have access to the api and download the JSON file using the function fromJSON from the package jsonlite as follows
data <- fromJSON('https://my.callofduty.com/api/papi-client/stats/cod/v1/title/mw/platform/psn/gamer/savyultras90/profile/type/wz')
I get the error message "Not permitted: not authenticated".
How can I authenticate in one website and stay logged in to call from the API which relies on that authentication?
Seeing I've recently had to develop a PHP API for Warzone, I might be able to guide you in the right direction on how to handle this. But first a few remarks:
You need to authenticate each user individually with the appropriate platform if you want to request that player's data
There is a throttle limit on the amount of API requests
The API of Call of Duty is under strict usage guidelines and should only be used by registered partners. Making usage of the API could result in claims and eventually lawsuits: link
There is no public documentation of the API and the API has changed in the past, breaking several 3rd party tools.
Nevertheless, the process involves several steps as described below:
Register the device making the call
https://profile.callofduty.com/cod/mapp/registerDevice
with a json body in the form of {"deviceId":"INSERT_ID_HERE"}
This will return a response with the authHeader which we will use for as Token in the next calls
Login with Activision credentials
https://profile.callofduty.com/cod/mapp/login
Set the following headers:
Authorization: "INSERT_AUTHHEADER_HERE"
x_cod_device_id: "INSERT_PREVIOUSLY_USED_DEVICEID_HERE"
This in terms will generate a dataset where we will save the following data from:
rtkn, ACT_SSO_COOKIE and atkn.
Make the wanted API call for data
We have all the data now required to make the API call.
For each request we will submit 3 headers:
Authorization: "INSERT_AUTHHEADER_HERE"
x_cod_device_id: "INSERT_PREVIOUSLY_USED_DEVICEID_HERE"
Cookie: ACT_SSO_LOCALE=en_GB;country=GB;API_CSRF_TOKEN=**GENERATE_CSRF_TOKEN**;rtkn=**RTKN_HERE**;ACT_SSO_COOKIE=**ACT_SSO_COOKIE_HERE**;atkn=**ATKN_HERE**
For more reference, you can always look through a Python library or NodeJS library which succesfully implemented the API.
I struggled with this yesterday but finally made some progress. The issue is that you have to obtain an authentication token. The steps can be followed here: https://documenter.getpostman.com/view/7896975/SW7aXSo5#a37a2e5b-84bb-441d-b978-0fd8d42ffd29 but not available in R though.
My code works at first, as long as you don't authenticate again (still trying to figure out why). Basically what I did was to translate the steps in the link and extracted the content in the response from GET:
# Get token ---------------------------------------------------------------
resp <- GET('https://profile.callofduty.com/cod/login')
cookies = c(
'XSRF-TOKEN' = resp$cookies$value[1]
,'new_SiteId' = resp$cookies$value[2]
,'comid' = resp$cookies$value[3]
,'bm_sz' = resp$cookies$value[4]
,'_abck' = resp$cookies$value[5]
# ,'ACT_SSO_COOKIE' = resp$cookies$value[6]
# ,'ACT_SSO_COOKIE_EXPIRY' = resp$cookies$value[7]
# ,'atkn' = resp$cookies$value[8]
# ,'ACT_SSO_REMEMBER_ME' = resp$cookies$value[9]
# ,'ACT_SSO_EVENT' = resp$cookies$value[10]
# ,'pgacct' = resp$cookies$value[11]
# ,'CRM_BLOB' = resp$cookies$value[12]
# ,'tfa_enrollment_seen' = resp$cookies$value[13]
)
headers = c(
)
params = list(
`new_SiteId` = 'cod',
`username` = 'USER',
`password` = 'PWD',
`remember_me` = 'true',
`_csrf` = resp$cookies$value[1]
)
# Authenticate ------------------------------------------------------------
resp_post <- POST('https://profile.callofduty.com/do_login?new_SiteId=cod',
httr::add_headers(.headers=headers),
query = params,
httr::set_cookies(.cookies = cookies))
cookies = c(
'XSRF-TOKEN' = resp_post$cookies$value[1]
,'new_SiteId' = resp_post$cookies$value[2]
,'comid' = resp_post$cookies$value[3]
,'bm_sz' = resp_post$cookies$value[4]
,'_abck' = resp_post$cookies$value[5]
,'ACT_SSO_COOKIE' = resp_post$cookies$value[6]
,'ACT_SSO_COOKIE_EXPIRY' = resp_post$cookies$value[7]
,'atkn' = resp_post$cookies$value[8]
,'ACT_SSO_REMEMBER_ME' = resp_post$cookies$value[9]
,'ACT_SSO_EVENT' = resp_post$cookies$value[10]
,'pgacct' = resp_post$cookies$value[11]
,'CRM_BLOB' = resp_post$cookies$value[12]
,'tfa_enrollment_seen' = resp_post$cookies$value[13]
)
headers = c(
)
params = list(
`new_SiteId` = 'cod',
`username` = 'USER',
`password` = 'PWD',
`remember_me` = 'true',
`_csrf` = resp_post$cookies$value[1]
)
# Get data:
resp_psn <- httr::GET(url = 'https://my.callofduty.com/api/papi-client/stats/cod/v1/title/mw/platform/psn/gamer/savyultras90/profile/type/wz',
httr::add_headers(.headers=headers),
query = params,
httr::set_cookies(.cookies = cookies))
resp_psn_json <- content(resp_psn)
Let me know if you've already managed to resolve this!
I am sending an email from a Shiny app via the SendGrid API using the HTTR package (using POST function with json encoding). I need to pass an R object in between the json quotes that are used to define the text body of the email:
I have tried to convert the R object to json as follows:
client_id<- "f432jj"
email_text<- paste("Below is your unique key:", client_id, "Please copy
your key to the clipboard, then click 'Begin'")
email_text<- jsonlite::toJSON(email_text)
Here is the json code into which I need to insert the email_text object.
body = '{"from": {"email":"xxx#gmail.com"},
"personalizations": [{"to": [{"email":"zzz#gmail.com"}],
"dynamic_template_data":{
"header":"A measure is ready to be completed",
"text": email_text,
"c2a_button":"Begin",
"c2a_link":"yyy#gmail.com"}}],
"template_id":"e-98766334"}'
When trying to pass in the email_text object as above and send the email, I get:
HTTP/1.1 400 Bad Request
I think this means the syntax is wrong.
Any help much appreciated.
Normally you wouldn't build your body JSON data as a string. You could build the list that represents the data and then let jsonlite turn it into a JSON string for you. Your example might look something like this
client_id<- "f432jj"
email_text<- paste("Below is your unique key:", client_id, "Please copy
your key to the clipboard, then click 'Begin'")
body <- list(
from = list(email="xxx#gmail.com"),
personalizations = list(list(to=list(list(email="zzz#gmail.com")))),
dynamic_template_data = list(
header="A measure is ready to be completed",
text = email_text,
c2a_button = "Begin",
c2a_link = "yyy#gmail.com"
),
template_id = "e-98766334"
)
jsonlite::toJSON(body,auto_unbox=TRUE)
I would like to connect to COINAPI resources. They provide two types of authorization. https://docs.coinapi.io/#authorization
Custom authorization header named X-CoinAPI-Key
Query string parameter named apikey
When I am using the first method, it is working with basic requests. But respond with an error in more advanced.
endpoint<-"/v1/exchangerate/BTC?apikey="
But when I specify endpoint like this:
endpoint <- "/v1/trades/BITSTAMP_SPOT_BTC_USD/history?time_start=2016-01-01T00:00:00/?apikey="
I got error 401.
The second method is not working so far, I do not really understand how can I specify custom header name here.
I need to get data from here:
https://rest.coinapi.io/v1/ohlcv/BTC/USD/history?period_id=1DAY&time_start=2017-01-02T00:00:00.0000000Z&time_end=2019-01-02T00:00:00.0000000Z&limit=10000&include_empty_items=TRUE
I would appreciate any help on this issue.
1. method (working)
library(httr)
library(jsonlite)
base <- "https://rest.coinapi.io"
endpoint <- "/v1/exchangerate/BTC?apikey="
api_key <- <KEY>
call <- paste0(base, endpoint, api_key)
call
get_prices <- GET(call)
http_status(get_prices)
class(get_prices)
get_prices_text <- content(get_prices, "text", encoding = 'UTF-8')
get_prices_json <- fromJSON(get_prices_text, flatten = TRUE)
names(get_prices_json)
get_prices_json$asset_id_base
head(get_prices_json$rates)
data<-as.data.frame(get_prices_json)
2. method (not working)
key<-<KEY>
GET(
url = sprintf("https://rest.coinapi.io/v1/exchangerate/BTC"),
add_headers(`Authorization` = sprintf("X-CoinAPI-Key: ", key))
) -> res
http_status(res)
From reading the examples in the documentation, it looks like it's just looking for a simple header, not an "Authorization" header specifically. Try this
GET(
url = sprintf("https://rest.coinapi.io/v1/exchangerate/BTC"),
add_headers(`X-CoinAPI-Key` = key)
) -> res
http_status(res)
I am trying to pull a list of Coinbase accounts into R using their API. I receive an authentication error saying "invalid signature". Something is clearly wrong when I create my sha256 signature but I can't figure out what the issue is. I have not had this same issue when accessing the GDAX API using a sha256 signature.
API Key Documentation
API key is recommend if you only need to access your own account. All API >key requests must be signed and contain the following headers:
CB-ACCESS-KEY The api key as a string
CB-ACCESS-SIGN The user generated message signature (see below)
CB-ACCESS-TIMESTAMP A timestamp for your request
All request bodies should have content type application/json and be valid JSON.
The CB-ACCESS-SIGN header is generated by creating a sha256 HMAC using the secret key on the prehash string timestamp + method + requestPath + body (where + represents string concatenation). The timestamp value is the same as the CB-ACCESS-TIMESTAMP header.
My Code
library(httr)
library(RCurl)
library(digest)
coinbase_url <- "https://api.coinbase.com"
coinbase_reqPath <- "/v2/accounts/"
coinbase_fullPath <- paste(coinbase_url, coinbase_reqPath,sep = "")
coinbase_key <- "XXXXMYKEYXXX"
coinbase_secret <- "XXXXXMYSECRETKEYXXXX"
cb_timestamp <- format(as.numeric(Sys.time()), digits=10)
coinbase_message <- paste0(cb_timestamp,"GET", coinbase_reqPath)
coinbase_sig <- hmac(key = coinbase_secret, object = coinbase_message, algo = "sha256", raw = F)
coinbase_acct <- content(GET(coinbase_fullPath,
add_headers(
"CB-ACCESS-KEY" = coinbase_key,
"CB-ACCESS-SIGN" = coinbase_sig,
"CB-ACCESS-TIMESTAMP" = cb_timestamp,
"Content-Type"="application/json")))
Sorry for not updating this earlier. The answer is simple, I just needed to remove the final forward slash in "/v2/accounts/" when specifying my request path.