I am trying to do a batch geocode with the Yahoo BOSS api from R.
It is currently throwing an error based on credentials - any idea how I can get this to succeed?
myapp <- oauth_app("yahoo",
key = "my key",
secret = "my secret"
)
yahoo <- oauth_endpoint("get_request_token", "request_auth", "get_token",
base_url = "https://yboss.yahooapis.com/geo/placefinder")
token <- oauth1.0_token(myapp, yahoo)
sig <- sign_oauth1.0(myapp, token$oauth_token, token$oauth_token_secret)
GET("https://yboss.yahooapis.com/geo/placefinder",
sig)
Unfortunately Yahoo uses a weird authentication strategy that isn't compatible with a simple oauth_endpoint function. You can see the general flow I use in the rydn package that #Scott pointed out here.
You might benefit from just using that package, or feel free to leverage the working example I have there in your own stuff.
Related
I'm trying out this function from the package edgarwebR
x <- paste0("https://www.sec.gov/Archives/edgar/data/",
"933691/000119312517247698/0001193125-17-247698-index.htm")
try(filing_information(x))
But it returns me the following
No encoding supplied: defaulting to UTF-8.
Error in check_result(res) :
EDGAR request blocked from Undeclared Automated Tool.
Please visit https://www.sec.gov/developer for best practices.
See https://mwaldstein.github.io/edgarWebR/index.html#ethical-use--fair-access for your responsibilities
Consider also setting the environment variable 'EDGARWEBR_USER_AGENT
So I went to SEC and the information tells me to declare my user agent in the request headers. I'm new to this. How do I create a user agent?
I use the edgar package so I'm not sure this is helpful. But this is how I would do it using edgar package
library("edgar")
useragent = "Your Name Contact#domain.com"
info <- getFilingInfo('933691', 2021, quarter = c(1,2,3,4), form.type = 'ALL', useragent)
Anyone worked with R for accessing Azure Data Lake (file upload/download from ADLS in R). If so, any code snippet will be helpful.
For ADLSgen2, you can use the AzureStor package, which is on CRAN. Example code from the README:
library(AzureStor)
# authenticate with AAD
token <- AzureRMR::get_azure_token("https://storage.azure.com",
tenant="myaadtenant", app="app_id", password="mypassword"))
ad_endp_tok <- storage_endpoint("https://mystorage.dfs.core.windows.net", token=token)
cont <- storage_container(ad_endp_tok, "myfilesystem")
list_storage_files(cont)
create_storage_dir(cont, "newdir")
storage_download(cont, "/readme.txt", "~/readme.txt")
storage_multiupload(cont, "N:/data/*.*", "newdir")
For ADLSgen1, you can use the AzureSMR package. However, note that AzureSMR is deprecated and no longer actively maintained.
I am trying to use GGmap to create a plot of vehicle car crashes by state. The map will have dots which are sized based on the number of car crashes in the state.
In particular I am trying to recreate the usa-plot shown in the visualizing clusters section of this blog post.
However, whenever I try to create the map I get this error.
Error in aperm.default(map, c(2, 1, 3)) :
invalid first argument, must be an array
I have setup the Google API and see that it is recieving hits. I have also enabled it and have the key.
In addition I have installed GGmap from the github account using this command:
devtools::install_github("dkahle/ggmap", ref = "tidyup", force=TRUE)
since the CRAN one isn't updated.
I have restarted and quit R several times as well but the error persists.
Even if I just simply run:
get_map()
it still results in the error:
Error in aperm.default(map, c(2, 1, 3)) :
invalid first argument, must be an array
Below is my code, it is similar to the code in the blog post:
mydata$State <- as.character(mydata$State)
mydata$MV.Number = as.numeric(mydata$MV.Number)
mydata = mydata[mydata$State != "Alaska", ]
mydata = mydata[mydata$State != "Hawaii", ]
devtools::install_github("dkahle/ggmap", ref = "tidyup", force=TRUE)
library(ggmap)
ggmap::register_google(key = "...") #my key is here
for (i in 1:nrow(mydata)) {
latlon = geocode(mydata[i,1])
mydata$lon[i] = as.numeric(latlon[1])
mydata$lat[i] = as.numeric(latlon[2])
}
mv_num_collisions = data.frame(mydata$MV.Number, mydata$lon, mydata$lat)
colnames(mv_num_collisions) = c('collisions','lon','lat')
usa_center = as.numeric(geocode("United States"))
USAMap = ggmap(get_googlemap(center=usa_center, scale=2, zoom=4),
extent="normal")
USAMap +
geom_point(aes(x=lon, y=lat), data=mv_num_collisions, col="orange",
alpha=0.4, size=mv_num_collisions$collisions*circle_scale_amt) +
scale_size_continuous(range=range(mv_num_collisions$collisions))
I expect the map to output like this
But I cannot seem to get passed this error.
If anyone can help that would be great.
Please let me know if you need any more information.
Thank you.
This error is due to the google key not having the appropriate API activity enabled for that key.
Go into the google API console and enable the API "Maps Static API" and it should work for you.
EDIT: Jan 2020 - I was doing some similar work and found that a similar API was failing because billing information had to be added to the project in the Google Cloud console before it would work.
Make sure to enable billing. You don't have to restrict api, but make sure all the api's you need are enabled. if you want to search location names, you'll need geocoding api in addition to static maps. ggmap from CRAN is OK now (don't need github version).
it´s necessary confirm your credit card in Google API, with this, your API key is activated and you can use ggmap normally
I have seen a few posts about this, but I have not been able to use any of the suggested code for the way I have my program set up. I am not very good at R but I need this information for a Political Science class. Could someone please help me figure out a way to avoid receiving the error message "Warning message: In twInterfaceObj$doAPICall(cmd, params, method, ...) : Rate limit encountered & retry limit reached - returning partial results." Thank you in advance! The code I am running is below:
# install and load packages
install.packages("twitteR")
install.packages("ROAuth")
install.packages("base64enc")
install.packages("plyr")
library("twitteR")
library("ROAuth")
library("base64enc")
library("plyr")
# Establish the connection (do this once)
cred$handshake(cainfo="cacert.pem")
#setup Twitter
setup_twitter_oauth('Consumer Key - XXXXXXXXXX', Consumer Secret -
'XXXXXXXXXX', API Key - 'XXXXXXXXXX', API Secret - 'XXXXXXX')
#store the user
user <- getUser('JackPosobiec')
#return a list of followers
Posobiecfollowers <- user$getFollowers(n=NULL)
Posobiecfollowers
#save followers in a data set
followers.df = ldply(followers, function(t) t$toDataFrame())
write.csv(followers.df, file = "JackPosobiecfollowersfile.csv")
I can not throttle-up my downloads by using the token issued to my app (on data.chicago.com portal, where I had to register)
Error 1:
token <- "___my_app_token__";
fdf <- read.socrata("h___s://data.cityofchicago.org/resource/7edu-s3u7.csv?$where=station_name=\"Foster Weather Station\"", token)
2016-10-06 10:39:53.685 getResponse:
Error in httr GET: 403 h___s://data.cityofchicago.org/resource/7edu-s3u7.csv?%24where=station_name%3D%22Foster%20Weather%20Station%22&app_token=%2524%2524app_token%3D___my_app_token_______
I have NO IDEA where did the first 'token' (2524 2524) come from, do you? Can somebody tell me? Maybe the author of the package is here?
Non-error:
fdf <- read.socrata("h___s://data.cityofchicago.org/resource/7edu-s3u7.csv?$where=station_name=\"Foster Weather Station\"")
WITHOUT A TOKEN (and not throttled-up) works perfectly well!
and this 'open source' h___s://github.com/Chicago/RSocrata/blob/master/R/RSocrata.R doesn't answer the question as well.
It looks like the syntax you're using to pass your app token is wrong. I'm no R expert, but I found this example in the documentation for the RSocrata library:
df <- read.socrata("http://soda.demo.socrata.com/resource/4334-bgaj.csv",
app_token = "__my_app_token__")
Try passing your app token as a named parameter instead of an indexed parameter, and see if that helps.