Cannot GeoCode with Tigris - r

I'm trying to generate census tracts geoids for a batch of addresses. When I use the "append_geoid" function in the tigris package, r returns "Error in call_geolocator(as.character(address$street[i]), as.character(address$city[i]), : Bad Request (HTTP 400)".
I used the example data given in the r documentation and it produced the same result. Code below. Any help on how to solve the issue is appreciated!
airports <- dplyr::data_frame(street = "700 Catalina Dr", city = "Daytona Beach", state = "FL")
append_geoid(airports, 'tr) # Populate Census Tract GEOID

EDIT: A fixed version of the package is on github:
remotes::install_github("walkerke/tigris")
Then try again
EDIT 2:
The version on github still seems to give errors, though different ones this time. The HTTP call succeeds, but the response doesn't contain what his function expects it to. I'd contact him or her.
My Initial Post:
I got the same message as you do.
I did: debug( call_geolocator )
And ran it again, this time stepping through it. After a few code lines it creates the url: https://geocoding.geo.census.gov/geocoder/geographies/address?street=700%20Catalina%20Dr&city=Daytona%20Beach&state=FL&benchmark=Public_AR_Census2010&vintage=Census2010_Census2010&layers=14&format=json
This url then fails. Opening this url in a browser also gives an error, saying invalid benchmark.
At this point it's about time to call the author and make him aware that his package is not working any more.
For reference, this is what the debug session looked like in my terminal, until I inspected the full url created and hit Q to stop debuging:
> debug( call_geolocator )
> append_geoid(airports, 'tract') # Populate Census Tract GEOID
debugging in: call_geolocator(as.character(address$street[i]), as.character(address$city[i]),
as.character(address$state[i]))
debug: {
call_start <- "https://geocoding.geo.census.gov/geocoder/geographies/address?"
if (is.na(zip)) {
url <- paste0("street=", utils::URLencode(street), "&city=",
utils::URLencode(city), "&state=", state)
}
if (!is.na(zip)) {
if (class(zip) == "character" & nchar(zip) == 5 & !grepl("\\D",
zip)) {
url <- paste0("street=", utils::URLencode(street),
"&city=", utils::URLencode(city), "&state=",
state, "&zip=", zip)
}
else {
message("'zip' (", paste0(zip), ") was not a 5-character-long string composed of :digits:. Using only street, city, state.")
url <- paste0("street=", utils::URLencode(street),
"&city=", utils::URLencode(city), "&state=",
state)
}
}
call_end <- "&benchmark=Public_AR_Census2010&vintage=Census2010_Census2010&layers=14&format=json"
url_full <- paste0(call_start, url, call_end)
r <- httr::GET(url_full)
httr::stop_for_status(r)
response <- httr::content(r)
if (length(response$result$addressMatches) == 0) {
message(paste0("Address (", street, " ", city, " ", state,
") returned no address matches. An NA was returned."))
return(NA_character_)
}
else {
if (length(response$result$addressMatches) > 1) {
message(paste0("Address (", street, " ", city, " ",
state, ") returned more than one address match. The first match was returned."))
}
return(response$result$addressMatches[[1]]$geographies$`Census Blocks`[[1]]$GEOID)
}
}
Browse[2]>
debug: call_start <- "https://geocoding.geo.census.gov/geocoder/geographies/address?"
Browse[2]>
debug: if (is.na(zip)) {
url <- paste0("street=", utils::URLencode(street), "&city=",
utils::URLencode(city), "&state=", state)
}
Browse[2]>
debug: url <- paste0("street=", utils::URLencode(street), "&city=",
utils::URLencode(city), "&state=", state)
Browse[2]>
debug: if (!is.na(zip)) {
if (class(zip) == "character" & nchar(zip) == 5 & !grepl("\\D",
zip)) {
url <- paste0("street=", utils::URLencode(street), "&city=",
utils::URLencode(city), "&state=", state, "&zip=",
zip)
}
else {
message("'zip' (", paste0(zip), ") was not a 5-character-long string composed of :digits:. Using only street, city, state.")
url <- paste0("street=", utils::URLencode(street), "&city=",
utils::URLencode(city), "&state=", state)
}
}
Browse[2]>
debug: call_end <- "&benchmark=Public_AR_Census2010&vintage=Census2010_Census2010&layers=14&format=json"
Browse[2]>
debug: url_full <- paste0(call_start, url, call_end)
Browse[2]>
debug: r <- httr::GET(url_full)
Browse[2]> url_full
[1] "https://geocoding.geo.census.gov/geocoder/geographies/address?street=700%20Catalina%20Dr&city=Daytona%20Beach&state=FL&benchmark=Public_AR_Census2010&vintage=Census2010_Census2010&layers=14&format=json"
Browse[2]> Q
Going to the human interface of this: https://geocoding.geo.census.gov/geocoder/locations/address?form
It does indeed look like the benchmark in that url above is no longer an available in the dropdown select box. Changing it to Public_AR_Census2020 instead gives another error, Invalid vintage in request. Changing 2010 to 2020 in that string results in a successfull HTTP request: https://geocoding.geo.census.gov/geocoder/geographies/address?street=700%20Catalina%20Dr&city=Daytona%20Beach&state=FL&benchmark=Public_AR_Census2020&vintage=Census2010_Census2010&layers=14&format=json .
This doesn't really help you much at this point, but at least you can contact the author with an indication that the problem can be solved and you could give him some info to start working with.
If you're savy, you could clone his package source and fix it yourself, offer the fix to him, but nevertheless use your own fixed package until he gets around.

Related

Conditional Handling in R

I've been trying to create an error message when the ouput entered is wrong, for example, in this code instead of entering 4 digits number, it is entered a character.
I keep receiving an error. Any tips?
get_age <- function() {
yob <- readline("Please enter your year of birth: ")
age <- 2022 - as.numeric(yob)
return(age)
}
if (get_age != as.numeric(yob)) {
withCallingHandlers(
warning = function(cnd){
readline("This is not a number. Please, try again.")
},
print("please, enter a numerical value"),
return(get_age())
)
}

R retreive information from Eikon cloud

I am trying to get data from Eikon Elektron cloud platform.
I ran following codes from https://github.com/Refinitiv/websocket-api/blob/master/Applications/Examples/R/market_price_authentication.R:
library(curl)
> content = paste("grant_type=", "password","&username=", user, "&password=", password, sep="")
> h <- new_handle(copypostfields = content)
> h
<curl handle> (empty)
> handle_setheaders(h,
+ "Content-Type" = "application/x-www-form-urlencoded"
+ )
> handle_setopt(h, ssl_verifypeer = FALSE, ssl_verifyhost = FALSE)
> auth_url = paste("https://", auth_hostname, sep="")# ":", auth_port, "/getToken", sep="")
> auth_url
[1] "https://api.refinitiv.com/auth/oauth2/v1/token"
> req <- curl_fetch_memory(auth_url, **handle = h**)
> req
$url
[1] "https://api.refinitiv.com/auth/oauth2/v1/token"
$status_code
[1] 400
$type
[1] "application/json"
**> h
<curl handle> (https://api.refinitiv.com/auth/oauth2/v1/token)**
> res_headers = parse_headers(req$headers)
> auth_json_string = rawToChar(req$content)
> auth_json = fromJSON(auth_json_string)
> cat(toJSON(auth_json, pretty=TRUE, auto_unbox=TRUE))
{
"error": "invalid_request"
}
As you can see, I got invalid request error. I think the problem lies in curl_fetch_memory and that the handle=h is using same input as auth_url, however it should use something similiar to the input of content. What can I change in my code to make it work?
I found solution how to access the URL. In github was wrongly written that app_id in R file should equal to the code from the App Key generator in Reuters. However, app_id and client_id are different things and you should add client_id=value from App Key generator (not app_id).+ also do not forget to include trapi and etc.. to your content.

Back testing for (HK) Stock Market with R

I complete my first back testing scripts with help of great people in Stackoverflow. However, when I try to run this by using the data of my local stock market (Hong Kong) it got an error. I cannot find out where the problem is. Please give me a hand to take a look my coding. thanks.
library(quantmod)
library(lubridate)
library(xlsx)
stock0<-getSymbols("^HSI",src="yahoo",from="1988-01-01",auto.assign=F)
stock0 <- to.weekly(stock0)
stock1<-na.locf(stock0)
stock1$SMA1<-SMA(Cl(stock1),n=1)
stock1$SMA30<-SMA(Cl(stock1),n=30)
stock1$SMACheck<-ifelse(stock1$SMA1>stock1$SMA30,1,0)
stock1$SMA_CrossOverUp<-ifelse(diff(stock1$SMACheck)==1,1,0)
stock1$SMA_CrossOverDown<-ifelse(diff(stock1$SMACheck)==-1,-1,0)
stock1<-stock1[index(stock1)>="1998-01-01",]
stock1_df<-data.frame(index(stock1),coredata(stock1))
colnames(stock1_df)<-c("Date","Open","High","Low","Close","Volume","Adj","SMA1","SMA30","EMACheck","EMACheck_up","EMACheck_down")
#To calculate the number of crossoverup transactions during the duration from 2016-01-01
sum(stock1_df$SMACheck_up==1 & index(stock1)>="2010-01-01",na.rm=T)
stock1_df$Date[stock1_df$SMACheck_up==1 & index(stock1)>="2010-01-01"]
sum(stock1_df$SMACheck_down==-1 & index(stock1)>="2010-01-01",na.rm=T)
stock1_df$Date[stock1_df$SMACheck_down==-1 & index(stock1)>="2010-01-01"]
stock1_df
#To generate the transcation according to the strategy
transaction_dates<-function(stock2,Buy,Sell)
{
Date_buy<-c()
Date_sell<-c()
hold<-F
stock2[["Hold"]]<-hold
for(i in 1:nrow(stock2)) {
if(hold == T) {
stock2[["Hold"]][i]<-T
if(stock2[[Sell]][i] == -1) {
#stock2[["Hold"]][i]<-T
hold<-F
}
} else {
if(stock2[[Buy]][i] == 1) {
hold<-T
stock2[["Hold"]][i]<-T
}
}
}
stock2[["Enter"]]<-c(0,ifelse(diff(stock2[["Hold"]])==1,1,0))
stock2[["Exit"]]<-c(ifelse(diff(stock2[["Hold"]])==-1,-1,0),0)
Buy_date <- stock2[["Date"]][stock2[["Enter"]] == 1]
Sell_date <- stock2[["Date"]][stock2[["Exit"]] == -1]
if (length(Sell_date)<length(Buy_date)){
#Sell_date[length(Sell_date)+1]<-tail(stock2[["Date"]],n=2)[1]
Buy_date<-Buy_date[1:length(Buy_date)-1]
}
return(list(DatesBuy=Buy_date,DatesSell=Sell_date))
}
#transaction dates generate:
stock1_df <- na.locf(stock1_df)
transactionDates<-transaction_dates(stock1_df,"SMACheck_up","SMACheck_down")
transactionDates
num_transaction1<-length(transactionDates[[1]])
Open_price<-function(df,x) {
df[which(df[["Date"]]==x)+1,][["Open"]]
}
transactions_date<-function(df,x) {
df[which(df[["Date"]]==x)+1,][["Date"]]
}
transactions_generate<-function(df,num_transaction)
{
price_buy<-sapply(1:num_transaction,function(x) {Open_price(df,transactionDates[[1]][x])})
price_sell<-sapply(1:num_transaction,function(x) {Open_price(df,transactionDates[[2]][x])})
Dates_buy<-as.Date(sapply(1:num_transaction,function(x) {transactions_date(df,transactionDates[[1]][x])}))
Dates_sell<-as.Date(sapply(1:num_transaction,function(x) {transactions_date(df,transactionDates[[2]][x])}))
transactions_df<-data.frame(DatesBuy=Dates_buy,DatesSell=Dates_sell,pricesBuy=price_buy,pricesSell=price_sell)
#transactions_df$return<-100*(transactions_df$pricesSell-transactions_df$pricesBuy)/transactions_df$pricesBuy
transactions_df$Stop_loss<-NA
return(transactions_df)
}
transaction_summary<-transactions_generate(stock1_df,num_transaction1)
transaction_summary$Return<-100*(transaction_summary$pricesSell-transaction_summary$pricesBuy)/transaction_summary$pricesBuy
transaction_summary
I complete my first back testing scripts with help of great people in Stackoverflow. However, when I try to run this by using the data of my local stock market (Hong Kong) it got an error. I cannot find out where the problem is. Please give me a hand to take a look my coding. thanks.

Error running R Instagram example from R-Bloggers?

I'm getting an error running the R Instagram query example here: https://www.r-bloggers.com/analyze-instagram-with-r/
I'm guessing it has something to do with my Instagram client (status is "Sandbox Mode"), but not sure what to do. Here's the R code and output:
Code:
require(RCurl)
require(httr)
full_url <- oauth_callback()
full_url <- gsub("(.*localhost:[0-9]{1,5}/).*", x=full_url, replacement="\1")
print(full_url)
app_name <- "teamusainrio"
client_id <- "a36424058cdf424c8e8b2d5cc2af1b15"
client_secret <- "398863caad6a4171ad10eb201870065b"
scope = "basic"
instagram <- oauth_endpoint(
authorize = "https://api.instagram.com/oauth/authorize",
access = "https://api.instagram.com/oauth/access_token")
myapp <- oauth_app(app_name, client_id, client_secret)
ig_oauth <- oauth2.0_token(instagram, myapp,scope="basic", type = "application/x-www-form-urlencoded",cache=FALSE)
Output:
Waiting for authentication in browser...
Press Esc/Ctrl + C to abort
Authentication complete.
Input:
tmp <- strsplit(toString(names(ig_oauth$credentials)), '"')
token <- tmp[[1]][4]
username <- "therock"
user_info <- fromJSON(getURL(paste('https://api.instagram.com/v1/users/search?q=',username,'&access_token=',token,sep="")),unexpected.escape = "keep")
received_profile <- user_info$data[[1]]
Output/error:
Error in user_info$data[[1]] : subscript out of bounds
If I run the query from the above code directly into my browser,
https://api.instagram.com/v1/users/search?q=therock&access_token=511932783.a364240.562161d569354bf78b043c98cf938235
I receive the following:
{"meta": {"code": 200}, "data": []}

loop web page R

I want to apply a loop to scrape data from multiple webpages in R. I'm running the next code:
city <- c("Spokane+Valley", "Spokane+-+West" , "Stanwood", "Steilacoom", "Stevenson", "Sudden+Valley", "Sultan", "Sumas", "Summit", "Summitview", "Sumner", "Sunnyside", "Sunnyslope", "Suquamish", "Tacoma+-+Central", "Tacoma+-+East", "Tacoma+-+NE", "Tacoma+-+NW", "Tacoma+-+SE", "Tacoma+-+South", "Tacoma+-+SW", "Tacoma+-+West", "Tanglewilde" , "Tenino", "Terrace+Heights", "Thrashers+Corner", "Tokeland", "Toledo" , "Toppenish", "Town+and+Country", "Tracyton" , "Trentwood", "Tukwila", "Tulalip+Bay" , "Tulalip+Indian+Reservation", "Tumwater", "Twisp", "Union+Gap" , "University+Place", "Vancouver", "Vancouver+Mall", "Veradale", "Walla+Walla", "Walla+Walla+East", "Waller", "Walnut+Grove", "Wapato", "Warden", "Washougal", "Wenatchee", "West+Clarkston-Highland", "West+Lake+Sammamish", "West+Longview", "West+Pasco", "West+Richland", "West+Side+Highway", "West+Valley", "Westport", "White+Center-Shorewood", "White+Salmon", "White+Swan", "Winlock", "Winslow", "Winthrop", "Woodinville", "Woodland", "Woodmont+Beach", "Yakima", "Yelm", "Zillah")
for(i in city){
url <- ("http://www.washingtongasprices.com/GasPriceSearch.aspx?typ=adv&fuel=D&srch=0&area=",i,"&site=Washington&station=All%20Stations&tme_limit=36")
}
But I having this message:
Error: unexpected ',' in:
"for(i in city){
url <- ("http://www.washingtongasprices.com/GasPriceSearch.aspx?typ=adv&fuel=D&srch=0&area=","
How can I solve it?
You need to paste your URL together
url <- paste0("http://www.washingtongasprices.com/GasPriceSearch.aspx?typ=adv&fuel=D&srch=0&area=",i,"&site=Washington&station=All%20Stations&tme_limit=36")

Resources