I have this code. I have my google API set up already, registered as well in R, Distance Matrix API has been initiated as well in the Google Cloud console.
Here is the dataframe I have, random 25 postal codes FROM and TO postal codes.
Dataset_test = data.frame(
FROM_POSTAL = c("V8A 0E5","T4G 6M4","V1N 8X3",
"C1B 5G1","R5H 2L4","H9S 8L4","L8E 4Y0","H2Y 7N6",
"K1B 7C0","G4A 5B0","E4P 3T2","E4V 5P4","H3J 1R5",
"G0B 4J7","E7A 6E7","E5B 2Y9","S4H 1T8","A2V 4G5",
"V8L 2A9","T9E 1M5","A5A 5M2","E4T 5B4","S2V 6C4",
"S9H 5P8","B1Y 0V0"),
TO_POSTAL = c("G0J 0B8","N0H 9N4","J9B 4Y4",
"L3Z 2Y7","E8K 4R4","B4P 7X9","S4H 2M0","A1Y 0B8",
"A1W 1E9","P9N 7X1","E4R 4B0","N0P 0M8","E1W 9Y7",
"T9W 8E2","G6X 4S9","A0E 0V4","J5X 7N8","N4N 8A1",
"V9K 0B9","L4G 3H7","E1W 0T2","G5R 9G3","L7C 9S2",
"E8P 2X6","E2A 2M1")
)
Here is the simple script I have to try to calculate the distance between the two postal codes by driving using Google's Distance Matrix API.
Driving_Distance = mapdist(from = Dataset_test[["FROM_POSTAL"]], to = Dataset_test[["TO_POSTAL"]], mode = c("driving")) %>% distinct()
When I run this, it throws an error in the Driving_Distance - says
Error: Argument 1 is a list, must contain atomic vectors
Your Canadian postal codes are hereby working with the mapdist() function.
The number of addresses used here were shortened for the sake of brevity.
A tibble was used instead of a dataframe so that the variables were character data types rather than factor data types. The actual Google API key that was used has been replaced with some text.
This was a good mapping question. The working code and output below:
library(ggmap)
library(plyr)
library(googleway)
library(tidyverse)
df = tibble(
FROM_POSTAL = c("V8A 0E5","T4G 6M4","V1N 8X3",
"C1B 5G1","R5H 2L4","H9S 8L4"),
TO_POSTAL = c("G0J 0B8","N0H 9N4","J9B 4Y4",
"L3Z 2Y7","E8K 4R4","B4P 7X9"))
dd <- apply(df, 1, function(x){
google_distance(origins = list(x["from"]),
destinations = list(x["to"]),
key="My_secret_key")
})
dd
Related
I need to get the list of all MCC with an Adwords account via Google API and R.
So far I've found some packages to get the list of all clientID within a single MCC but I've found no example to get the list of all MCC within an Adwords account.
Do someone have experience on this topic?
So far I've tried:
library(RAdwordsPlus)
library(RAdwords)
google_auth <- doAuth()
api_version <- "v201809"
customer_id <- "MCC-MAIN-CODE"
request <- RAdwordsPlus::managed.customer.request(fields = c("Name", "CustomerId"))
r <- RAdwordsPlus::get.service(request = request,
cid = customer_id,
auth = google_auth,
api.version = api_version,
user.agent = "r-adwordsplus-test",
verbose = FALSE,
raw = FALSE,
partial.failure = FALSE)
Code ended up with this error:
Warning message:
In parser(response) : x is not a valid managed.customer
My Account structure is something like:
Main MCC
Customer 1 (client_id_1)
Camp_#1
Camp_#2
Customer 2 (client_id_2)
Camp_#1
Camp_#2
Customer 3 (client_id_3)
Camp_#1
Camp_#2
As stated, my goal will be to get all the client_id in order to gathering data for every Customer in the account
Thanks.
Looks like JB already answered your question in his docs at:
https://jburkhardt.github.io/RAdwords/faq/#list-account-ids
List account IDs
How to list all AdWords account IDs which are in my MCC?
We would love to implement this feature! Unfortunately the Adwords API
reporting service does not allow to query the account information on
client center level.
However the good is, you only need to authenticate once in order to
access all accounts within your MCC. Best practice is to create a
vector containing the account IDs and loop over the vector.
Example of that would be something like:
load('.google.auth.RData')
adwords_accounts <- c(
"495-862-1111",
"613-408-2222",
"564-802-3333",
"902-758-4444",
"536-035-5555",
"708-304-6666",
"429-737-7777",
"532-474-8888")
#
account_performance <- statement(select= c('Date','AccountDescriptiveName','Cost','Clicks'),
report="ACCOUNT_PERFORMANCE_REPORT",
start="2019-01-01",
end=as.character(Sys.Date()))
#
list_of_data <- lapply(adwords_accounts, function(x) getData(clientCustomerId = x, google_auth = google_auth, statement = account_performance))
adwords_data <- do.call(rbind,list_of_data)
Here is my sample dataset (called origAddress):
lat lng
1.436316 103.8299
1.375093 103.8516
1.369347 103.8398
1.367353 103.8426
I have many more rows of latitude and longitude numbers (330) and I would like to find the address. I have used this for loop to do that:
for(i in 1:nrow(origAddress))
{
# Print("Working...")
result <- google_reverse_geocode(location = c(origAddress$lat[i],origAddress$lng[i]),
key = key,
location_type = "rooftop")
if(is.null(result) || length(dim(result)) < 2 || !nrow(result)) next
origAddress$venadd <- geocode_address(result)
}
It works for the first three or four rows but then returns the same address as the first row although the latitude and longitude numbers are definitely different. I have looked at other stackoverflow questions(here) and tried to copy their approach with similar bad results.
Please help!
It looks like the calls to google_geocode can return more than one address for each lat/longitude pair thus you could be overwriting your data in the output data frame.
Also, I am not sure your if statement is evaluating properly.
Here is my attempt on your problem:
library(googleway)
origAddress<-read.table(header = TRUE, text = "lat lng
1.436316 103.8299
1.375093 103.8516
1.369347 103.8398
1.367353 103.8426")
#add the output column
origAddress$venadd<-NA
for(i in 1:nrow(origAddress))
{
# Print("Working...")
result <- google_reverse_geocode(location = c(origAddress$lat[i],origAddress$lng[i]),
key=key,
location_type = "rooftop")
#add a slight pause so not to overload the call requests
Sys.sleep(1)
if(result$status =="OK" ){
#multiple address can be returned with in gecode request picks the first one
origAddress$venadd[i] <- result$results$formatted_address[1]
#use this to collect all addresses:
#paste(result$results$formatted_address, collapse = " ")
}
}
Since the call to google_reverse_geocode returns the address, I just pull the first address from the result saving a call to the internet (performance improvement). Also since the call returns a status, I check for an OK and if exist save the first address.
Hope this helps.
I'm using the R package rnoaa(along with it required other packages) to gather historical weather data. I wrote this nestled loop to gather all the data sets but I keep getting errors when I run it. It seems to run for a second fine
The loop:
require('triebeard')
require('bindr')
require('colorspace')
require('mime')
require('curl')
require('openssl')
require('R6')
require('urltools')
require('httpcode')
require('stringr')
require('assertthat')
require('bindrcpp')
require('glue')
require('magrittr')
require('pkgconfig')
require('rlang')
require('Rcpp')
require('BH')
require('plogr')
require('purrr')
require('stringi')
require('tidyselect')
require('digest')
require('gtable')
require('plyr')
require('reshape2')
require('lazyeval')
require('RColorBrewer')
require('dichromat')
require('munsell')
require('labeling')
require('viridisLite')
require('data.table')
require('rjson')
require('httr')
require('crul')
require('lubridate')
require('dplyr')
require('tidyr')
require('ggplot2')
require('scales')
require('XML')
require('xml2')
require('jsonlite')
require('rappdirs')
require('gridExtra')
require('tibble')
require('isdparser')
require('geonames')
require('hoardr')
require('rnoaa')
install.package('ncdf4')
install.packages("devtools")
library(devtools)
install_github("rnoaa", "ropensci")
library(rnoaa)
list <- buoys(dataset='wlevel')
lid <- data.frame(list$id)
foo <- for(range in 1990:2017){
for(bid in lid){
bid_range <- buoy(dataset = 'wlevel', buoyid = bid, year = range)
bid.year.data <- data.frame(bid.year$data)
write.csv(bid.year.data, file='cwind/bid_range.csv')
}
}
The response:
Using c1990.nc
Using
Error: length(url) == 1 is not TRUE
It saves the first data-set but it does not apply the for in the file name it just names it bid_range.csv.
This error message shows that there are no any data of a given station id in 1990. Because you were using for loop, once it gots an error, it stops.
Here I introduce the use of tidyverse to download the NOAA buoy data. A lot of the following functions are from the purrr package, which is part of the tidyverse.
# Load packages
library(tidyverse)
library(rnoaa)
Step 1: Create a "Grid" containing all combination of id and year
The expand function from tidyr can create the combination of different values.
data_list <- buoys(dataset = 'wlevel')
data_list2 <- data_list %>%
select(id) %>%
expand(id, year = 1990:2017)
Step 2: Create a "safe" version that does not break when there is no data.
Also make this function suitable for the map2 function
Because we will use map2 to loop through all the combination of id and year using the map2 function by its .x and .y argument. We modified the sequence of argument to create buoy_modify. We also use the safely function to create a safe version of buoy_modify. Now when it meets error, it will store the error message and moves to the next one rather than breaks.
# Modify the buoy function
buoy_modify <- function(buoyid, year, dataset, ...){
buoy(dataset, buoyid = buoyid, year = year, ...)
}
# Creare a safe version of buoy_modify
buoy_safe <- safely(buoy_modify)
Step 3: Apply the buoy_safe function
wlevel_data <- map2(data_list2$id, data_list2$year, buoy_safe, dataset = "wlevel")
# Assign name for the element in the list based on id and year
names(wlevel_data) <- paste(data_list2$id, data_list2$year, sep = "_")
After this step, all the data were downloaded in wlevel_data. Each element in wlevel_data has two parts. $result shows the data if the download is successful, otherwise, it shows NULL. $error shows NULL if the download is successful, otherwise, it shows the error message.
Step 4: Access the data
transpose can turn a list "inside out". So now wlevel_data2 has two elements: result and error. We can store these two and access the data.
# Turn the list "inside out"
wlevel_data2 <- transpose(wlevel_data)
# Get the error message
wlevel_error <- wlevel_data2$error
# Get he result
wlevel_result <- wlevel_data2$result
# Remove NULL element in wlevel_result
wlevel_result2 <- wlevel_result[!map_lgl(wlevel_result, is.null)]
When using SearchTwitter, I converted to dataframe and then exported to JSON. However, all the text is in one line, etc (sample below). I need to separate so that each tweet is its own.
phish <- searchTwitteR('phish', n = 5, lang = 'en')
phishdf <- do.call("rbind", lapply(phish, as.data.frame))
exportJson <-toJSON(phishdf)
write(exportJson, file = "phishdf.json")
json_phishdf <- fromJSON(file="phishdf.json")
I tried converting to a list and am wondering if maybe converting to a data frame is a mistake.
However, for a list, I tried:
newlist['text']=phish[[1]]$getText()
But this will just give me the text for the first tweet. Is there a way to iterate over the entire data set, maybe in a for loop?
{"text":["#ilazer #abbijacobson I do feel compelled to say that I phind phish awphul... sorry, Abbi!","#phish This on-sale was an embarrassment. Something needs to change.","FS: Have 2 Tix To Phish In Chula Vista #Phish #facevaluetickets #phish #facevalue GO: https://t.co/dFdrpyaotp","RT #WKUPhiDelt: Come unwind from a busy week of class and kick off the weekend with a Phish Fry! 4:30-7:30 at the Phi Delt house. Cost is $\u2026","RT #phish: Tickets for Phish's July 15 & 16 shows at The Gorge go on sale in fifteen minutes at 1PM ET: https://t.co/tEKLNjI5u7 https://t.c\u2026"],
"favorited":[false,false,false,false,false],
"favoriteCount":[0,0,0,0,0],
"replyToSN":["rAlexandria","phish","NA","NA","NA"],
"created":[1456521159,1456521114,1456521022,1456521016,1456520988],
"truncated":[false,false,false,false,false],
"replyToSID":["703326502629277696","703304948990222337","NA","NA","NA"],
"id":["703326837720662016","703326646074343424","703326261045829632","703326236722991105","703326119328686080"],
"replyToUID":["26152867","14503997","NA","NA","NA"],"statusSource":["Mobile Web (M5)","Twitter for iPhone","CashorTrade - Face Value Tickets","Twitter for iPhone","Twitter for Android"],
"screenName":["rAlexandria","adamgelvan","CashorTrade","Kyle_Smith1087","timogrennell"],
"retweetCount":[0,0,0,2,5],
"isRetweet":[false,false,false,true,true],
"retweeted":[false,false,false,false,false],
"longitude":["NA","NA","NA","NA","NA"],
"latitude":["NA","NA","NA","NA","NA"]}
I followed your code and don't have the issue you're describing. Are you using library(twitteR) and library(jsonlite)?
Here is the code, and a screenshot of it working
library(twitteR)
library(jsonlite)
phish <- searchTwitteR('phish', n = 5, lang = 'en')
phishdf <- do.call("rbind", lapply(phish, as.data.frame))
exportJson <-toJSON(phishdf)
write(exportJson, file = "./../phishdf.json")
## note the `txt` argument, as opposed to `file` used in the question
json_phishdf <- fromJSON(txt="./../phishdf.json")
My problem is as detailed below:
My input data is of the format as given in the small example below:
USERID LONGITUDE LATITUDE
1 -8.79659 55.879554
2 -6.874743 56.87896
3 -3.874743 58.87896
4 -10.874743 80.87896
I have used the follwoing code to reverse geocode the latitiude and longitude
dset <- as.data.frame(dataset[,2:3])
dset <- na.omit(dset)
library (ggmap)
location <- dset
nrow(location)
locaddr <- matrix(0,nrow(location),1)
location <- as.matrix(location)
for (i in 1:nrow(location))
{
locaddr[i,] <- revgeocode(location[i,], output = c("address"), messaging = FALSE, sensor = FALSE, override_limit = FALSE)
}
Now certain longitude-latitude return NA from Google Maps API. But when this happens the for loop is terminated for some reason. I would like to circumvent this and continue processing for the remaining data points. One idea I had was the following pseudocode:
if i = nrow(location)
continue
else
repeat revgeocode for loop here
end-for
end-if.
Kindly advise how this can be done or if there is a better way to do this.
Thank you in advance for your time and help.
No need to use a for-loop here. I recommand you to use lapply to avoid side effect, and pre-allocate problems:
locaddr <- lapply(seq(nrow(location)), function(i){
revgeocode(location[i,],
output = c("address"),
messaging = FALSE,
sensor = FALSE,
override_limit = FALSE)
})