Why is the Service Unavailable Error using Lapply? - r

I am using the spotifyr library where I want to find audio features for multiple tracks. For example I can do this in order to find the audio features of a specific song using it's id.
analysis2 <- get_track_audio_features("2xLMifQCjDGFmkHkpNLD9h",
authorization = get_spotify_access_token())
Yesterday, I wrote this function below that takes all the tracks in a dataframe and finds the audio features for all of them and stores them in a list and it was working fine.
get_analysis <- function(track_id)
{
analysis <- get_track_audio_features(track_id,
authorization = get_spotify_access_token())
}
tracks_list <- lapply(all_tracks$track.id, get_analysis)
Now I am getting an error saying Request failed [503] and Error in get_track_audio_features(track_id, authorization = get_spotify_access_token()) : Service Unavailable (HTTP 503).
I am still able to find the audio features of a specific song so I am not sure which service is unavailable.

I suspect you are reaching a song in your data for which the response is denied from spotify. You could try adding an error-catching mechanism to see which one it is:
get_analysis <- function(track_id){
tryCatch(
expr = {
get_track_audio_features(track_id, authorization = get_spotify_access_token())
},
error = function(e){
print(track_id)
}) -> analysis
return(analysis)
}
tracks_list <- lapply(all_tracks$track.id, get_analysis)
I looked at the source code for the package and didn't see any sneaky rate-limiting issues and the Web API page shows error 503 as a generic error that needs waiting to be resolved (https://developer.spotify.com/documentation/web-api/). Thus you could also try just adding a 10 minute wait (I couldn't find how long exactly it is on Spotify's website):
get_analysis <- function(track_id){
tryCatch(
expr = {
get_track_audio_features(track_id, authorization = get_spotify_access_token()) -> output
return(output)
},
error = function(e){
print(track_id)
return(e)
}) -> output
}
wait.function <- funciton(){
Sys.sleep(600)
}
get_analysis_master <- function(all_tracks){
k <- 1
tracks_list <- list()
for(track.id in all_tracks$track.id){
get_analysis(track.id) -> output
if(!inherits(output, "error")){
tracks_list[[k]] <- output
k <- k + 1
} else {
wait.function()
}
return(tracks_list)
}
get_analysis_master(all_tracks) -> tracks_list

Related

R AzureGraph | How to get callRecords from Graph

With the premise that it's the first time I'm working with Graph API... I can't seem to get a handle on how to retrieve callRecords with the AzureGraph package. I'm using an existing app registration.
I wrote my code based on the vignette of the package (which uses examples for delegated authentication), however I'm not sure how to proceed...
This is what works:
library(AzureAuth)
library(AzureGraph)
library(tidyverse)
library(dplyr)
library(rjson)
library(jsonlite)
library(RCurl)
# Create graph login
gr <- create_graph_login()
# My info
me <- gr$get_user()
# App Authentication
AppID <- "xAppID"
TenantID <- "xTenantId"
Secret <- "xSecret"
App <- gr$get_app(AppID)
service <- App$get_service_principal()
# Graph API Information
Version <- getOption("azure_graph_api_version")
Endpoint <- "https://graph.microsoft.com"
Tok <- AzureAuth::get_azure_token(Endpoint, tenant = TenantID, app = AppID, password = Secret)
And this is where I get stuck:
firstpage <-call_graph_endpoint(Tok, "me/memberOf")
pager <- ms_graph_pager$new(Tok, firstpage)
pager$has_data()
pager$value
isFALSE(pager$has_data())
is.null(pager$value)
This gives me an error:
> firstpage <-call_graph_endpoint(Tok, "me/memberOf")
Error in process_response(res, match.arg(http_status_handler), simplify) :
Bad Request (HTTP 400). Failed to complete operation. Message:
/me request is only valid with delegated authentication flow.
> pager <- ms_graph_pager$new(Tok, firstpage)
> pager$has_data()
[1] FALSE
> pager$value
list()
> isFALSE(pager$has_data())
[1] TRUE
> is.null(pager$value)
[1] FALSE
I then tried to change the operation parameter, but I cannot understand what I'm supposed to fetch. This is one random attempt:
firstpage <-call_graph_endpoint(Tok, "$metadata#calls")
pager <- ms_graph_pager$new(Tok, firstpage)
pager$has_data()
pager$value
isFALSE(pager$has_data())
is.null(pager$value)
This doesn't give me any errors, but I don't know what to do with it:
Second way:
firstdf <- call_graph_endpoint(Tok, operation = "$metadata#calls", options = list('$top'=1), simplify=TRUE)
pager <- ms_graph_pager$new(Tok, firstdf)
df <- NULL
while (pager$has_data()) {
df <- vctrs::vec_rbin(df, pager$value)
This throws me a bad request error.
So... my first problem is that I don't know what my operation string should look like.
Thanks in advance for throwing some light on this...

R - curl (not httr) POST request w/ JSON body

Let me start by saying that I understand how to do a POST request using "httr" and "crul" packages. I am working on developing an asynchronous method to sending multiple POST request with unique JSON body requests using the basic "curl" package. I have legitimate reasons for trying this with this package, but more importantly I'm just determined to get it to work. This may not be possible, or I may even be trying to wrong functions in "curl"...but wanted to see if anyone had any ideas.
I am trying to send a post request using curl_fetch_multi() as a POST request with a JSON in the body like this...
{
"configuration": {
"Id": 4507
},
"age": 0,
"zip": 32411,
"Date": "2020-12-23"
}
I have succeeded in at least getting getting error messages back form the API indicating an invalid body input using something along the lines of starting with an object containing each body i need to submit
library(curl)
library(jsonlite)
library(magrittr)
pool <- new_pool()
# results only available through call back function
cb <- function(req){cat("done:", req$url, ": HTTP:", req$status, "\n", "content:", rawToChar(req$content), "\n")}
# Create request for each body
for(i in 1:nrow(df)){
curl_fetch_multi(
"http://api.com/values?api_key=1234",
done = cb,
pool = pool,
handle = new_handle() %>%
handle_setopt(post = TRUE) %>%
handle_setheaders("Content-Type"="application/vnd.v1+json") %>%
handle_setform(body = df$body[[i]]) ###df$body[[i]] is a JSON string
)
}
# This actually performs requests
out <- multi_run(pool = pool)
done: http://api.com/values?api_key=1234 : HTTP: 400
content: {"errors":[{"code":"Service.input.invalid","message":"Invalid input"}]}
done: http://api.com/values?api_key=1234 : HTTP: 400
content: {"errors":[{"code":"Service.input.invalid","message":"Invalid input"}]}
....
I'm 90% positive it has to do with how it's attempting to call the JSON in handle_setform() setting of the handle. This is about where I am over my head and documentation is scarce.
Also, I am pretty sure the JSON is structured properly, as I can use them in other packages with no problem.
Any assistance would be greatly appreciated.
Found the solution!!
Needed to use following settings with handle_setopts()
for(i in 1:nrow(df)){
curl_fetch_multi(
"http://api.com/values?api_key=1234",
done = cb,
pool = pool,
handle = new_handle() %>%
handle_setheaders("Content-Type"="application/v1+json") %>%
handle_setopt(customrequest = "POST") %>%
handle_setopt(postfields = df$body[[i]]) #df$body is list of JSON
)
}
out <- multi_run(pool = pool)

Asynchronous API endpoints with plumber

How do I define asynchronous API endpoints in plumber?
I didn't really find plumber-specific documentation on the topic except this example and this GitHub issue
When I try to reproduce the example, I get an error that R doesn't know how to turn a promise into JSON (at least that's what I think the problem is):
<simpleError: No method asJSON S3 class: promise>
Example
library(promises)
sleep_count <- 5
# add 5 seconds of sleep time
add_async_sleep <- function(p) {
n <- 20
for (i in 1:(sleep_count * n)) {
p <- then(p, function(value) {
Sys.sleep(1/n)
"" # return value
})
}
p
}
# use name_ as a placeholder for name when there are extra args
time <- function(name_, name = name_) {
paste0(name, ": ", Sys.time())
}
new_promise <- function() {
promise(function(resolve, reject){ resolve(NULL) })
}
#' #get /async
function() {
new_promise() %>%
add_async_sleep() %...>%
time("async")
}
Say this code lives in file plumber.R, then you should be able to start the API server and bring up Swagger with
r <- plumber::plumb(here::here("plumber.R"))
r$run()
Once I try out the endpoint /async, my R console reports
Starting server to listen on port 7361
Running the swagger UI at http://127.0.0.1:7361/__swagger__/
<simpleError: No method asJSON S3 class: promise>
and Swagger looks like this:
Disclaimer
I'm new to future and promises and only made it mid-way through the docs on https://rstudio.github.io/promises/ yet.

Updating Google Tag Manager container via API using R: invalidArgument error

I am trying to write an R script to programmatically update a Google Tag Manager container via API and I have hit a bit of a wall getting it to work, as it keeps returning an invalid argument error. The problem is that I can't quite figure out what the problem is.
The documentation for the API call is here:
https://developers.google.com/tag-manager/api/v2/reference/accounts/containers/update
Here's the code:
library(httr)
url_base <- 'https://www.googleapis.com/tagmanager/v2'
url_path <- paste('accounts',account_id,'containers',container_id,sep='/')
api_url <- paste(url_base,url_path,sep='/')
#since the instructions indicate that the request body parameters are all optional, let's just send a new name
call <- PUT(api_url,
add_headers(Authorization = paste("Bearer", gtm_token$credentials$access_token)),
encode = 'json',
body = list(name = 'new name'))
call_content <- content(call,'parsed')
This is a pretty standard API call to the GTM API, and in fact I have written a bunch of functions for other GTM API methods that work in the same way, so I am a bit perplexed as to why this one keeps failing:
$error
$error$errors
$error$errors[[1]]
$error$errors[[1]]$domain
[1] "global"
$error$errors[[1]]$reason
[1] "invalidArgument"
$error$errors[[1]]$message
[1] "Bad Request"
$error$code
[1] 400
$error$message
[1] "Bad Request"
It seems like the issue is in the message body, but it's not clear if the issue is down to the API expecting different information / more parameters, when the documentation suggests that all of the parameters are optional.
OK, so the documentation is lacking here. This works if you include a name at least. Here's a working function:
gtm_containers_update <- function(account_id,container_id,container_name,usage_context,domain_name,notes,token) {
require(httr)
token$refresh()
#create the post url
api_url <- paste('https://www.googleapis.com/tagmanager/v2','accounts',account_id,'containers',container_id,sep='/')
#create the list with required components
call_body <- list(name = container_name,
usageContext = list(usage_context),
notes = notes,
domainName = domain_name)
call <- POST(url,
add_headers(Authorization = paste("Bearer", token$credentials$access_token)),
encode = 'json',
body = call_body)
print(paste('Status code:',call$status_code))
}

open.connection failing in geocoding function

I'm currently running a geocoding function (using the google_places function in the googleway package). The function will run for a while (I have almost 3k locations), then throw the following error:
Error in open.connection(con, "rb") :
schannel: next InitializeSecurityContext failed: SEC_E_ILLEGAL_MESSAGE (0x80090326) - This error usually occurs when a fatal SSL/TLS alert is received (e.g. handshake failed). More detail may be available in the Windows System event log.
Having consulted the system event log, I found the following information:
The machine-default permission settings do not grant Local Activation permission for the COM Server application with CLSID
{9BA05972-F6A8-11CF-A442-00A0C90A8F39}
and APPID
{9BA05972-F6A8-11CF-A442-00A0C90A8F39}
I'm not really sure what to do with this information. From my limited knowledge, it appears this is some sort of security/firewall issue. How should I go about giving R the permissions needed to run this function?
I am running Windows 10 with Windows Defender as antivirus/firewall. For reference, this is the function I am using for geocoding:
metro.locater <- function(lat, lon){
library(googleway)
#putting latitude and longitude into the same vector
latlon <- c(lat, lon)
#getting places result
res <- google_places(location = latlon,
place_type = "subway_station", radius = 50000,
rankby="distance",
key = "myKey")
#condition handling
if(res$status == 'OK'){
closest <- res$results[1:3, ]
return(closest)} else {
try(return(res$status))
}
}
I was able to fix the issue by using an adverb I'd used with another geocoding function that attempts to run the function 5 times when it fails to provide results. Given that this worked, it seems likely that this was just a transient error rather than a systemic issue.
The adverb I used:
safely <- function(fn, ..., max_attempts = 5) {
function(...) {
this_env <- environment()
for(i in seq_len(max_attempts)) {
ok <- tryCatch({
assign("result", fn(...), envir = this_env)
TRUE
},
error = function(e) {
FALSE
}
)
if(ok) {
return(this_env$result)
}
}
msg <- sprintf(
"%s failed after %d tries; returning NULL.",
deparse(match.call()),
max_attempts
)
warning(msg)
NULL
}
}
Taken from Repeating values in loop until error disappears.

Resources