When I run this code:
url = f"https://traffic.ls.hereapi.com//traffic/6.3/incidents.json?proximity=37.3541,-121.9552?apiKey={api_key}"
Scrape Here.com for any traffic incidents in the last 24 hours for the set proximity
res = requests.get(url)
date_requested = time.strftime('%Y-%m-%d')
time_requested = time.strftime('%H:%M')
date_requested; time_requested #print date & time of request
I get this error:
<html><body><p>{"error":"Unauthorized","error_description":"ApiKey is missing."}</p></body></html>
Can anyone tell me why? I know the API key is valid because I generated a new one and cut and pasted it into the code above so it has to be the structure of my query, no?
I would really appreciate any and all help.
Problem is in your URL part replace second ? with & as below-
url = f"https://traffic.ls.hereapi.com//traffic/6.3/incidents.json?proximity=37.3541,-121.9552&apiKey={}"
Related
I have this script to take each domain names in Dataframe and perform a "inurl:domain automation testing" Google search for each of them. I will scrape the 1st search result and add to my Dataframe.
import random
# Convert the Domain column in Dataframe into a list
working_backlink = backlink_df.iloc[23:len(backlink_df['Domain']), 1:22]
working_domain = working_backlink["Domain"]
domain_list = working_domain.values.tolist()
# Iterate through the list and perform query search for them
for x in range(23, len(domain_list)):
sleeptime = random.randint(1,10)
time.sleep(sleeptime)
for i in domain_list:
query = "inurl:{} automation testing".format(i)
delay = random.randint(10,30)
for j in search(query, tld="com", num=1,stop=1,pause=delay):
working_backlink.iat[x,5] = j
# Show Dataframe
working_backlink.head(n=40)
I tried using sleeptime and random delay time to prevent HTTP 429 error, but it still doesn't work. Could you suggest any solution to this? Thanks a lot!
I can retrieve EPPO DB info from GET requests.
I am looking for help to retrieve the info from POST requests.
Example code and other info in the linked Rmarkdown HTMP output
As suggested, I have gone trough the https://httr.r-lib.org/ site.
Interesting. I followed the links to https://httr.r-lib.org/articles/api-packages.html and then to https://cdn.zapier.com/storage/learn_ebooks/e06a35cfcf092ec6dd22670383d9fd12.pdf.
I suppose that the arguments for the POST() function should be (more or less) as follows, but yet the response is always 404
url = "https://data.eppo.int/api/rest/1.0/"
config = list(authtoken=my_authtoken)
body = list(intext = "Fraxinus anomala|Tobacco ringspot virus|invalide name|Sequoiadendron giganteum")
encode = "json"
#handle = ???
Created on 2021-04-26 by the reprex package (v0.3.0)
How do I find the missing pieces?
It is a little bit tricky:
You need to use correct url with name of the service from https://data.eppo.int/documentation/rest, e.g. to use Search preferred names from EPPOCodes list:
url = "https://data.eppo.int/api/rest/1.0/tools/codes2prefnames"
Authorization should be passed to body:
body = list(authtoken = "yourtoken", intext = "BEMITA")
So, if you want to check names for two eppocodes: XYLEFA and BEMITA the code should look like:
httr::POST(
url = "https://data.eppo.int/api/rest/1.0/tools/codes2prefnames",
body = list(authtoken = "yourtoken", intext = "BEMITA|XYLEFA")
)
Nonetheless, I would also recommend you to just use the pestr package. However, to find eppocodes it uses SQLite database under the hood instead of EPPO REST API. Since the db is not big itself (circa 40MB) this shouldn't be an issue.
I found the easy way following a suggestion in the DataCamp course:
"To find an R client for an API service search 'CRAN '"
I found the 'pestr' package that gives great access to EPPO database.
I still do not know how to use the POST() function myself. Any hint on that side is warmly welcome.
Here is a solution to loop over names to get EPPO-codes. Whit minor adjustments this also works for distribution and other information in the EPPO DB
# vector of species names
host_name <- c("Fraxinus anomala", "Tobacco ringspot virus", "invalide name",
"Sequoiadendron giganteum")
EPPO_key <- "enter personal key"
# EPPO url
path_eppo_code <- "https://data.eppo.int/api/rest/1.0/tools/names2codes"
# epty list
my_list <- list()
for (i in host_name) {
# POST request on eppo database
response <- httr::POST(path_eppo_code, body=list(authtoken=EPPO_key,
intext=i))
# get EPPO code
pest_eppo_code <- strsplit(httr::content(response)[[1]], ";")[[1]][2]
# add to list
my_list[[i]] <- pest_eppo_code
}
# list to data frame
data <- plyr::ldply(my_list)
I am learning how to use API in R and it is going well for the most part, but I am having trouble getting any data from the league of legends API.
For reference, I used this article as a start (https://www.dataquest.io/blog/r-api-tutorial/) and cop
res <- GET("http://api.open-notify.org/astros.json")
res
This worked just fine and has a 200 status, but I am not interested in that data.
What I want is data about league of legends, so I am trying to use:
base.url <- "https://na1.api.riotgames.com"
path <- "/lol/champion-mastery/v4/champion-masteries/by-summoner/"
API_Key <- read.table("riotkey.txt")
API_KEY <- API_Key$V1
Summoner_ID <- read.table("summonerID.txt")
SUMMONER_ID <- Summoner_ID$V1
path <- paste0(path,SUMMONER_ID)
LoL_API_Test <- GET(base.url, path = path,
add_headers(Authorization = API_KEY))
LoL_API_Test
This is Riot's explanation for the 403 error - Forbidden. "This error indicates that the server understood the request but refuses to authorize it. There is no distinction made between an invalid path or invalid authorization credentials (e.g., an API key)"
I am certain that my API key and summoner ID are correct.
So I assume the issue has to be with how I am requesting the data.
What am I doing wrong?
This particular API expects the API key to be passed in a header called "X-Riot-Token", not "Authorization". Change your call to
LoL_API_Test <- GET(base.url, path = path,
add_headers("X-Riot-Token" = API_KEY))
Im attempting to setup some R code to create a new work item task in Azure Devops. Im okay with a mostly empty work item to start with if thats okay to do (my example code is only trying to create a work item with a title).
I receive a 203 response but the work item doesn't appear in Devops.
Ive been following this documentation from Microsoft, I suspect that I might be formatting the body incorrectly.
https://learn.microsoft.com/en-us/rest/api/azure/devops/wit/work%20items/create?view=azure-devops-rest-5.1
Ive tried updating different fields and formatting the body differently with no success. I have attempted to create either a bug or feature work item but both return the same 203 response.
To validate that my token is working I can GET work item data by ID but the POST continues to return a 203.
require(httr)
require(jsonlite)
url <- 'https://dev.azure.com/{organization}/{project}/_apis/wit/workitems/$bug?api-version=5.1'
headers = c(
'Authorization' = sprintf('basic %s',token),
'Content-Type' = 'application/json-patch+json',
'Host' = 'dev.azure.com'
)
data <- toJSON(list('body'= list("op"= "add",
"path"= "/fields/System.AreaPath",
"value"= "Sample task")), auto_unbox = TRUE, pretty = TRUE)
res <- httr::POST(url,
httr::add_headers(.headers=headers),
httr::verbose(),
body = data)
Im expecting a 200 response (similar to the example in the link above) and a work item task in Azure DevOps Services when I navigate to the website.
Im not the best with R so please be detailed. Thank you in advanced!
The POST continues to return a 203.
The HTTP response code 203 means Non-Authoritative Information, it should caused by your token format is converted incorrectly.
If you wish to provide the personal access token through an HTTP
header, you must first convert it to a Base64 string.
Refer to this doc described, if you want to use VSTS rest api, you must convert your token to a Base64 string. But in your script, you did not have this script to achieve this convert.
So, please try with the follow script to convert the token to make the key conformant with the requirements(load the base64enc package first):
require(base64enc)
key <- token
keys <- charToRaw(paste0(key,":token"))
auth <- paste0("Basic ",base64encode(keys))
Hope this help you get 200 response code
I know this question is fairly old, but I cannot seem to find a good solution posted yet. So, I will add my solution in case others find themselves in this situation. Note, this did take some reading through other SO posts and trial-and-error.
Mengdi is correct that you do need to convert your token to a Base64 string.
Additionally, Daniel from this SO question pointed out that:
In my experience with doing this via other similar mechanisms, you have to include a leading colon on the PAT, before base64 encoding.
Mengdi came up big in another SO solution
Please try with adding [{ }] outside your request body.
From there, I just made slight modifications to your headers and data objects. Removed 'body' from your json, and made use of paste to add square brackets as well. I found that the Rcurl package made base64 encoding a breeze. Then I was able to successfully create a blank ticket (just titled) using the API! Hope this helps someone!
library(httr)
library(jsonlite)
library(RCurl)
#user and PAT for api
userid <- ''
token= 'whateveryourtokenis'
url <- 'https://dev.azure.com/{organization}/{project}/_apis/wit/workitems/$bug?api-version=5.1'
#create a combined user/pat
#user id can actually be a blank string
#RCurl's base64 seemed to work well
c_id <- RCurl::base64(txt = paste0(userid,
":",
devops_pat
),
mode = "character"
)
#headers for api call
headers <- c(
"Authorization" = paste("Basic",
c_id,
sep = " "
),
'Content-Type' = 'application/json-patch+json',
'Host' = 'dev.azure.com'
)
#body
data <- paste0("[",
toJSON(list( "op"= "add",
"path"= "/fields/System.Title",
"value"= "API test - please ignore"),
auto_unbox = TRUE,
pretty = TRUE
),
"]"
)
#make the call
res <- httr::POST(url,
httr::add_headers(.headers=headers),
httr::verbose(),
body = data
)
#check status
status <- res$status_code
#check content of response
check <- content(res)
Okay guys, I have, what I'm sure, is an entry-level problem. Still, I cannot explain it. Here's my code and its error:
> sample1 = readHTMLTable(http://www.pro-football-reference.com/boxscores/201609150buf.htm, which = 16)
Error: unexpected '/' in "sample1 = readHTMLTable(http:/"
It's having a problem with the second front-slash? Not only does every URL have two front-slashes, but I've poured through countless examples of this function, both on this site and others, and they've all formatted this code in this way. So, what am I doing wrong?
Additionally, I've tried it without the back-slashes:
> sample1 = readHTMLTable(www.pro-football-reference.com/boxscores/201609150buf.htm, which = 16)
Error: unexpected symbol in "sample1 = readHTMLTable(www.pro-football-reference.com/boxscores/201609150buf.htm"
Here, I'm not even sure which symbol it's talking about.
Please explain.
The issue is that you need to place your url in quotes (""). The following does return the table from your specified url:
sample1 = readHTMLTable("www.pro-football-reference.com/boxscores/201609150buf.htm")
As you probably know, the "which=" parameter is used to select which of the tables in that page you would like to retrieve. However my own attempts show that only 1 and 2 work. Could you tell me which table you are attempted to read into R? If this method doesn't end up working you can also attempt to read in the entirety of the webpage and parse out the table in question.
Hope this helps get things started!