R opening connection to web API with token and reading jason files - r

Question

I am trying to connect to Web API with token and read jason files from it using jsonlite
library(httr)
library(jsonlite)
Token="xmYeeiLGrHJNaYBWgrMfLbpZOwNgpOAh"
source="https://www.ncdc.noaa.gov/cdo-web/api/v2/%s.json"
GET(source)
POST(source,token = "xmYeeiLGrHJNaYBWgrMfLbpZOwNgpOAh")
jsonFiles=fromJSON(source)
but I keep getting this error:
Error in open.connection(con, "rb") : HTTP error 400.
What could I be doing wrong?

Answer 1

Have a look here. I adjusted the source since you had an invalid endpoint:
library(httr)
library(jsonlite)
Token <- "xmYeeiLGrHJNaYBWgrMfLbpZOwNgpOAh"
# Fetch all available datasets - https://www.ncdc.noaa.gov/cdo-web/webservices/v2#datasets
source <- "https://www.ncdc.noaa.gov/cdo-web/api/v2/datasets"
response <- GET(source, add_headers(token = Token))
#POST(source, token = Token) # Not needed
# Not actually fetching all datasets, but rather a list of all datasets
raw <- content(response, as="text")
results <- fromJSON(raw)
# Convert to dataframe/tibble
library(tidyverse)
df <- results$results %>% as_tibble()
head(df)
#> # A tibble: 6 x 6
#> uid mindate maxdate name datacoverage id
#> <chr> <chr> <chr> <chr> <dbl> <chr>
#> 1 gov.noaa.ncdc… 1763-01-… 2018-08-… Daily Summaries 1 GHCND
#> 2 gov.noaa.ncdc… 1763-01-… 2018-07-… Global Summary … 1 GSOM
#> 3 gov.noaa.ncdc… 1763-01-… 2018-01-… Global Summary … 1 GSOY
#> 4 gov.noaa.ncdc… 1991-06-… 2018-08-… Weather Radar (… 0.95 NEXRAD2
#> 5 gov.noaa.ncdc… 1994-05-… 2018-08-… Weather Radar (… 0.95 NEXRAD3
#> 6 gov.noaa.ncdc… 2010-01-… 2010-01-… Normals Annual/… 1 NORMAL…
Created on 2018-08-18 by the reprex package (v0.2.0).
Related to your original code:
source="https://www.ncdc.noaa.gov/cdo-web/api/v2/%s.json"
If this was indeed a valid endpoint (which it isn't), I think what you would want is something like this:
filename <- "dataset_name"
source <- sprintf("https://www.ncdc.noaa.gov/cdo-web/api/v2/%s.json", filename)