Nested lists multiple levels get elements by name - r

Question

I have a nested lists with multiple levels and I want to extract elements by name.
I have a datasets with many metrics. The head looks like:
Metric
Url
Domestic
https://www.ons.gov.uk/economy/grossdomesticproductgdp/timeseries/njiq/ukea/data
International
https://www.ons.gov.uk/economy/grossvalueaddedgva/timeseries/abml/pn2/data
I retrieve JSON for all Url:
metrics <- lapply(dataset$Url, function(i) fromJSON(content(GET(i), as = 'text')))
I get from the JSON the sublists with years
metrics_years <- lapply(metrics, function (i) i$years)
Now I have a nested list with three levels. I get what I want for one metric if I do:
sapply(metrics_years[["Domestic"]], '[[', 'year')
But I cannot type for all of them. How can I obtain the same for all metrics without typing them one by one:
sapply(metrics_years[["Domestic"]], '[[', 'year')
sapply(metrics_years[["International"]], '[[', 'year')
...

Answer 1

There isn't very much data in these files. I'm not sure what you're expecting. There are data frames called years and quarters in both lists. They both contain dates and values. Most of the content is NA. Each is a time-series data set starting in 1962. You don't need httr, either. fromJSON accepts a URL as a source without content or GET. m <- map(1:2, ~fromJSON(d[.x], flatten = T) where d is a vector with the two URLs. I think that's what #John Nielsen was stating. I used map, but lapply works fine, too.
You're using brackets, but you don't have to.
library(tidyverse) # for map and ggplot
library(jsonlite)
# get the JSON data and flatten it
mm <- map(1:2, ~fromJSON(d[.x], flatten = T))
# name each list, based on the URL
names(mm) <- c("Domestic", "International")
attributes(mm) # validate change
You can access the content with the environment pane, to get an idea of what you have.
You can use $ to access the content or [[]]. You can use numbers or names with [[]].
all.equal(mm[[1]][[1]], mm$Domestic$years)
# [1] TRUE
all.equal(mm[["Domestic"]][["years"]], mm$Domestic$years)
# [1] TRUE
To extract the data or to make a data frame-only object, you just need to assign it to an object name.
newDF <- mm$Domestic$years
You don't have to create a new object, though. If you wanted to plot the Domestic years data frame, you can use it as it is.
mm$Domestic$years %>%
mutate(year = as.integer(year),
value = as.numeric(value)) %>%
ggplot(aes(year, value)) + geom_path() + theme_bw()
The Domestic quarters data
mm$Domestic$quarters %>%
mutate(value = as.numeric(value)) %>%
ggplot(aes(seq_along(value), value)) + geom_path() +
scale_x_continuous(labels = mm$Domestic$quarters[seq(1, 220, 50), ]$date,
breaks = seq(1, 220, 50)) +
xlab("") + theme_bw()
You can do the same thing with the International data.
mm$International$years %>%
mutate(year = as.integer(year),
value = as.numeric(value)) %>%
ggplot(aes(year, value)) + geom_path() + theme_bw()