JSON Post R - Web scraping to a Dataframe in R - r

Question

I'm trying to capture a data pull from the twelvedata.com webpage. I want to pull historical equity prices for several stocks with a long time frame.
The instructions are located here: https://twelvedata.com/docs#complex-data
This request requires a JSON POST. I'm struggling to make this work. This is what I have thus far.
url <- "https://api.twelvedata.com/complex_data?apikey=myapikey"
requestBody <- paste0('{"symbols" : "AAPL",
"intervals" : "1day",
"start_date" : "2021-05-01",
"methods" : "symbol"}')
res <- httr::POST(url = url,
body = requestBody,
encode = "json")
test_1 <- content(res, as="text") %>% fromJSON()
test_2 <- as.data.frame(rjson::fromJSON(test_1))
Any help would be greatly appreciated. Thank you for your time.

Answer 1

Perhaps use toJSON to create requestBody. Here is an example:
requestBody = toJSON(list(
symbols=c("AAPL"),
intervals=c("1day"),
start_date=unbox("2021-05-01"),
end_date=unbox("2021-12-31"),
methods="time_series")
)
res <- httr::POST(url = url,body = requestBody,encode = "json")
The following will then return this type of structure:
test_1 <- httr::content(res, as="text") %>% rjson::fromJSON()
do.call(rbind, lapply(test_1$data[[1]]$values,data.frame)) %>% head()
Output:
datetime open high low close volume
1 2021-12-30 179.47000 180.57001 178.09000 178.20000 59773000
2 2021-12-29 179.33000 180.63000 178.14000 179.38000 62348900
3 2021-12-28 180.16000 181.33000 178.53000 179.28999 79144300
4 2021-12-27 177.09000 180.42000 177.07001 180.33000 74919600
5 2021-12-23 175.85001 176.85001 175.27000 176.28000 68227500
6 2021-12-22 173.03999 175.86000 172.14999 175.64000 92004100
Update: Multiple Symbols.
If you have multiple symbols, requestBody should be updated like this:
requestBody = toJSON(list(
symbols=c("AAPL", "MSFT"),
intervals=c("1day"),
start_date=unbox("2021-05-01"),
end_date=unbox("2021-12-31"),
methods="time_series")
)
and you can extract each into its own data frame within a list, like this
lapply(test_1$data, function(s){
stock = s$meta$symbol
do.call(rbind, lapply(s$values, data.frame)) %>% mutate(symbol = stock)
})