How to import data table of a Google Chart in R? - r

Question

I try to figure out how to read data table of Google Chart in R.
For example the source code of this page contains historical Peercoin daily prices. I would like to copy in a R matrix the content of the data table that begin at line 497 with :
var data = google.visualization.arrayToDataTable([
['Period', right_title_name],
['2014/10/01 18:00', 0.01189974],
['2014/10/02 18:00', 0.01194000],
['2014/10/03 18:00', 0.01171897],
['2014/10/04 18:00', 0.01199999],
['2014/10/05 18:00', 0.01200000],
['2014/10/06 18:00', 0.01188685],
['2014/10/07 18:00', 0.01161999],
// data here
]);
I've installed several packages like RCurl, XML and data.table and follow examples from related questions (i.e. using fread, readHTMLTable and getURL) but I'm facing different issues to read the correct data from the source code. Too much noise I can't filter out. For example with RCurl :
library(RCurl)
address <- "http://alt19.com/19R/chart_showing_btc.php?shw=1&label=LTC_BTC&source=cryptsy&period=1day"
data <- getURL(address)
data has all the data but I'm not able to select date and prices with strsplit(data, "some code here").
May somebody suggest me a idea to achieve this ?
Thank you,
Florent

Answer 1

Probably there's a better way but what I usually do after getting the page source with getURL, as you posted, is to use some string manipulations.
My try:
pageSource <- getURL(address)
index1<-str_locate(pageSource,"'Period', right_title_name],")[[2]]
sourceCut1<-substr(pageSource,index1+1,nchar(pageSource))
index2<-str_locate(sourceCut1,"]);")[[1]]
sourceCut2<-substr(sourceCut1,1,index2-1)
#sourceCut2 is the part of page source with the data
data<-str_trim(strsplit(sourceCut2,"\n")[[1]]) #split data rows
dates<-gsub("^.*'([0-9/: ]+).*$", "\\1", data) #extract dates
dates<-as.POSIXct(dates,format="%Y/%m/%d %H:%M")
values<-as.numeric(gsub("^.*,([0-9 .]+).*$", "\\1", data)) #extract numeric values
mydata<-data.frame(dates=dates,values=values)
Note that it will continue working only if the structure of the data (date format, blank spaces, square brackets) remains unchanged, otherwise you will probably need to modify some of the regex.

Answer 2

This answer is highly specific to your situation (and that URL) but it may be enough of a base for others with similar challenges. You can use the V8 package to parse & interpret javascript, so I grab the page, extract the javascript for the table, do some cleanup of it so it can be interpreted pretty easily then post-process the conversion. It's not pretty and others might be able to optimize it, but it will get you what you need:
library(V8)
library(httr)
library(stringr)
library(dplyr)
library(magrittr)
pg <- GET("http://alt19.com/19R/chart_showing_btc.php?shw=1&label=LTC_BTC&source=cryptsy&period=1day")
content(pg, as="text") %>%
str_extract("(google\\.visualization\\.arrayToDataTable.*\\]\\);)") %>%
str_replace("google\\.visualization\\.arrayToDataTable\\(", "[") %>%
str_replace("\\)", "]") %>%
str_replace("right_title_name", "'right_title_name'") -> tbl
ct <- new_context()
ct$eval(tbl) %>%
str_split(",") %>%
extract2(1) %>%
matrix(ncol=2, byrow=TRUE) %>%
data.frame(stringsAsFactors=FALSE) %>%
tail(-1) %>%
select(timestamp=1, value=2) %>%
mutate(timestamp=as.Date(as.POSIXct(timestamp)),
value=as.numeric(value)) -> dat
glimpse(dat)
## Observations: 227
## Variables:
## $ timestamp (date) 2014-10-01, 2014-10-02, 2014-10-03, 2014-1...
## $ value (dbl) 0.01189974, 0.01194000, 0.01171897, 0.01199...
library(ggplot2)
ggplot(dat, aes(timestamp, value)) + geom_line(size=0.5) + theme_bw()