R: Extracting latitude, longitude and time from JSON file - r

Question

I have a .json file (more than 100,000 lines) containing the following information:
POST /log?lat=36.804121354&lon=-1.270256482&time=2016-05-18T17:39:59.004Z
{ 'content-type': 'application/x-www-form-urlencoded',
'content-length': '29',
host: 'ip_address:port',
connection: 'Keep-Alive',
'accept-encoding': 'gzip',
'user-agent': 'okhttp/3.7.0' }
BODY: lat=36.804121354&lon=-1.270256482
POST /log?lat=36.804123256&lon=-1.270254711&time=2016-05-18T17:40:13.004Z
{ 'content-type': 'application/x-www-form-urlencoded',
'content-length': '29',
host: 'ip_address:port',
connection: 'Keep-Alive',
'accept-encoding': 'gzip',
'user-agent': 'okhttp/3.7.0' }
BODY: lat=36.804123256&lon=-1.270254711
POST /log?lat=36.804124589&lon=-1.270255641&time=2016-05-18T17:41:05.004Z
{ 'content-type': 'application/x-www-form-urlencoded',
'content-length': '29',
host: 'ip_address:port',
connection: 'Keep-Alive',
'accept-encoding': 'gzip',
'user-agent': 'okhttp/3.7.0' }
BODY: lat=36.804124589&lon=-1.270255641
.......
The above information repeats with updated latitude, longitude and time. Using R, how can I extract latitude, longitude and time from this file? and store them in a dataframe like this:
id lat lon time
1 36.804121354 -1.270256482 2016-05-18 17:39:59
2 36.804123256 -1.270254711 2016-05-18 17:40:13
3 36.804124589 -1.270255641 2016-05-18 17:41:05

Answer 1

It doesn't appear your data is strictly JSON. Since the requested data is all contained on the "Post" lines, an one solution is to filter those lines out and then parse them.
#Read lines
x<-readLines("test.txt")
#Find lines beginning with "POST"
posts<-x[grep("^POST", x)]
#Remove the prefix: "POST /log?"
posts<-sub("^POST /log\\?", "", posts)
#split remaining fields on the &
fields<-unlist(strsplit(posts, "\\&"))
#remove the prefixes ("lat=", "lon=", "time=")
fields<-sub("^.*=", "", fields)
#make a dataframe (assume the fields are always in the same order)
df<-as.data.frame(matrix(fields, ncol=3, byrow=TRUE), stringsAsFactors = FALSE)
names(df)<-c("lat", "lon", "time")
#convert the columns to the proper type.
df$lat<-as.numeric(df$lat)
df$lon<-as.numeric(df$lon)
df$time<-as.POSIXct(df$time, "%FT%T", tz="UTC")