Extracting Details from Google Earth KML File in R - r

Question

I am trying to take the details from a series of locations in a Google Earth kml file.
Getting the ids and coordinates works but for the name of location (which is located in the first table cell (td tag) of the Description), when I do it for ALL the locations, it returns the same value for all of them (Stratford Road - the name of the first location).
library(sf)
library(tidyverse)
library(rvest)
removeHtmlTags <- function(htmlString) {
return(gsub("<.*?>", "", htmlString))
}
getHtmlTableCells<- function(htmlString) {
# Convert html to html doc
htmldoc <- read_html(htmlString)
# get html for each cell (i.e. within <td></td>)
table_cells_with_tags <- html_nodes(htmldoc, "td")
# remove the html tags (<td></td>)
return(removeHtmlTags(table_cells_with_tags))[1]
}
download.file("https://www.dropbox.com/s/ohipb477kqrqtlz/AQMS_2019.kml?dl=1","aqms.kml")
locations <- st_read("aqms.kml", stringsAsFactors = FALSE) %>%
rename(id = Name) %>%
mutate(latitude = st_coordinates(geometry)[,1],
longitiude = st_coordinates(geometry)[,2],
name = getHtmlTableCells(Description)[1]) %>%
st_drop_geometry()
Now if I use the function on a particular location and get the first table cell (td), then it works, returning Stratford Road and Selly Oak for the first as below.
getHtmlTableCells(locations$Description[1])[1]
getHtmlTableCells(locations$Description[2])[1]
What am I doing wrong?

Answer 1

read_html is not vectorised - it does not accept a vector of different html to parse. We can apply your function over each element of the vector:
locations <- st_read("aqms.kml", stringsAsFactors = FALSE)
locations %>%
rename(id = Name) %>%
mutate(latitude = st_coordinates(geometry)[,1],
longitiude = st_coordinates(geometry)[,2],
name = sapply(Description, function(x) getHtmlTableCells(x)[1])) %>%
st_drop_geometry()
#> latitude longitiude name
#> 1 -1.871622 52.45920 Stratford Road
#> 2 -1.934559 52.44513 Selly Oak (Bristol Road)
#> 3 -1.830070 52.43771 Acocks Green
#> 4 -1.898731 52.48180 Colmore Row
#> 5 -1.896764 52.48607 St Chads Queensway
#> 6 -1.891955 52.47990 Moor Street Queensway
#> 7 -1.918173 52.48138 Birmingham Ladywood
#> 8 -1.902121 52.47675 Lower Severn Street
#> 9 -1.786413 52.56815 New Hall
#> 10 -1.874989 52.47609 Birmingham A4540 Roadside
Alternatively, since you're making use of regex anyway within your function, you could make use of stringr::str_extract to extract your text (which is already vectorised).
library(sf)
library(tidyverse)
locations <- st_read("aqms.kml", stringsAsFactors = FALSE) %>%
rename(id = Name) %>%
mutate(latitude = st_coordinates(geometry)[,1],
longitiude = st_coordinates(geometry)[,2],
name = str_extract(Description, '(?<=Location</td> <td>)[^<]+')) %>%
st_drop_geometry()
Where (?<=Location</td> <td>) is a lookbehind for the Location td tag that precedes our name, and [^<]+ matches anything up to the next tag following the name.

Answer 2

Your getHtmlTableCells function isn't vectorized. If you pass it a single html string, it works fine, but if you pass it multiple strings it will only process the first. Also, you have put a [1] after the return statement, which doesn't do anything. It needs to be inside the brackets. One you do this, it is easy to vectorize the function using sapply.
So make a tiny change in your function...
getHtmlTableCells <- function(htmlString) {
# Convert html to html doc
htmldoc <- read_html(htmlString)
# get html for each cell (i.e. within <td></td>)
table_cells_with_tags <- html_nodes(htmldoc, "td")
# remove the html tags (<td></td>)
return(removeHtmlTags(table_cells_with_tags)[1])
}
and vectorize it like this:
download.file("https://www.dropbox.com/s/ohipb477kqrqtlz/AQMS_2019.kml?dl=1","aqms.kml")
locations <- st_read("aqms.kml", stringsAsFactors = FALSE) %>%
rename(id = Name) %>%
mutate(latitude = st_coordinates(geometry)[,1],
longitiude = st_coordinates(geometry)[,2],
name = sapply(as.list(Description), getHtmlTableCells)) %>%
st_drop_geometry()
Which gives the correct result:
locations$name
#> [1] "Stratford Road" "Selly Oak (Bristol Road)"
#> [3] "Acocks Green" "Colmore Row"
#> [5] "St Chads Queensway" "Moor Street Queensway"
#> [7] "Birmingham Ladywood" "Lower Severn Street"
#> [9] "New Hall" "Birmingham A4540 Roadside"