Scraping <li> elements with Rvest - r

Question

Good morning,
I'm new to scraping with R, and I'm having a hard time to scrape a list of elements from a webpage in a useful manner.
This is my script
library(rvest)
url <- read_html("https://www.pole-emploi.fr/annuaire/provins-77070")
webpage <- url %>%
html_nodes('.zone') %>%
html_text()
webpage
When I run the script all the elements appear squeezed together without any whitespace between, which is comprehensible since each item is enclosed in a single tag.
[1] "77114GouaixHerméNoyen-sur-SeineVilliers-sur-Seine"
[2] "77118BalloyBazoches-lès-BrayGravon"
I would like to have them either like this (or separated by commas)
[1] "77114 Gouaix Hermé Noyen-sur-Seine Villiers-sur-Seine"
[2] "77118 Balloy Bazoches-lès-Bray Gravon"
Or even better on a tidy format
Postal City
77114 Gouaix
77114 Hermé
77114 Noyen-sur-Seine
77114 Villiers-sur-Seine
I have tried to find other selector or Xpaths in the page without success. The most I have got is to select one single element of the list.
Any help would be greatly apprecaited.
Thanks in advance.

Answer 1

Each list element looks like this (truncated for brevity):
<li class="zone">\n<span class="code-postal">77114</span><ul>\n<li>Gouaix</li>\n<li>Hermé</li>\n ...
So, each one has a set of child nodes that look uniform. We can target the <span> and the <li> elements in the nested <ul> to get what you want:
library(rvest)
library(tidyverse)
pg <- read_html("https://www.pole-emploi.fr/annuaire/provins-77070")
html_nodes(pg, ".zone") %>%
map_df(~{
data_frame(
postal = html_node(.x, "span") %>% html_text(trim=TRUE),
city = html_nodes(.x, "ul > li") %>% html_text(trim=TRUE)
)
})
## # A tibble: 95 x 2
## postal city
## <chr> <chr>
## 1 77114 Gouaix
## 2 77114 Hermé
## 3 77114 Noyen-sur-Seine
## 4 77114 Villiers-sur-Seine
## 5 77118 Balloy
## 6 77118 Bazoches-lès-Bray
## 7 77118 Gravon
## 8 77126 Châtenay-sur-Seine
## 9 77126 Égligny
## 10 77134 Les Ormes-sur-Voulzie
## # ... with 85 more rows
the tidyverse method with explicit anonymous function (vs .x via formula function):
html_nodes(pg, ".zone") %>%
map_df(function(x) {
data_frame(
postal = html_node(x, "span") %>% html_text(trim=TRUE),
city = html_nodes(x, "ul > li") %>% html_text(trim=TRUE)
)
})
and, a pure base R version:
elements <- html_nodes(pg, ".zone")
lapply(elements, function(x) {
data.frame(
postal = html_text(html_node(x, "span"), trim=TRUE),
city = html_text(html_nodes(x, "ul > li"), trim=TRUE),
stringsAsFactors = FALSE
)
}) -> tmp
Reduce(rbind.data.frame, tmp)
# or
do.call(rbind.data.frame, tmp)