How to break Char Vectors up at defined terms? - web-scraping

Question

I am scraping data off a website using rvest() for the first time.
It gave me a character vector that I am trying to split and convert to a data frame with columns.
How do you turn this vector:
char.vector <- c("John DoeTeacherSpeaksEnglishJapaneseRateUSD 10Video Intro","JaneTutorSpeaksJapaneseFrenchRateUSD 15Video Intro")
...into this data frame with columns:
Name
Role
English
Japanese
French
Rate_USD
John Doe
Teacher
1
1
0
10
Jane
Tutor
0
1
1
15
Splitting on spaces or character position is problematic. Is there maybe a way to create a vector of all the words I want to split at and use it as the split argument?
split.vector <- c("Teacher", "Tutor", "Speaks", "English", "Japanese", "French", "Rate", "Video")
My code and url:
EN.char <- read_html("https://www.italki.com/teachers/english") %>%
html_nodes(".teacher-card") %>%
html_text()
EN.char

Answer 1

So, as there are potentially different numbers of languages for each entry, I shamelessly took the approach used by #akrun in their answer here, whereby read.dcf is used to map out all the languages present, and put NA where a language is not present for a given entry. After having read this article I saw that:
The DCF rules as implemented in R are:
A database consists of one or more records, each with one or more
named fields. Not every record must contain each field, a field may
appear only once in a record.
Regular lines start with a non-whitespace character.
Regular lines are of form tag:value, i.e.,
have a name tag and a value for the field, separated by : (only the
first : counts). The value can be empty (=whitespace only).
Lines starting with whitespace are continuation lines (to the preceding
field) if at least one character in the line is non-whitespace.
Records are separated by one or more empty (=whitespace only) lines.
In order to resolve a malformed line error, I needed to fix rule 3 and insert ":" so as to have tag:value pairs.
Following #akrun's example, I wrapped the read.dcf within an as.data.frame call. I kept the unnecessary, in this case, if statement, for safety of missing entries. I doubt languages would be missing given the nature of the site.
I then used replace_na() to switch out NA with "0" and cast all the language columns as integer.
I then generated another DataFrame, with all the other details required, which only had one entry per field in the record (row). As I was sure of matching, and ordered, rows, I then used cbind() to join the DataFrames column-wise.
N.B. Given my location, the USD values are actually showing as GBP.
R:
library(purrr)
library(dplyr)
library(stringr)
library(rvest)
page <- read_html("https://www.italki.com/teachers/english")
entries <- page %>% html_elements(".teacher-card")
language_df <- map_dfr(entries, ~ {
new <- .x %>%
html_elements("p + div > div") %>%
html_text() %>%
paste0(., ":1")
if (length(new) > 0) {
as.data.frame(read.dcf(textConnection(new)))
} else {
NULL
}
}) %>%
mutate(across(.cols = everything(), ~ as.integer(tidyr::replace_na(.x, 0))))
details_df <- map_dfr(entries, ~
data.frame(
name = .x %>% html_element("div > .overflow-hidden") %>% html_text(),
role = .x %>% html_element("span + div > .text-tiny") %>% html_text(),
rate_usd = .x %>% html_element(".flex-1:nth-child(1) > div > span") %>% html_text()
))
results <- cbind(details_df, language_df)
Results: