how to extract the first paragraphs from text in dataframe? - r

Question

Consider this dataframe
library(dplyr)
library(stringr)
mydf <- data_frame(text = c('Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. \nUt enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. \nDuis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum',
'Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. \nUt enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. \nDuis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum',
'this is a short text without paragraphs! HA!!!'))
I would like to create a column first_paragraphs that only includes the first two paragraphs from the text stored in the mytext column. As you can see, sometimes there is not even one paragraph (row 3). In that case, leaving the text as-is is OK.
I have tried the following, but without success.
#this function finds the position of the second \n in the data
myend <- function(text){
myend <- str_locate_all(text, "\n")[[2]] %>% as_tibble() %>% pull(end)
myend
}
mydf <-mydf %>% mutate(thresh = myend(text),
#here I only keep text until that threshold
first_paragraphs= str_sub(text, 1, thresh))
Error in mutate_impl(.data, dots) :
Evaluation error: subscript out of bounds.
What is wrong here?
expected output is:
data_frame(text = c('Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. \nUt enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. ',
'Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. \nUt enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. ',
'this is a short text without paragraphs! HA!!!'))
Many thanks!

Answer 1

Here is a base R solution with strsplit:
mydf$firstparagraph = paste(strsplit(mydf$text, "\n")[[1]][1:2], collapse = "\n")
Result:
> mydf$firstparagraph
[1] "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. \nUt enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. "
Edit:
With OP's updated dataset, below is a way to extract the first two paragraphs of each row of text:
mydf$firstparagraph = sapply(strsplit(mydf$text, "\n"),
function(x) sub("\nNA$", "", paste(x[1:2], collapse = "\n")))
For better readability, you can use pipes from dplyr:
library(dplyr)
mydf$text %>%
strsplit("\n") %>%
sapply(function(x){
x[1:2] %>%
paste(collapse = "\n") %>%
sub("\nNA$", "", .)
})
or in tidyverse:
library(stringr)
library(purrr)
mydf %>%
mutate(firstparagraph = map(strsplit(text, "\n"), ~{
.[1:2] %>%
paste(collapse = "\n") %>%
str_replace("\nNA$", "")
}))
Result:
> mydf$firstparagraph
[1] "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. \nUt enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. "
[2] "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. \nUt enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. "
[3] "this is a short text without paragraphs! HA!!!"
sapply is needed since column text now has multiple rows, so strsplit will output a list with each element corresponding to a row in text. sub is used to remove the extra \nNA's for rows where there are less than two paragraphs.

Answer 2

This will get you the first two paragraphs in the variable "first_paragraphs", plus the "thresh variable":
mydf <- data_frame(text = paste0(
'Lorem ipsum dolor sit amet, '
'consectetur adipiscing elit, '
'sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. '
'\nUt enim ad minim veniam, '
'quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. '
'\nDuis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. '
'Excepteur sint occaecat cupidatat non proident, '
'sunt in culpa qui officia deserunt mollit anim id est laborum'))
mydf <- mydf %>% mutate(thresh = str_locate_all(mydf$text, "\n")[[1]][2, 2],
first_paragraphs = str_sub(text, 1, thresh))