How to Extract keywords from a Data Frame in R - r

Question

I am new to text-mining in R. I want to remove stopwords (i.e. extract keywords) from my data frame's column and put those keywords into a new column.
I tried to make a corpus, but it didn't help me.
df$C3 is what I currently have. I would like to add column df$C4, but I can't get it to work.
df <- structure(list(C3 = structure(c(3L, 4L, 1L, 7L, 6L, 9L, 5L, 8L,
10L, 2L), .Label = c("Are doing good", "For the help", "hello everyone",
"hope you all", "I Hope", "I need help", "In life", "It would work",
"On Text-Mining", "Thanks"), class = "factor"), C4 = structure(c(2L,
4L, 1L, 6L, 3L, 7L, 5L, 9L, 8L, 3L), .Label = c("doing good",
"everyone", "help", "hope", "Hope", "life", "Text-Mining", "Thanks",
"work"), class = "factor")), .Names = c("C3", "C4"), row.names = c(NA,
-10L), class = "data.frame")
head(df)
# C3 C4
# 1 hello everyone everyone
# 2 hope you all hope
# 3 Are doing good doing good
# 4 In life life
# 5 I need help help
# 6 On Text-Mining Text-Mining

Answer 1

This solution uses packages dplyr and tidytext.
library(dplyr)
library(tidytext)
# subset of your dataset
dt = data.frame(C1 = c(108,20, 999, 52, 400),
C2 = c(1,3,7, 6, 9),
C3 = c("hello everyone","hope you all","Are doing good","in life","I need help"), stringsAsFactors = F)
# function to combine words (by pasting one next to the other)
f = function(x) { paste(x, collapse = " ") }
dt %>%
unnest_tokens(word, C3) %>% # split phrases into words
filter(!word %in% stop_words$word) %>% # keep appropriate words
group_by(C1, C2) %>% # for each combination of C1 and C2
summarise(word = f(word)) %>% # combine multiple words (if there are multiple)
ungroup() # forget the grouping
# # A tibble: 2 x 3
# C1 C2 word
# <dbl> <dbl> <chr>
# 1 20 3 hope
# 2 52 6 life
The problem here is that the "stop words" built in that package filter out some of the words you want to keep. Therefore, you have to add a manual step where you specify words you need to include. You can do something like this:
dt %>%
unnest_tokens(word, C3) %>% # split phrases into words
filter(!word %in% stop_words$word | word %in% c("everyone","doing","good")) %>% # keep appropriate words
group_by(C1, C2) %>% # for each combination of C1 and C2
summarise(word = f(word)) %>% # combine multiple words (if there are multiple)
ungroup() # forget the grouping
# # A tibble: 4 x 3
# C1 C2 word
# <dbl> <dbl> <chr>
# 1 20 3 hope
# 2 52 6 life
# 3 108 1 everyone
# 4 999 7 doing good

Answer 2

This is one of the first things I did in R, it may not be the best but something like:
library(stringi)
df2 <- do.call(rbind, lapply(stop$stop, function(x){
t <- data.frame(c1= df[,1], c2 = df[,2], words = stri_extract(df[,3], coll=x))
t<-na.omit(t)}))
Example data:
df = data.frame(c1 = c(108,20,99), c2 = c(1,3,7), c3 = c("hello everyone", "hope you all", "are doing well"))
stop = data.frame(stop = c("you", "all"))
Then after you can reshapedf2 using:
df2 = data.frame(c1 = unique(u$c1), c2 = unique(u$c2), words = paste(u$words, collapse= ','))
Then cbind df and df2

Answer 3

I would use the tm-package. It has a little dictionary with english stopwords. You can replace these stopwords with a white space using gsub():
library(tm)
prep <- tolower(paste(" ", df$C3, " "))
regex_pat <- paste(stopwords("en"), collapse = " | ")
df$C4 <- gsub(regex_pat, " ", prep)
df$C4 <- gsub(regex_pat, " ", df$C4)
# C3 C4
# 1 hello everyone hello everyone
# 2 hope you all hope
# 3 Are doing good good
# 4 In life life
# 5 I need help need help
You can easily add new words like c("hello", "othernewword", stopwords("en")).