Replace loop with apply function - r

Question

I'm using a for loop to create a document term matrix. My actual problem uses an obscure package called RMeCab to tokenize Japanese text, but here a more standard equivalent using strsplit. My current code:
Documents <- data.frame(Names= c("A","B"),Texts=c("A string of words","A different string"), stringsAsFactors = FALSE)
OUTPUT <- NULL
COMBINED <- NULL
i <- 1
for (i in 1:length(Documents$Texts)){
OUTPUT <- data.frame(unlist(strsplit(Documents$Texts, " ")))
OUTPUT$doc <- Documents$Names[i]
COMBINED <- rbind(COMBINED, OUTPUT)
}
Document_Term_Matrix <- as.data.frame.matrix(table(COMBINED))
It works, but I'd like to use a more efficient apply function. If I run
L_OUTPUT <- lapply(Documents[,2],function(x) strsplit(x, " "))
I get the separate words as elements of a list, but how do I append the document name from Documents$Names?
More specifically with a list structure:
[[1]]
これ です は ぺん
1 1 1 1
[[2]]
です は 人 彼
1 1 1 1
How do I get a data with a column like this
これ は ぺん です 彼 は 人 です
And the second column showing the documents names
One One One One Two Two Two Two
Those words corresponding to the list elements [[1]], [[2]], etc.

Answer 1

It is better to use packages such as tm for these kind of operations, but here is a solution using base R,
list1 <- strsplit(Documents$Texts, ' ')
v1 <- unique(unlist(list1))
Document_Term_Matrix <- as.data.frame(t(sapply(v1, function(i) lapply(list1, function(j)
sum(grepl(i, j))))))
names(Document_Term_Matrix)<- Documents$Names
Document_Term_Matrix
# A B
#A 1 1
#string 1 1
#of 1 0
#words 1 0
#different 0 1

Answer 2

you can use functions from tm package which are suitable for large text datasets:
library(tm)
# create corpora from your documents
corp = VCorpus(DataframeSource(Documents), readerControl = list(reader = readTabular(mapping = list(content = "Texts", id = "Names"))))
# create term document matrix
tdm = TermDocumentMatrix(corp, control = list(tokenize = function(x) unlist(strsplit(as.character(x), "[[:space:]]+"))
, stopwords = FALSE
, tolower = TRUE
, weighting = weightTf))
inspect(tdm)
# get the result as matrix
tdm.m = matrix(tdm, nrow = tdm$nrow, ncol = tdm$ncol)
rownames(tdm.m) = tdm$dimnames$Terms
colnames(tdm.m) = tdm$dimnames$Docs
I also think there is a mistake in your question (but I cannot add comments).
You're missing [i] in your for cycle, so you get the total number of terms in all the documents. It should be something like this:
OUTPUT <- data.frame(unlist(strsplit(Documents$Texts[i], " ")))