How to read .txt tables into R - r

Question

I have a .txt file which I think was output from STATA, but I'm not sure. It is a list of tables formatted like this:
Q1 | Freq. Percent Cum.
------------+-----------------------------------
answer | 35 21.08 21.08
text | 4 2.41 23.49
words | 35 21.08 44.5
something | 38 22.89 67.47
blah | 54 32.53 100.00
------------+-----------------------------------
Total | 166 100.00
Q2 | Freq. Percent Cum.
------------------+-----------------------------------
foo | 1 0.60 0.60
blahblah | 11 6.63 7.23
etc | 26 15.66 22.89
more text | 82 49.40 72.29
answer | 7 4.22 76.51
survey response | 39 23.49 100.00
------------------+-----------------------------------
Total | 166 100.00
Q3 | Freq. Percent Cum.
------------+-----------------------------------
option | 7 4.22 4.22
text | 24 14.46 18.67
blahb | 25 15.06 33.73
more text | 82 49.40 83.13
etc | 28 16.87 100.00
------------+-----------------------------------
Total | 166 100.00
It continues for about 200 questions and their respective survey answers. Does anyone know how I can quickly read each survey question into separate data frames in R?

Answer 1

No need for scan():
txt <- " Q1 | Freq. Percent Cum.
------------+-----------------------------------
answer | 35 21.08 21.08
text | 4 2.41 23.49
words | 35 21.08 44.5
something | 38 22.89 67.47
blah | 54 32.53 100.00
------------+-----------------------------------
Total | 166 100.00
Q2 | Freq. Percent Cum.
------------------+-----------------------------------
foo | 1 0.60 0.60
blahblah | 11 6.63 7.23
etc | 26 15.66 22.89
more text | 82 49.40 72.29
answer | 7 4.22 76.51
survey response | 39 23.49 100.00
------------------+-----------------------------------
Total | 166 100.00
Q3 | Freq. Percent Cum.
------------+-----------------------------------
option | 7 4.22 4.22
text | 24 14.46 18.67
blahb | 25 15.06 33.73
more text | 82 49.40 83.13
etc | 28 16.87 100.00
------------+-----------------------------------
Total | 166 100.00"
library(purrr)
You can just as easily read from a file vs the above text vector. The main goal here is to remove cruft from the data and get it into a form we can work with, so we get rid of the dashed lines and the Total line, and convert spaces to commas. This makes a big assumption about your data format so it needs to be consistent.
readLines(textConnection(txt)) %>%
discard(~grepl("(----|Total)", .)) %>%
gsub("[[:space:]]*\\|[[:space:]]*", ",", .) %>%
gsub("[[:space:]][[:space:]]+", ",", .) %>%
gsub("^,", "", .) -> lines
There's a blank line between the tables. This is another assumption the code makes. We find that blank line and extract the lines that are between blanks (including the start and end of the text). Then we read that into a data frame with read.csv.
starts <- c(1, which(lines=="")+1)
ends <- c(which(lines=="")-1, length(lines))
map2(starts, ends, function(start, end) {
read.csv(textConnection(lines[start:end]), stringsAsFactors=FALSE)
})
This results in a list of data frames:
## [[1]]
## Q1 Freq. Percent Cum.
## 1 answer 35 21.08 21.08
## 2 text 4 2.41 23.49
## 3 words 35 21.08 44.50
## 4 something 38 22.89 67.47
## 5 blah 54 32.53 100.00
##
## [[2]]
## Q2 Freq. Percent Cum.
## 1 foo 1 0.60 0.60
## 2 blahblah 11 6.63 7.23
## 3 etc 26 15.66 22.89
## 4 more text 82 49.40 72.29
## 5 answer 7 4.22 76.51
## 6 survey response 39 23.49 100.00
##
## [[3]]
## Q3 Freq. Percent Cum.
## 1 option 7 4.22 4.22
## 2 text 24 14.46 18.67
## 3 blahb 25 15.06 33.73
## 4 more text 82 49.40 83.13
## 5 etc 28 16.87 100.00
But, I think this might be more useful as one big data frame:
map2_df(starts, ends, function(start, end) {
df <- read.csv(textConnection(lines[start:end]), stringsAsFactors=FALSE)
colnames(df) %>%
tolower() %>%
gsub("\\.", "", .) -> cols
question <- cols[1]
cols[1] <- "text"
setNames(df, cols) %>%
mutate(question=question) %>%
mutate(n=1:nrow(.)) %>%
select(question, n, text, freq, percent, cum) %>%
mutate(percent=percent/100, cum=cum/100)
})
## question n text freq percent cum
## 1 q1 1 answer 35 0.2108 0.2108
## 2 q1 2 text 4 0.0241 0.2349
## 3 q1 3 words 35 0.2108 0.4450
## 4 q1 4 something 38 0.2289 0.6747
## 5 q1 5 blah 54 0.3253 1.0000
## 6 q2 1 foo 1 0.0060 0.0060
## 7 q2 2 blahblah 11 0.0663 0.0723
## 8 q2 3 etc 26 0.1566 0.2289
## 9 q2 4 more text 82 0.4940 0.7229
## 10 q2 5 answer 7 0.0422 0.7651
## 11 q2 6 survey response 39 0.2349 1.0000
## 12 q3 1 option 7 0.0422 0.0422
## 13 q3 2 text 24 0.1446 0.1867
## 14 q3 3 blahb 25 0.1506 0.3373
## 15 q3 4 more text 82 0.4940 0.8313
## 16 q3 5 etc 28 0.1687 1.0000