Read specific csv files in multiple directories with R - r

Question

I need to read specific csv files stored in multiple directories with R. Each directory contains these files (and others) which however are listed under different names but with distinct characters that make them recognisable.
Let's suppose the csv files I want to read contains the following distinct character: '1' (file 1) and '2' (file 2).
Here's the code I tried so far:
# This is the main directory where all your the sub-dir with files are stored
common_path = "~/my/main/directory"
# Extract the names of the sub-dir
primary_dirs = list.files(common_path)
# Create empty list of lists
data_lst = rep(list(list()), length(primary_dirs)) # one list per each directory
# These are the 2 files (by code) that I need to read
names_csv = c('1', '2')
#### Nested for loop reading the csv files into the list of lists
for (i in 1:length(primary_dirs)) {
for (j in 1:length(names_csv)) {
data_lst[[i]][j] = read.csv(paste('~/my/main/directory/', primary_dirs[i],
'/name_file', names_csv[j], '.csv', sep = ''))
}
}
### End of nested loop
The issue here is that the code works only if the names of the files are identical within each directory. But this is not the case. Each directory has different file names but the file names contain the distinct characters '1' and '2'.
E.g. in this case my files in all directories are called 'name_file1.csv' and 'name_file2.csv'. But in my real case the names of files are something like: dir 1 -> 'name_bla_1.csv', 'name_bla_2.csv'; dir 2 -> 'name_gya_1.csv' 'name_gya_2.csv'; etc...
How can I read these 2 files from all my directories with files having different names?
Thanks

Answer 1

You're making things much too complicated. list.files can search recursively (within directories), can return the full file path so you don't have to worry about pasteing together where the file path, and can match regex patterns.
files_to_read = list.files(
path = common_path, # directory to search within
pattern = ".*(1|2).*csv$", # regex pattern, some explanation below
recursive = TRUE, # search subdirectories
full.names = TRUE # return the full path
)
data_lst = lapply(files_to_read, read.csv) # read all the matching files
To learn more about regex, I'd recommend regex101.com. .*, (1|2) matches 1 or 2, and $ matches the end of the string, so ".*(1|2).*csv$" will match all strings that contain a 1 or 2 and end in csv.

Answer 2

If you simply want to read in any matching filenames from any subdirectories, you could try this:
regular_expression <- "name_[A-z]+_"
names_csv <- c('1', '2')
names_to_read <- paste0(regular_expression, names_csv, "\\.csv", collapse = "|")
fileList <- list.files(pattern = names_to_read, path = common_path,
recursive = TRUE, full.names = TRUE)
data_lst <- lapply(files_to_read, function(x) read.csv(x))
The output should be a list, where each entry is one of your csv files.
It wasn't clear to me if you wanted to maintain separation based on the directory each file was read from, so I haven't included that.