I have the following file names in a folder:
1_myfile.txt, 2_myfile.txt, 3_myfile.txt, and 4_best_myfile.txt, 5_best_myfile.txt, 6_best_myfile.txt.
I would like to use regex in pattern = "" when listing files with list.files() in order to subset files containing "_myfile.txt" from files containing "_best_myfile.txt". I tried using:
files = list.files(path = ".", "*[^best_myfile.txt]$")
Unfortunately it does not work because it subsets only files that do not end with .txt. How can I solve this?
We can modify the pattern to "\\d+_best_myfile\\.txt"
files <- list.files("\\d+_best_myfile\\.txt")
It implies one or more numbers (\\d+) followed by a _ and the string best_myfile.txt. Also, note that some characters needs to be escaped i.e. . is a metacharacter and it implies any character. So, to get the literal dot character, we need to escape (\\) it
Related
I´m making a list of fasta files and read them from a folder. The file name should be assigned as list element name w/o the .fa file format.
I´m using list.files to asses the files in the directory "Folder"
filenames <- list.files("Folder",pattern = ".fa",full.names = T)
and than read the fasta files in.
list <- lapply(filenames, FUN=readDNAStringSet, use.names=T, format="fasta")
I found this code using setNames to define the list element name.
list<- setNames(list, substr(list.files("Folder", pattern=".fa"), 1,15 ))
But my file names have different length (makes it difficult to use the START to STOP (,1, 15)) and for further processing I would like to get rid of the .fa
The files would look like:
Gene1.fa
Gene12.fa
Gene22a.fa
Gene123abc.fa
I´m using DECIPHER but I guess this is a more base R question?
Inorder to remove the substring at the end, we could use substr as well, but make sure to index the first/last from the end instead from the beginning as it is varying
v1 <- list.files("Folder", pattern=".fa")
substring(v1, first = 1, last = nchar(v1) -3)
#[1] "Gene1" "Gene12" "Gene22a" "Gene123abc"
Or another option is sub to match the dot (. - metacharacter that matches for any character, so escape (\\) it to get the literal meaning) followed by 'fa' at the end ($) of the string and replace it with blank ("")
sub("\\.fa$", "", v1)
I wish to use R to read multiple csv files from a single folder. If I wanted to read every csv file I could use:
list.files(folder, pattern="*.csv")
See, for example, these questions:
Reading multiple csv files from a folder into a single dataframe in R
Importing multiple .csv files into R
However, I only wish to read one of four subsets of the files at a time. Below is an example grouping of four files each for three models.
JS.N_Nov6_2017_model220_N200.csv
JS.N_Nov6_2017_model221_N200.csv
JS.N_Nov6_2017_model222_N200.csv
my.IDs.alt_Nov6_2017_model220_N200.csv
my.IDs.alt_Nov6_2017_model221_N200.csv
my.IDs.alt_Nov6_2017_model222_N200.csv
parms_Nov6_2017_model220_N200.csv
parms_Nov6_2017_model221_N200.csv
parms_Nov6_2017_model222_N200.csv
supN_Nov6_2017_model220_N200.csv
supN_Nov6_2017_model221_N200.csv
supN_Nov6_2017_model222_N200.csv
If I only wish to read, for example, the parms files I try the following, which does not work:
list.files(folder, pattern="parm*.csv")
I am assuming that I may need to use regex to read a given group of the four groups present, but I do not know.
How can I read each of the four groups separately?
EDIT
I am unsure whether I would have been able to obtain the solution from answers to this question:
Listing all files matching a full-path pattern in R
I may have had to spend a fair bit of time brushing up on regex to apply those answers to my problem. The answer provided below by Mako212 is outstanding.
A quick REGEX 101 explanation:
For the case of matching the beginning and end of the string, which is all you need to do here, the following prinicples apply to match files that are .csv and start with parm:
list.files(folder, pattern="^parm.*?\\.csv")
^ asserts we're at the beginning of the string, so ^parm means match parm, but only if it's at the beginning of the string.
.*? means match anything up until the next part of the pattern matches. In this case, match until we see a period \\.
. means match any character in REGEX, so we need to escape it with \\ to match the literal . (note that in R you need the double escape \\, in other languages a single escape \ is sufficienct).
Finally csv means match csv after the .. If we were going to be really thorough, we might use \\.csv$ using the $ to indicate the end of the string. You'd need the dollar sign if you had other files with an extension like .csv2. \\.csv would match .csv2, where as \\.csv$ would not.
In your case, you could simply replace parm in the REGEX pattern with JS, my, or supN to select one of your other file types.
Finally, if you wanted to match a subset of your total file list, you could use the | logical "or" operator:
list.files(folder, pattern = "^(parm|JS|supN).*?\\.csv")
Which would return all the file names except the ones that start with my
The list.files statement shown in the question is using globs but list.files accepts regular expressions, not globs.
Sys.glob To use globs use Sys.glob like this:
olddir <- setwd(folder)
parm <- lapply(Sys.glob("parm*.csv"), read.csv)
parm is now a list of data frames read in from those files.
glob2rx Note that the glob2rx function can be used to convert globs to regular expressions:
parm <- lapply(list.files(folder, pattern = glob2rx("parm*.csv")), read.csv)
I have some files in the directory and their extensions all same. I want list all files but later on would like to ignore some of the file names that contain certain strings. I tried grepl using this answer using-r-to-list-all-files-with-a-specified-extension.
For instance in this example I would like exclude the files which contains 'B' in it. So tried,
file_names <- c('AA','BA','KK','CB')
files <- paste0(file_names,'.txt')
Filter_files <- files[-grepl('.*B.txt|.B*.txt', files)]
Filter_files
"BA.txt" "KK.txt" "CB.txt"
Interestingly only AA.txt excluded!
This will work:
file_names <- c('AA','BA','KK','CB')
files <- paste0(file_names,'.txt')
Filter_files <- files[!grepl('.*B.*\\.txt', files)]
Filter_files
## "AA.txt" "KK.txt"
These are the changes I made:
Instead of -, the grepl is preceded by !, which negates the result of the grepl (i.e., the TRUE results become FALSE and viceversa).
To capture all the Bs regardless of where they are located, I search for any character (that's what . indicates) appearing 0 or more times (indicated by the * sign). This way whether the B is at the beginning or the end of the filename it is equally captured.
Since . has the special meaning of 'any character' in regex, to have a literal dot in the expression you have to escape it, thus \\. before the txt extension.
I have a folder which contains files of the following names "5_45name.Rdata"and "15_45name.Rdata".
I would like to list only the ones starting with "5", in the example above this means that I would like to exclude "15_45name.Rdata".
Using list.files(pattern="5_*name.Rdata"), however, will list both of these files.
Is there a way to tell list.files() that I want the filename to start with a specific character?
We need to use the metacharacter (^) to specify the start of the string followed by the number 5. So, it can a more specific pattern like below
list.files(pattern ="^5[0-9]*_[0-9]*name.Rdata")
Or concise if we are not concerned about the _ and other numbers following it.
list.files(pattern = "^5.*name.Rdata")
I need to select files which start with "M" and end with ".csv". I can easily select files which start with "M" : list.files(pattern="^M"), or files which end with "csv": list.files(pattern = ".csv"). But how to select files which satisfy both conditions at the same time?
You could try glob2rx
lf <- list.files("path_to_directory/", pattern=glob2rx("M*.csv"))
which translates to:
glob2rx("M*.csv")
[1] "^M.*\\.csv$"
The pattern argument takes a regular expression:
list.files(pattern='^M.*csv')
To be more specific, your second expression:
list.files(pattern='.csv')
Is matching all files with the string csv preceded by any character. To be explicit and only match files with a .csv extension:
list.files(pattern='\\.csv$')