I would like to scan multiple files for strings in R and know which file names have that string.
Is there a way to do this with something like grep, cat, readLines in a function maybe?
If I scan the files using:
fileNames <- Sys.glob("*.csv")
then maybe something like:
for (f in fileNames) {
stuff <- read.csv(fileName, sep = ",")
grep("string")
}
names(res) <- substr(filenames, 1, 30)
Or maybe even better, a loop like this:
for( f in filenames ){
cat("string", file=f)
}
for( f in filenames) {
cat(readLines(f), sep="\n")
}
This code doesn't work, I'm just trying to think this through. Im certain there is a better way to do this. It sounds simple but I cant get it right.
I want to scan files for strings and then have the output of the filenames where the string was found. I have not found an example to do this in R.
Suggestions?
note that in your first code example you use f as a loop variable while inside the loop you use fileName instead (also R is case sensitive so fileNames and filenames are different objects).
if it's unlikely that your search string contains the CSV delimiter, you can indeed use readLines(..) together with grep(..). grep(..) then returns a list of line numbers where the string occurs. Try the following code:
fileNames <- Sys.glob("*.csv")
for (fileName in fileNames) {
if (length(grep("string", readLines(fileName))) > 0) { print(fileName)}
}
Related
eem_import_dir is supposed to "Reads Rdata and RDa files with one eemlist each. The eemlists are combined into one and returned." However, in my case, only one file is read at the time, and thus no combination is happening...
I don't expect any error in the function which is part of the staRdom package. I guess my limited R knowledge limits my understanding of the function and what could be wrong.
All files are the same class (eemlist) and in the same format. Tried changing the folder, filenames, etc. Can someone please help me understand the requirements of the function? Why is only one file read at the time and not all combined?
function (dir)
{
eem_files <- dir(dir, pattern = ".RData$|.RDa$", ignore.case = TRUE) %>%
paste0(dir, "/", .)
for (file in eem_files) {
file <- load(file)
if (get(file) %>% class() == "eemlist") {
if (exists("eem_list"))
eem_list <- eem_bind(eem_list, get(file))
else eem_list <- get(file)
}
else {
warning(paste0(file, " is no object of class eemlist!"))
}
NULL
}
eem_list
}
I have multiple similarly named but different folders, each containing similarly named but different csv files.
For example, I have three folders named "output", each containing "image.csv" and "cells.csv".
How do I loop through each "output" folder, then read each csv files in the folder and apply function onto these files?
Here's what I tried :
Firstly, I list the folders named "output":
dirs<-list.dirs()
dirs<-dirs[grepl("output",dirs)]
Then I want to set up a function to join both csv files, something like below (codes are incomplete though, please help to correct this):
object_extraction<-function(x){ image<-read.csv(image.csv, header=T, sep=",")
cells<-read.csv(cells.csv, header=T, sep=",")
object<-dplyr::inner_join(cells,image,by="ImageNumber")
return(object)}
Finally I want to loop the function above through the "output" folders
object<-list()
for(i in 1:length(dirs)){
object[[i]]<-object_extraction(dirs[i])
Thank you
Make the path to read csv dynamic in your function
object_extraction<-function(x){
image<-read.csv(paste0(x, '/image.csv'), header=T, sep=",")
#header = T and sep = ',' is default in read.csv so this should
#work without specifying them as well.
cells<-read.csv(paste0(x, '/cells.csv'))
object<-dplyr::inner_join(cells,image,by="ImageNumber")
return(object)
}
and then apply the function to each folder.
dirs <- list.dirs(recursive=FALSE)
dirs <- grep('output', dirs, value = TRUE)
result <- lapply(dirs, object_extraction)
Two errors I can spot in your code:
You need to use the directory name form the dirs variable, eg:
object_extraction<-function(x){
image<-read.csv(file.path(x, "image.csv"), header=T, sep=",")
cells<-read.csv(file.path(x, "cells.csv"), header=T, sep=",")
object<-dplyr::inner_join(cells,image,by="ImageNumber")
return(object)
}
And the file names should be strings, "image.csv" and "cells.csv"
HTH
I want to, programmatically, source all .R files contained within a given array retrieved with the Sys.glob() function.
This is the code I wrote:
# fetch the different ETL parts
parts <- Sys.glob("scratch/*.R")
if (length(parts) > 0) {
for (part in parts) {
# source the ETL part
source(part)
# rest of code goes here
# ...
}
} else {
stop("no ETL parts found (no data to process)")
}
The problem I have is I cannot do this or, at least, I get the following error:
simpleError in source(part): scratch/foo.bar.com-https.R:4:151: unexpected string constant
I've tried different combinations for the source() function like the following:
source(sprintf("./%s", part))
source(toString(part))
source(file = part)
source(file = sprintf("./%s", part))
source(file = toString(part))
No luck. As I'm globbing the contents of a directory I need to tell R to source those files. As it's a custom-tailored ETL (extract, transform and load) script, I can manually write:
source("scratch/foo.bar.com-https.R")
source("scratch/bar.bar.com-https.R")
source("scratch/baz.bar.com-https.R")
But that's dirty and right now there are 3 different extraction patterns. They could be 8, 80 or even 2000 different patterns so writing it by hand is not an option.
How can I do this?
Try getting the list of files with dir and then using lapply:
For example, if your files are of the form t1.R, t2.R, etc., and are inside the path "StackOverflow" do:
d = dir(pattern = "^t\\d.R$", path = "StackOverflow/", recursive = T, full.names = T)
m = lapply(d, source)
The option recursive = T will search all subdirectories, and full.names = T will add the path to the filenames.
If you still want to use Sys.glob(), this works too:
d = Sys.glob(paths = "StackOverflow/t*.R")
m = lapply(d, source)
I have original folder with 2000 files (patha), now I want to copy only the file which match my requirement (list in grdc_no) to new path (pathb). Here is my performance:
grdc_no <- grdc$grdc_no
# list of file name satisfied with my requirement
all_files <- list.files("patha", full.names = TRUE)
for (f in all_files) {
for (i in 1:length(grdc_no)) {
if (f == grdc_no[i]) {
file.copy(f, "pathb")
} else {}
}
}
However, it does not work. Any advice for me in this case? Many thanks
You can easily do this without a loop (and especially a nested one) using lapply:
lapply(all_files[basename(all_files) %in% grdc_no],function(x) file.copy(x,"pathb"))
This will index files from all_files with a matching filename in the vector grdc_no and apply file.copy to it.
I need to create a function in R that reads all the files in a folder (let's assume that all files are tables in tab delimited format) and create objects with same names in global environment. I did something similar to this (see code below); I was able to write a function that reads all the files in the folder, makes some changes in the first column of each file and writes it back in to the folder. But the I couldn't find how to assign the read files in to an object that will stay in the global environment.
changeCol1 <- function () {
filesInfolder <- list.files()
for (i in 1:length(filesInfolder)){
wrkngFile <- read.table(filesInfolder[i])
wrkngFile[,1] <- gsub(0,1,wrkngFile[,1])
write.table(wrkngFile, file = filesInfolder[i], quote = F, sep = "\t")
}
}
You are much better off assigning them all to elements of a named list (and it's pretty easy to do, too):
changeCol1 <- function () {
filesInfolder <- list.files()
lapply(filesInfolder, function(fname) {
wrkngFile <- read.table(fname)
wrkngFile[,1] <- gsub(0, 1, wrkngFile[,1])
write.table(wrkngFile, file=fname, quote=FALSE, sep="\t")
wrkngFile
}) -> data
names(data) <- filesInfolder
data
}
a_list_full_of_data <- changeCol1()
Also, F will come back to haunt you some day (it's not protected where FALSE and TRUE are).
add this to your loop after making the changes:
assign(filesInfolder[i], wrkngFile, envir=globalenv())
If you want to put them into a list, one way would be, outside your loop, declare a list:
mylist = list()
Then, within your loop, do like so:
mylist[[filesInfolder[i] = wrkngFile]]
And then you can access each object by looking at:
mylist[[filename]]
from the global env.