Hey I want to ask how to copy multiple files from multiple folders to a single folders using R language
Assuming there are three folders:
desktop/folder_A/task/sub_task/
desktop/folder_B/task/sub_task/
desktop/folder_C/task/sub_task/
In each of the sub_task folder, there are multiple files. I want to copy all the files in the sub_task folders and paste them in a new folder (let's name this new folder as "all_sub_task") on desktop. Can anyone show me how to do it in R using the loop or apply function? Thanks in advance.
Here is an R solution.
# Manually enter the directories for the sub tasks
my_dirs <- c("desktop/folder_A/task/sub_task/",
"desktop/folder_B/task/sub_task/",
"desktop/folder_C/task/sub_task/")
# Alternatively, if you want to programmatically find each of the sub_task dirs
my_dirs <- list.files("desktop", pattern = "sub_task", recursive = TRUE, include.dirs = TRUE)
# Grab all files from the directories using list.files in sapply
files <- sapply(my_dirs, list.files, full.names = TRUE)
# Your output directory to copy files to
new_dir <- "all_sub_task"
# Make sure the directory exists
dir.create(new_dir, recursive = TRUE)
# Copy the files
for(file in files) {
# See ?file.copy for more options
file.copy(file, new_dir)
}
Edited to programmatically list sub_task directories.
This code should work. This function takes one directory -for example desktop/folder_A/task/sub_task/- and copies everything there to a second one. Of course you can use a loop or apply to use more than one directory at once, as the second value is fixed sapply(froms, copyEverything, to)
copyEverything <- function(from, to){
# We search all the files and directories
files <- list.files(from, r = T)
dirs <- list.dirs(from, r = T, f = F)
# We create the required directories
dir.create(to)
sapply(paste(to, dirs, sep = '/'), dir.create)
# And then we copy the files
file.copy(paste(from, files, sep = '/'), paste(to, files, sep = '/'))
}
Related
I am trying to read all files in a specific sub-folder of the wd. I have been able to add a for loop successfully, but the loop only looks at files within the wd. I thought the command line:
directory <- 'folder.I.want.to.look.in'
would enable this but the script still only looks in the wd. However, the above command does help create a list of the correct files. I have included the script below that I have written but not sure what I need to modify to aim it at a specific sub-folder.
directory <- 'folder.I.want.to.look.in'
files <- list.files(path = directory)
out_file <- read_excel("file.to.be.used.in.output", col_names = TRUE)
for (filename in files){
show(filename)
filepath <- paste0(filename)
## Import data
data <- read_excel(filepath, skip = 8, col_names = TRUE)
data <- data[, -c(6:8)]
further script
}
The further script is irrelevant to this question and works fine. I just can't get the loop to look over each file in files from directory. Many thanks in advance
Set your base directory, and then use it to create a vector of all the files with list.files, e.g.:
base_dir <- 'path/to/my/working/directory'
all_files <- paste0(base_dir, list.files(base_dir, recursive = TRUE))
Then just loop over all_files. By default, list.files has recursive = FALSE, i.e., it will only get the files and directory names of the directory you specify, rather than going into each subfolder. Setting recursive = TRUE will return the full filepath excluding your base directory, which is why we concatenate it with base_dir.
I am trying to copy a large number of files from one folder to another. We need to restructure the folders, so there is a translation from the old folder path to a new one. The old folder structure is also nested.
Currently the code I have is not throwing any errors, but is returning false on executing the file.copy for all files.
ETA: When I copy a single file, it works.
allFilePaths <- list.files('./oldTopLevelFolder', recursive = TRUE)
testIds <- c(1:4)
otherTestIds <- c(5:8)
allNewFolders <- paste('newTopLevelFolder', testIds, 'aFolderName', otherTestIds, sep = '/')
lapply(allNewFolders, dir.create, recursive = TRUE)
file.copy(from=allFilePaths, to=allNewFolders,
copy.mode = TRUE)
file.copy can copy multiple files, but only to a single destination folder by the looks of it.
In order to copy a bunch of files into varying destination folders, the following will do the job, where allOldFilePaths is a column containing the old filepath where each file currently exists, and allNewFilePaths is a column containing the new folder path for each file.
# function to copy a single file
copySingleFile <- function(oldPath, newPath) {
file.copy(from=oldPath, to=newPath,
copy.mode = TRUE)
}
# copy each file to its new folder path
mapply(copySingleFile, allFilePathsWithRoot, allNewFilePaths)
I have list of zipped files with text files in a directory which contains empty folders in between. I want to unzip the files and after unzip want to delete the empty folders if exists. Need help in doing this in R.
You can use list.files(include.dirs=TRUE) to get a list of files and directories, then use file.info to check if this is a directory. Since file.info is showing 0 for size when it is a directory, you need to list all the files inside the directory and get the size to check if its empty. Then you can delete the directory if the size is 0 using unlink:
lapply(list.files(include.dirs=TRUE, full.names=TRUE), function(x) {
fi <- file.info(x)
if (fi$isdir) {
f <- list.files(x, all.files=TRUE, recursive=TRUE, full.names=TRUE)
sz <- sum(file.info(f)$size)
#as precaution, print to make sure before using unlink(x, TRUE)
if (sz==0L) print(x)
}
})
You can use the function icesTAF::remdir(path, recursive = T) to delete empty folders.
for i in 1:[your folders count]{
icesTAF::rmdir(list.files([your path containing folders], full.names = T)[i], recursive = T)
}
I have created a list of files by some conditions and I want to copy only the files from that list to a new folder and subfolders like in the origin folder.
The structure of the folders is year/month/day.
This is the code I tried:
from.dir <- "J:/Radar_data/Beit_Dagan/RAW/2018"
## I want only the files from the night
to.dir <- "J:/Radar_data/Beit_Dagan/night"
files <- list.files(path = from.dir, full.names = TRUE, recursive =
TRUE)
## night_files is a vector I created with the files I need - only during the night
for (f in night_files) file.copy(from = f, to = to.dir)
But I get all the files in one folder
part of my list look like this:
[1] "J:/Radar_data/Beit_Dagan/H5/2018/03/10/TLV180310142554.h5"
[2] "J:/Radar_data/Beit_Dagan/H5/2018/03/10/TLV180310142749.h5"
[3] "J:/Radar_data/Beit_Dagan/H5/2018/03/10/TLV180310143054.h5"
Is there a way to keep the structure of the folder and the subfolders when copying?
I want to get the same structure of year/month/day in the new "night" folder
You need to use the flag recursive = T inside the copy call, so you don't really need to loop inside the dir.
from = paste0(getwd(),"/output/","output_1")
to = paste0(getwd(),"/output/","output_1_copy")
file.copy(from, to, recursive = T)
Note that you need to create the /output_1_copy directory previously to the call. Yo can do it manually or using dir.create(...).
You just need:
file.copy(from = from.dir, to = to.dir,recursive=T)
I have a list of many directories, each of which have 5 files inside, from these files within each directory I want to select one (for example let us say the one with extension .txt) and compile a list of these .txt files....how do I create a loop that selects txt files from a list of directories in R?
You can do:
dir(path = ".", pattern = "\\.txt$", full.names = TRUE, recursive = TRUE)
Where path is the root that contains all the folders you want to look up, pattern is regular expression that matches the files you are interested in (in the example all the files with the .txt extension, full.names return the full path of the files, and recursiveto explore all the subfoders in path. This returns a vector with the full path for the files that match your query.
list.files is a vectorized function already, so you can pass the vector of directories to it, no loop needed.
my_dirs <- c("foo/bar", "foo/baz")
all_text_files <- list.files(my_dirs, pattern = "\\.txt$", full.names = TRUE)
If you want a list separating files by directory...
split(all_text_files, dirname(all_text_files))
If you have the list of directory names in dirs,
you can get the .txt files in all of them as a vector with:
files <- unlist(lapply(dirs, function(dir) list.files(path = dir, pattern = '\\.txt$')))
You can achieve the same using a loop as you asked,
but it's less elegant, and I don't recommend it:
files <- c()
for (dir in dirs) {
files <- c(files, list.files(path = dir, pattern = '\\.txt$'))
}