I'm trying to load multiple .rds files that are save in the same directory. I have made a function for that and I iterate on a list of the files dir to load it but it doesn't work, see below that I write:
markerDir="..."
markerFilesList <- list.files(markerDir,pattern = ".rds", recursive = TRUE, include.dirs = TRUE)
readRDSfct <- function(markerFilesList,markerDir,i){
print(paste0("Reading the marker file called :",basename(markerFilesList[[i]])))
nameVariableTmp=basename(markerFilesList[[i]])
nameVariable=gsub(pattern = "\\.rds", '',nameVariableTmp)
print(paste0("file saved in varibale called:", nameVariable))
currentRDSfile = readRDS(paste0(markerDir,markerFilesList[[i]])) #nameVariable
return(currentRDSfile)
}
for (i in 1:length(markerFilesList)){
readRDSfct(markerFilesList, markerDir, i)
}
Does anyone has a suggestion for me to do it ?
thanks in advance!
As I understand it correctly, you want to just load all the RDS, which are saved in the same directory in the R environment?
To load and bind all .RDS in one directory i am using something like this:
List_RDS = list.files(pattern="*.RDS")
List_using = lapply(List_RDS, readRDS)
Data_bind <-do.call("rbind", List_using)
Related
I am writing a script in R and i am trying to retrieve files from different folder in for loop.
i am doing something like this but didn't work. How can i read files in for loop from different folders?
index2<-list.files(path="/path/to/folder2",pattern = "*.entire.txt",)
index1<-list.files(path="/path/to/folder1",pattern = "*.file.txt",)
for (y in 1:length(index2) )
{
for (x in 1:length(index1) )
{
ac2<-read.table(paste0(file = "/path/to/folder2",index2[y],header = FALSE))
ac1<-read.table(paste0(file = "/path/to/folder1",index1[x],header = FALSE))
}
P.S = I know about lapply function but i am here reading files performing calculations and saving in new files so i want to use with for loop. So if anyone can recommend how can i read files from different folders in for loop?
I am using the reactiveFileReader function in an R Shiny app to read in and visualize data that are being logged in real time (1 second data). The data files are being stored in a single folder but a new file is created every hour. I am using a reactive file path that should update when a new file is created and begin reading in and plotting data from the new file, however, reactiveFileReader does not seem to be reactive to changes in the filePath argument. The app just freezes at the last data point from the previous file until I refresh the whole app, at which time the new file is visualized.
Here is a snippet of the code I am using to read in the data.
dir <- "/media/sf_D_DRIVE"
file_name <- reactive({paste0(dir, "/", list.files(dir)[length(list.files(dir))])})
df_1 <- reactiveFileReader(intervalMillis = 100, session = session , filePath = file_name, readFunc = read.csv)
I've also tried putting the filePath directly as an argument.
dir <- "/media/sf_D_DRIVE"
df_1 <- reactiveFileReader(intervalMillis = 100, session = session , filePath = paste0(dir, "/", list.files(dir)[length(list.files(dir))]), readFunc = read.csv)
Does anyone have any thoughts on how to have the filePath update reactively and the app read and visualize data from the new files added to the folder without having to refresh?
Unfortunately, it is challenging to provide a reproducible example since you would need the data logging and file creation side of the example to make it work. Hopefully someone still has some ideas! Thanks!
reactiveFileReader is based on reactivePoll. You can use reactivePoll to achieve what you want:
df_1 <- reactivePoll(
1000, session,
checkFunc = function(){
# this function returns the most recent modification time in the folder
files <- list.files(dir, full.names = TRUE)
info <- file.info(files)
max(info$mtime)
},
valueFunc = function(){
# this function returns the content of the most recent file in the folder
files <- list.files(dir, full.names = TRUE)
info <- file.info(files)
i <- which.max(info$mtime)
read.csv(files[i])
}
)
I found an old thread (How do you read a password protected excel file into r?) that recommended that I use the following code to read in a password protected file:
install.packages("excel.link")
library("excel.link")
dat <- xl.read.file("TestWorkbook.xlsx", password = "pass", write.res.password="pass")
dat
However, when I try to do this my R immediately crashes. I've tried removing the write.res.password argument, and that doesn't seem to be the issue. I have a hunch that excel.link might not work with the newest version of R, so if you know of any other ways to do this I'd appreciate the advice.
EDIT: Using read.xlsx generates this error:
Error in .jcall("RJavaTools", "Ljava/lang/Object;", "newInstance", .jfindClass(class), :
org.apache.poi.poifs.filesystem.OfficeXmlFileException:
The supplied data appears to be in the Office 2007+ XML.
You are calling the part of POI that deals with OLE2 Office Documents.
You need to call a different part of POI to process this data (eg XSSF instead of HSSF)
You can remove the password of the excel file without knowing it with the following function (adapted version of code available at https://www.r-bloggers.com/2018/05/remove-password-protection-from-excel-sheets-using-r/)
remove_Password_Protection_From_Excel_File <- function(dir, file, bool_XLSXM = FALSE)
{
initial_Dir <- getwd()
setwd(dir)
# file name and path after removing protection
if(bool_XLSXM == TRUE)
{
file_unlocked <- stringr::str_replace(basename(file), ".xlsm$", "_unlocked.xlsm")
}else
{
file_unlocked <- stringr::str_replace(basename(file), ".xlsx$", "_unlocked.xlsx")
}
file_unlocked_path <- file.path(dir, file_unlocked)
# create temporary directory in project folder
# so we see what is going on
temp_dir <- "_tmp"
# remove and recreate _tmp folder in case it already exists
unlink(temp_dir, recursive = TRUE)
dir.create(temp_dir)
# unzip Excel file into temp folder
unzip(file, exdir = temp_dir)
# get full path to XML files for all worksheets
worksheet_paths <- list.files(paste0(temp_dir, "/xl/worksheets"), full.name = TRUE, pattern = ".xml")
# remove the XML node which contains the sheet protection
# We might of course use e.g. xml2 to parse the XML file, but this simple approach will suffice here
for(ws in worksheet_paths)
{
file_Content <- readLines(ws, encoding = "windows1")
# the "sheetProtection" node contains the hashed password "<sheetProtection SOME INFO />"
# we simply remove the whole node
out <- str_replace(file_Content, "<sheetProtection.*?/>", "")
writeLines(out, ws)
}
worksheet_Protection_Paths <- paste0(temp_dir, "/xl/workbook.xml")
file_Content <- readLines(worksheet_Protection_Paths , encoding = "windows1")
out <- stringr::str_replace(file_Content, "<workbookProtection.*?/>", "")
writeLines(out, worksheet_Protection_Paths)
# create a new zip, i.e. Excel file, containing the modified XML files
old_wd <- setwd(temp_dir)
files <- list.files(recursive = T, full.names = F, all.files = T, no.. = T)
# as the Excel file is a zip file, we can directly replace the .zip extension by .xlsx
zip::zip(file_unlocked_path, files = files) # utils::zip does not work for some reason
setwd(old_wd)
# clean up and remove temporary directory
unlink(temp_dir, recursive = T)
setwd(initial_Dir)
}
Once the password is removed, you can read the Excel file. This approach works for me.
I am using the command file.copy in R and it throws an error, but I can't spot the reason.
file.copy(from="Z:/Ongoing/Test", to = "C:/Users/Darius/Desktop", overwrite = TRUE, recursive = TRUE)
Warning message:
In file.copy(from = "Z:/Ongoing/Test",:
problem copying Z:/Ongoing/Test to C:/Users/Darius/Desktop/Test: No such file or directory
Can anyone see the problem? The command line doesn't work even though it only gives you a warning message.
Actually, I don't think there is any straight forward way to copy a directory. I have written a function which might help you.
This function takes input two arguments:
from: The complete path of directory to be copied
to: The location to which the directory is to be copied
Assumption: from and to are paths of only one directory.
dir.copy <- function(from, to){
## check if from and to directories are valid
if (!dir.exists(from)){
cat('from: No such Directory\n')
return (FALSE)
}
else if (!dir.exists(to)){
cat('to: No such Directory\n')
return (FALSE)
}
## extract the directory name from 'from'
split_ans <- unlist(strsplit(from,'/'))
dir_name <- split_ans[length(split_ans)]
new_to <- paste(to,dir_name,sep='/')
## create the directory in 'to'
dir.create(new_to)
## copy all files in 'to'
file_inside <- list.files(from,full.names = T)
file.copy(from = file_inside,to=new_to)
## copy all subdirectories
dir_inside <- list.dirs(path=from,recursive = F)
if (length(dir_inside) > 0){
for (dir_name in dir_inside)
dir.copy(dir_name,new_to)
}
return (TRUE)
}
The file.copy() doesn't create directories. So it'll only work if you're copying to folders that already exist.
Had similar issue:
This blog was helpful. Slightly modified the code by adding full.names=T and overwrite = T.
current.folder <- "E:/ProjectDirectory/Data/"
new.folder <- "E:/ProjectDirectory/NewData/"
list.of.files <- list.files(current.folder, full.names = T)
# copy the files to the new folder
file.copy(list.of.files, new.folder, overwrite = T)
I want to save the my tab delim files manually. I mean that I want user to choose the directory and file name when he wants to save the data. (For an example I have merged individual files into single file and want to save it.)
Usually I use write.table but in write.table we define the directory path and file name within that function but I want a function in which user can save file with any name in his desired directory.
Just use the file.choose() function,like this:
write.table(yerdata, file = file.choose(new = TRUE))
On Windows, at least, that will bring up a dialog for save-as commands.
Annoyingly the tcltk package doesn't have a function for 'Save As', it only has a file selector for choosing an existing file.
Luckily you can take the DIY approach via some tcl calls:
require(tcltk)
write.table(yerdata,file = tclvalue(tcl("tk_getSaveFile")))
The tk_choose.files function source could be used as a template to write a nicer interface to tcl("tk_getSaveFile") if needed. Does seem to be a glaring omission in package:tcltk though...
Using gWidgets:
gfile("Save yerdata", type = "save", handler = function(h, ...)
{
write.table(yerdata, file = h$file)
})
One (perhaps less than ideal) option would be to use readline to prompt the user for the full path and file name (or just the file name if you want to programmatically choose the directory) and then simply pass that value on the write.table. Here's a sketch:
FILE_PATH <- readline(prompt = "Enter a full path and file name: ")
#Some checking to make sure you got a valid file path...
write.table(yerdata, file = FILE_PATH)
Note that as per ?readline this will really only work when running R interactively.
As it is 2017 now, the tcltk2 package is an improvement of tcltk:
library(tcltk2)
filename <- tclvalue(tkgetSaveFile())
if (!nchar(filename)) {
tkmessageBox(message = "No file was selected!")
} else {
tkmessageBox(message = paste("The file selected was", filename))
}
And the use of filters, let's say one should only save as JPG/JPEG:
jpeg_filename <- tclvalue(
tkgetSaveFile(initialfile = "foo.jpg",
filetypes = "{ {JPEG Files} {.jpg .jpeg} } { {All Files} * }")
)