I tried some options from stackoverflow(e.g.1) but this also doens't work so maybe there is a mistake in my code:
fileConn<-file("outputR.txt")
for (i in 1:length(lines)){
line = lines[i]
fields = strsplit(line, "\t")[[1]]
id = fields[1]
goIDs = fields[2:length(fields)]
list = as.list(GOCCANCESTOR[goIDs])
text = paste(toString(id), ":", toString(goIDs))
cat(text, file=fileConn, append=TRUE, sep = "\n")
}
close(fileConn)
when I run this code it keeps overwriting the data in the outputR.txt file.
Any suggestions to fix this problem?
the problem is that you are using a Fileconnection in combination with cat then the append won't work. There are several option you could use, the most easy one is to this:
first "create" the file, if you want to add a header for example:
header = "some header"
## if you don't want to use a header then leave the header blank
header =""
cat(text, file="outputR.txt", append=FALSE, sep = "\n")
notice the append = FALSE this is necessary if you want to clear the file if it already exist otherwise you have to use append = TRUE
the you can write text to it using:
text = text = paste(toString(id), ":", toString(goIDs))
cat(text file="outputR.txt", append=TRUE, sep = "\n")
You have two options here:
1.
Open the file in write mode:
lines <- c("aaaaa", "bbbb")
fileConn<-file("test.txt", "w")
for (i in 1:length(lines)){
line = lines[i]
cat(line, file=fileConn, append=TRUE, sep = "\n")
}
close(fileConn)
2
Use the write function with the append argument:
lines <- c("aaaaa", "bbbb")
for (i in 1:length(lines)){
line = lines[i]
write(line,file="test2",append=TRUE)
}
As the help page for cat states:
append: logical. Only used if the argument file is the name of file (and not a connection or "|cmd"). If TRUE output will be appended to file; otherwise, it will overwrite the contents of file.
thus, if you use a connection in the file argument the value of the append argument is ignored.
simply specify the file argument as name of file:
cat(text, file="outputR.txt", append=TRUE, sep = "\n")
alternatively you can open the file connection with the correct mode specified
w+ - Open for reading and writing, truncating file initially.
fileConn <- file("outputR.txt", open = "w+")
for (i in 1:length(lines)){
text <- paste("my text in line", i)
cat(text, file = fileConn, sep = "\n")
}
close(fileConn)
Related
I am running the following loop in R:
for (i in 1:4) {
do some operation on these files
output.file <- 'results.csv'
write.table(file = output.file, x = MYFILE, append=TRUE, col.names=FALSE, row.names = FALSE, sep = "\t")
}
Since all output files have the same header, I want to save only the first one at the beginning and delete all the others. Alternatively, to add a header consequently. With col.names=FALSE I have no header at all. How to do this in R?
data <- read.table(
file = "Data/data_2021_03.txt",
header = TRUE,
sep = "\t",
stringsAsFactors = TRUE
)
I have large TXT files that I try to read with R. Problem is that in the header a hash tag # is use. R sees this as a comment en stops reading the line after the '#'. But I get the error that I have more columns than headers. For one file I replace the '#' for a '', in a txt editor. That works for one file. But I don't like to change my txt files. So how can I read a txt file with a '#' and replace it a '' in R?
Just set the comment character als empty string (see last line):
data <- read.table(
file = "Data/data_2021_03.txt",
header = TRUE,
sep = "\t",
stringsAsFactors = TRUE,
comment.char=""
)
I have 100 text file in a folder. I can use this function below to read all the files and store it into myfile.
file_list <- list.files("C:/Users/User/Desktop/code/Test/", full=T)
file_con <- lapply(file_list, function(x){
return(read.table(x, head=F, quote = "\"", skip = 6, sep = ","))
})
myfile <- do.call(rbind, file_con)
My question is how I can read the first file in the Test folder before I read the second file. All the text file name also are different and I cannot change it to for example number from 1 to 100. I was thinking of maybe can add a integer no infront of all my text file, then use a for loop to match the file and call but is this possible?
I need to read the first file then do some calculation and then export the result into result.txt before read the second file.but now I'm doing it manually and I have almost 800 file, so it will be a big trouble for me to sit and wait it to compute. The code below is the one that I current in use.
myfile = read.table("C:/Users/User/Desktop/code/Test/20081209014205.txt", header = FALSE, quote = "\"", skip = 0, sep = ",")
The following setup will read one file at the time, perform an analysis,
and save it back with a slightly modified name.
save_file_list <- structure(
.Data = gsub(
pattern = "\\.txt$",
replacement = "-e.txt",
x = file_list),
.Names = file_list)
your_function <- function(.file_content) {
## The analysis you want to do on the content of each file.
}
for (.file in file_list) {
.file_content <- read.table(
file = .file,
head = FALSE,
quote = "\"",
skip = 6,
sep = ",")
.result <- your_function(.file_content)
write.table(
x = .result,
file = save_file_list[.file])
}
Now I can read a file and do calculation using
for(e in 1:100){
myfile = read.table(file_list[e], header = FALSE, quote = "\"", skip = 0, sep = ",");
while(condition){
Calculation
}
myresult <- file.path("C:/Users/User/Desktop/code/Result/", paste0("-",e, ".txt"));
write.table(x, file = myresult, row.names=FALSE, col.names=FALSE ,sep = ",");
Now my problem is how I can make my output file to have the same name of the original file but add a -e value at the back?
I am working on combining csv files into one large csv file that will not be able to fit into my machine's RAM. Is there anyway to go about doing that in R? I realize that I could load each individual csv file into R and append the file to an existing database table but for quirky reasons I'm looking to end up with a large csv file.
Try to read each csv file one by one and write out with write.table and option append = T.
Something like this:
read one csv file;
write.table(..., append = T) to the final csv file;
remove the table with rm();
gc().
Repeate until all files are written out.
You can use the option append = TRUE
first <- data.frame(x = c(1,2), y = c(10,20))
second <- data.frame(c(3,4), c(30,40))
write.table(first, "file.csv", sep = ",", row.names = FALSE)
write.table(second, "file.csv", append = TRUE, sep = ",", row.names = FALSE, col.names = FALSE)
First create 3 test files and then create a variable Files containing their names. We used Sys.glob to do get the vector of file names but you may need to modify this statement. Then define outFile as the name of the output file. For each component of Files read in the file with that name and write it out. If it is the first file then write it all out and if it is a subsequent file write it all except for the header being sure to use append = TRUE. Note that L is overwritten each time a file is read in so that only one file takes up space at a time.
# create test files using built in data frame BOD
write.csv(BOD, "BOD1.csv", row.names = FALSE)
write.csv(BOD, "BOD2.csv", row.names = FALSE)
write.csv(BOD, "BOD3.csv", row.names = FALSE)
Files <- Sys.glob("BOD*.csv") # modify as appropriate
outFile <- "out.csv"
for(f in Files) {
L <- readLines(f)
if (f == Files[1]) cat(L, file = outFile, sep = "\n")
else cat(L[-1], file = outFile, sep = "\n", append = TRUE)
}
# check that the output file was written properly
file.show(outFile)
The loop could alternately be replaced with this:
for(f in Files) {
d <- read.csv(f)
first <- f == Files[1]
write.table(d, outFile, sep = ",", row.names = FALSE, col.names = first, append = !first)
}
In R I have a code that runs a loop over a list of search words that are downloaded from a website. Each search word is then saved as a csv file:
...some code...
x <- try(read.table(text=res, sep=",", col.names=c("Week", "TrendsCount"), skip=31, nrows=515))
for(i in 1:iterations){
...some code...
filename <- paste(wordlist[i], "csv", sep = ".")
write.table(x, file = filename, sep = ";", col.names = NA, qmethod = "double")
}
Sometimes the list will contain a search word that returns an error when code is executed, because the word does not exist on the website. This stops the loop. How can I make the loop skip the write.table part and just continue with the next word in the list?
just put write.table in
try(..., silent=T)