I am new to programming.Currently doing R programming in coursera and got this error while doing assignment named "pollutantmean".I searched in forums and stackoverflow,but couldnt able to fix it.Appreciate your help.Thanks.
I got this error :
Error in file(file, "rt") : cannot open the connection In addition: Warning message:
In file(file, "rt") : cannot open file 'NA': No such file or directory
Note: I have a folder "specdata" which is the working directory.This "specdata" has all the 332 csv files.I want to calculate mean for one of the pollutant column named "pollutant" in those files and "directory" is the location of those files."id" is a integer vector mentioning the monitor number.so,here is my code:
pollutantmean <- function(directory, pollutant, id = 1:332) {
files_full <- list.files(directory, full.names = TRUE)
dat <- data.frame()
for (i in id) {
dat <- rbind(dat, read.csv(files_full[i]))
}
mean(dat[, pollutant], na.rm = TRUE)
}
pollutantmean("specdata","sulfate",id = 1:10)
As per the error, it seems that list.files is returning empty vector for files names.
list.files returns only file nam inside the directory by default. So in that case, you should set the working directory first, so that read.csv will read the files inside the working directory.
setwd("<directory name>")
Now your function should work fine.
Related
I have 332 csv files in "specdata" directory and when I am trying to get mean of pollutants of selected files, getting this error: error in file(file, "rt") : cannot open the connection
setwd("C:\\Users\\1911120\\Desktop\\GIT")
pollutantmean <- function(directory, pollutant,id=1:332){
for (file in id){
path <- paste(getwd(),"/directory","/",sprintf("%03d",file),".csv",sep="")
data <- read.csv(path)
pollutant_data <- data[pollutant]
mean_data <- c(mean_data,pollutant_data[!is.na(pollutant_data)])
}
mean(mean_data)
}
pollutantmean("specdata", "sulfate", 1:10)
Fresh lettuce here so don't laugh at my questions:
Say I have a folder containing 40 individual .txt files and I would like to convert them into .csv format.
To have an end product : a new folder with 40 individual .csv files.
I have seen similar question posted and their code, however the code did run but the .csv files is nothing like the orginal .txt file: all the data are scrambled.
Since I want to keep the header, and I want to read all the data/rows in the .txt file. I made some cosmetic changes to the code, still didnt run and returned a warning "Error in file(file, "rt") : cannot open the connection
In addition: Warning message:
In file(file, "rt") :
cannot open file 'C:/Users/mli/Desktop/All TNFa controls in the training patients ctrl_S1018263__3S_TNFaCHx_333nM+0-1ugml_none.txt': Invalid argument"
My code as below:
directory <- "C:/Users/mli/Desktop/All TNFa controls in the training patients"
ndirectory <- "C:/Users/mli/Desktop/All TNFa controls in the training patients/CSV"
file_name <- list.files(directory, pattern = ".txt")
files.to.read <- paste(directory, file_name, sep="\t")
files.to.write <- paste(ndirectory, paste0(sub(".txt","", file_name),".csv"), sep=",")
for (i in 1:length(files.to.read)) {
temp <- (read.csv(files.to.read[i], sep="\t", header = T))
write.csv(temp, file = files.to.write[i])
}
When you paste your path with the name of your file at line 4 and 5, use /, to obtain a new path in a character string. The sep value is what the function will put when it will paste together multiple strings.
> paste('hello','world',sep=" ")
[1] "hello world"
> paste('hello','world',sep="_")
[1] "hello_world"
This is different of the sep value you need in read.csv that define the character between each column of you csv file.
This question already has answers here:
Reading multiple files into multiple data frames
(2 answers)
Closed 5 years ago.
d <- NULL
datafiles <- list.files(path = "C:___")
for (i in datafiles){
print(i)
j <- read.csv(i, header = T)
j$file <- i
d <- rbind(d, j)
}
When I ran just the print line, all of the csv names in the folder were displayed, but everything beyond the j <- line, an error pops up. When I ran the entire code, the error I got says:
Error in file(file, "rt") : cannot open the connection In addition:
Warning message:
In file(file, "rt") :
cannot open file 'xxx.csv': No such file or directory
Any suggestions would be appreciated, thanks!
You could try:
library(tidyverse)
file_list <- list.files()
df <- map_dfr(file_list, read_csv)
Or:
file_list <- list.files()
df <- do.call("rbind", lapply(file_list, read_csv))
Make sure to set your working directory correctly with setwd().
I am having a folder where I have a lot of csv files in it.
Can I read all of them, for example as zoo object without knowing the file name?
UPDATE
I tried that:
files <- list.files( "C://Users//ramid//Desktop//Files//" );
(na.omit(files))
for( i in files ) {
filePath <- gsub(" ","", paste("C://Users//ramid//Desktop//Files//",files[i],".csv"), fixed=TRUE)
cat(filePath)
df <- read.csv(gsub(" ","", filePath, fixed=TRUE), header = TRUE, sep = ";",stringsAsFactors=FALSE)
}
However I am getting an error:
Error in file(file, "rt") : cannot open the connection
In addition: Warning message:
In file(file, "rt") :
cannot open file 'C://Users//ramid//Desktop//Files//NA.csv': No such file or directory
I do not have any NA in my files list.
I'd use a combination of list.files and lapply:
list_of_files = list.files('.', pattern = '*csv', full.names = TRUE)
list_of_csv_contents = lapply(list_of_files, read.csv)
list_of_zoo = lapply(list_of_csv_contents, zoo)
Or wrap both the read.csv and zoo in one step:
read_into_zoo = function(path) {
contents = read.csv(path)
zoo_contents = zoo(contents)
return(zoo_contents)
}
list_of_zoo = lapply(list_of_files, read_into_zoo)
This strategy of storing things in lists/arrays/vectors/matrices and using apply style looping is a strategy that works very well in R.
I am trying to read multiple files and then combine them into one data frame. The code that I am using is as follows:
library(plyr)
mydata = ldply(list.files(path="Data load for stations/data/Predicted",pattern = "txt"), function(filename) {
dum = read.table(filename,skip=5, header=F, sep=" ")
#If you want to add the filename as well on the column
dum$filename = filename
return(dum)
})
The error that I am getting is as follows:
Error in file(file, "rt") : cannot open the connection
In addition: Warning message:
In file(file, "rt") :
cannot open file 'mobdata201001.txt': No such file or directory
The data files can be found on https://www.dropbox.com/sh/827kmkrwd0irehk/BFbftkks42
Any help is highly appreciated.
Alternatively you can use argument full.names in list.files:
list.files(path="Data load for stations/data/Predicted",
pattern = "txt", full.names=TRUE)
It will add automatically the full path before the file name.
Try the following code:
library(plyr)
path <- "Data load for stations/data/Predicted/"
filenames <- paste0(path, list.files(path, pattern = "txt"))
mydata = ldply(filenames, function(filename) {
dum = read.table(filename,skip=5, header=F, sep=" ")
#If you want to add the filename as well on the column
dum$filename = filename
return(dum)
})
I think what is happening is you're generating a list of files relative to the path in list.files, and then asking read.table to take the filename without the rest of the path...