I'm new to R and programming and taking a Coursera course. I've asked in their forums, but nobody can seem to provide an answer in the forums. To be clear, I'm trying to determine why this does not output.
When I first wrote the program, I was getting accurate outputs, but after I tried to upload, something went wonky. Rather than producing any output with [1], [2], etc. when I run the program from RStudio, I only get the the blue +++, but no errors and anything I change still does not produce an output.
I tried with a previous version of R, and reinstalled the most recent version 3.2.1 for Windows.
What I've done:
Set the correct working directory through RStudio
pol <- function(directory, pol, id = 1:332) {
files <- list.files("specdata", full.names = TRUE);
data <- data.frame();
for (i in ID) {
data <- rbind(data, read.csv(files_list[i]))
}
subset <- subset(data, ID %in% id);
polmean <- mean(subset[pol], na.rm = TRUE);
polmean("specdata", "sulfate", 1:10)
polmean("specdata", "nitrate", 70:72)
polmean("specdata", "nitrate", 23)
}
Can someone please provide some direction - debug help?
when I adjust the code the following errors tend to appear:
ID not found
Missing or unexpected } (although I've matched them all).
The updated code is as follow, if I'm understanding:
data <- data.frame();
files <- files[grepl(".csv",files)]
pollutantmean <- function(directory, pollutant, id = 1:332) {
pollutantmean <- mean(subset1[[pollutant]], na.rm = TRUE);
}
Looks like you haven't declared what ID is (I assume: a vector of numbers)?
Also, using 'subset' as a variable name while it's also a function, and pol as both a function name and the name of one of the arguments of that same function is just asking for trouble...
And I think there is a missing ")" in your for-loop.
EDIT
So the way I understand it now, you want to do a couple of things.
Read in a bunch of files, which you'll use multiple times without changing them.
Get some mean value out of those files, under different conditions.
Here's how I would do it.
Since you only want to read in the data once, you don't really need a function to do this (you can have one, but I think it's overkill for now). You correctly have code that makes a vector with the file names, and then loop over over them, rbinding them to each other. The problem is that this can become very slow. Check here. Make sure your directory only contains files that you want to read in, so no Rscripts or other stuff. A way (not 100% foolproof) to do this is using files <- files[grepl(".csv",files)], which makes sure you only have the csv's (grepl checks whether a certain string is a substring of another, and returns a boolean the [] then only keeps the elements for which a TRUE was returned).
Next, there is 'a thing you want to do multiple times', namely getting out mean values. This is where you'd use a function. Apparently you want to get the mean for different types of pollution, and you want this in restricted IDs.
Let's assume that 1. has given you a dataframe df with a column named Type for the type of pollution and a column called Id that somehow represents a sort of ID (substitute with the actual names in your script - if you don't have a column for ID, I'll edit the answer later on). Now you want a function
polmean <- function(type, id) {
# some code that returns the mean of a restricted version of df
}
This is all you need. You write the code that generates df, you then write a function that will get you what you want from that dataframe, and then you call it for the circumstances you want to use it in (the three polmean calls at the end of your original code, but now without the first argument as you no longer need this).
Ok - I finally solved this. Thanks for the help.
I didn't need to call "specdata" in line 2. the directory in line 1 referred to the correct directory.
My for/in statement needed to refer the the id in the first line not the ID in the dataset. The for/in statement doesn't appear to need to be indented (but it looks cleaner)
I did not need a subset
The last 3 lines for pollutantmean did not need to be a part of the program. These are used in the R console to call the results one by one.
Related
So, Im creating a loop in R that reads through multiple csv files in a directory called "specdata", and afterwards, tells you the mean of a particular colum in common inside those files. This function is represented in the next parragraph the arguments you specify are the directory in which those files are located, the colum you want means to be calculated, and id sequence, that tells you how many files do you want to read depending of de object number represented throudh subsetting []
I made a querie about this function before, and it was solved, now, it works, and gives a result. But it gives an incorrect one, it gives NA or NAN always, when it should give a number.
pollutantmean <- function(directory,pollutant,id) {
for (i in id) {archivo <- list.files(directory,full.names = TRUE)
datapollution <- rbind(read.csv(archivo[i],header = TRUE))
datamatrix <- data.matrix(datapollution)
resultmean <- mean(datamatrix[pollutant],na.rm = TRUE)}
print(resultmean)}
why is it not working? my theory is that im aplying rbind incorrectly.
It's difficult to provide more specific help due to the lack of sample data/code, but I see a couple of issues with your code.
There is no need to repeatedly list.file inside the for loop.
In fact, there is no need for a for loop here, and it will be faster to do something like
archive <- list.files(directory, full.names = TRUE)
datapollution <- do.call(rbind, lapply(archive, read.csv))
PS. For maximum help here on SO, it's always best to provide a minimal & reproducible example including sample data.
I have a large number of files (in GB size).I want to run a for loop in which I call some files, do so processing that creates some files, bind them together, and save it.
AA<-c(1,6)
BB<-c(5,10)
for(i in length(AA)){
listofnames<-list.files(pattern="*eng")
listofnames<- listofnames[c(paste(AA[i],BB[i],sep=":"))]
listoffiles <- lapply( listofnames, readRDS)
}
But listofnames has NA. What I am doing wrong?
It took me a while looking at your code to realize that you were actually trying to construct a character representation of the expression 1:5 that was supposed to index a vector by position. This is very wrong; you just can't paste together arbitrary R commands/expressions and expect to drop them in to you code wherever. (Technically, there are tools that do that sort of thing, but they are discouraged.)
Probably you're looking to do something closer to:
listofnames <- list.files(pattern="*eng")
ind <- rep(1:5,each = 5,length.out = length(listofnames))
listofnames_split <- split(listofnames,ind)
for (i in seq_along(listofnames_split)){
my_data <- lapply(listofnames_split[[i]], readRDS)
#Do processing here
#...
rm(my_data) #Assuming memory really is a problem
}
But I'm just sketching out hypothetical code here, I can't really match it to your exact situation since your example isn't really fully fleshed out.
I'm still a rookie to the R world, in a very accelerated class with limited/no guidance. My assignment is to build a custom function that reads in a specific .csv, and take some specific columns out to be analyzed. Could anyone please offer some advice? The "sample code" I was given looks like this:
AnnualLekSurvey=function(data.in,stat.year){
d1=subset(data.in,year==stat.year)
d2=d1[c("year","complex","tot_male")]
attach(d2)}
So when it's complete and I run it, I should be able to say:
AnnualLekSurvey(gsg_lek,2006)
where "gsg_lek" is the name of the file I want to import, and 2006 is the values from the "year" column that I want to subset. "complex" and "tot_male" will be the variable to be analyzed by "year", but I'm not worried about that code right now.
What I'm confused about is; how do I tell R that gsg_lek is a .csv file, and tell it to look in the proper directory for it when I run the custom function?
I saw one other vaguely similar example on here, and they had to use the if() and paste() commands to build the string of the file name - that seems like too much arbitrary work, unless I'm just being lazy...
Any help would be appreciated.
You can make a function like this:
AnnualLekSurvey <- function(csvFile, stat.year)
{
d1 <- read.csv(paste("C:/",csvFile,".csv", sep=""),header=T, sep=",")
d2 <- subset(d1, year==stat.year)
d2 <- d2[, c("year","complex","tot_male")]
return(d2)
}
The argument 'csvFile' in the function is the basename of your csv file. In this particular example, this has to be in your C:/ folder. If your file is in some other folder, you have to change the "C:/" in the function to the folder where your csv file is located.
Running the function:
data <- AnnualLekSurvey("gsg_lek", "2006")
Note that the arguments has to be within the quotes. 'data' will now contain the columns year, complex and tot_male of gsg_lek.csv corresponding to the year 2006
guys, thanks for read this. This is my first time writing a program so pardon me if I make stupid questions.
I have bunch of .csv files named like: 001-XXX.csv;002-XXX.csv...150-XXX.csv. Here XXX is a very long name tag. So it's a little annoying that every time I need to type read.csv("001-xxx.csv"). I want to make a function called "newread" that only ask me for the first three digits, the real id number, to read the .csv files. I thought "newread" should be like this:
newread <- function(id){
as.character(id)
a <- paste(id,"-XXX.csv",sep="")
read.csv(a)
}
BUt R shows Error: unexpected '}' in "}" What's going wrong? It looks logical.
I am running Rstudio on Windows 8.
as.character(id) will not change id into a character string. Change it to:
id = as.character(id)
Edit: According to comments, you should call newread() with a character paramter, and there is no difference between newread(001) and newread(1).
This is not specifically an answer to your question (others have covered that), but rather some advice that may be helpful for accomplishing your task in a different way.
First, some of the GUI's for R have file name completion. You can type the first part: read.csv("001- and then hit a key or combination of keys (In the windows GUI you press TAB) and the rest of the filename will be filled in for you (as long as it is unique).
You can use the file.choose or choose.files functions to open a dialog box to choose your file using the mouse: read.csv(file.choose()).
If you want to read in all the above files then you can do this in one step using lapply and either sprintf or list.files (or others):
mycsvlist <- lapply( 1:150, function(x) read.csv( sprintf("%03d-XXX.csv", x) ) )
or
mvcsvlist <- lapply( list.files(pattern="\\.csv$"), read.csv )
You could also use list.files to get a list of all the files matching a pattern and then pass one of the returned values to read.csv:
tmp <- list.files(pattern="001.*csv$")
read.csv(tmp[1])
I am a newcomer to R. Last week I had a long and complicated function working perfectly. The program was letting me pick a subset of columns and doing various manipulations on that subset. The function must work 'function(arg1=first_header_name, arg2=second_header_name,....)'. I have cleared the console, removed the old history file. I have read the manual again, I have checked the .csv file to make sure everything there is still the same. I have gone back and reworked it all step by step and I have the place where this new problem occurs. As it is a very long function, I am only going to reproduce it in a simplified version of the part that is suddenly not working.
elbow <- function(arg1,arg2) {
my_data <- read.csv("data.csv", header=TRUE, sep=",")
average_A <- (arg1 + arg2)
average_A
}
elbow(A3,A5)
# Error in elbow(A3, A5) : object 'A3' not found
Column headers are A3,A4,A5,A7,A8,A9,B2,B3,B5,B6,B7,B9
What stupid little error am I making? This is driving me batty. It has to be something trivial.
Here's my guess at what might work the way you wanted:
elbow <- function(arg1,arg2) {
my_data <- read.csv("data.csv", header=TRUE, sep=",")
average_A <- my_data[[arg1]] + my_data[[arg2]] # "[[" evaluates args
average_A
}
elbow('A3','A5') # entered a character literals
You should realize that the rest of my_data will have evaporated and be garbage collected after return from the elbow call. I could have showed you how to use your original expression following attach(), which would have been arguably safe within that function, but that would have violated my religious principles.
Probably during your last session you had objects named A3 or A5 in your workspace (either defined explicitly, or perhaps you had loaded and attached the data). The function was working because those objects were there, but it wasn't actually doing what you thought it was doing, so in a new session with a new workspace--without those objects--it's not working. Your function as written doesn't actually do anything with the dataset (my_data) which you are reading in inside of it; I suspect you want something like this:
elbow <- function(arg1, arg2) {
my_data <- read.csv("data.csv",header=TRUE,sep=",")
average_A <- my_data[,arg1] + my_data[,arg2]
return(average_A)
}
You will also need to use quotes when calling the function, e.g.
elbow('A3','A5')