I want to create a funtion that creates new data frames using some variables from other data frames. For that I thing I need to print the variable names in my own function somehow.
The variables come from two data frames (asd and tetracam) which have six variables in common, the bands "w530", "w550", "w570", "670", "w700" and "w800". So, I want to create six data frames, one for each band. One by one I could write like this:
# Band w530
w530<-data.frame(tetracam$filename,tetracam$time,tetracam$type,tetracam$w530,asd$w530)
names(w530)<-c("filename","time","type","tetracam","asd")
w530<-w530[order(w530$time),]
It works fine but I'd like to do it as a function in order to run for all bands. I thought I have to replace all the w530 in the code above for a dinamic object. As I thought of using some of the apply family. So, I first created a list with the names of my common variables:
bands<-c("w530","w550","w570","670","w700","w800")
Then, I tried several ways, for example, using cat or sprintf that would use the strings from the list to fill my function. But it didn't work. Actually, I'm not sure which apply family function I would use. If it's possible to use any in this case:
my.fun<- function(band){
sprintf("%s<-data.frame(tetracam$filename,tetracam$time,tetracam$type,asd$%s,tetracam$%s)",band,band,band)
sprintf("names(%s)<-c('filename','time','type','asd','tetracam')",band)
sprintf("%s[order(%s$time),]",band,band)
}
Any help is appreciated.
Trick is to access data.frame column using df[varName] idiom.
fun1 <- function(band, tetracam, asd){
df<-data.frame(tetracam$filename,tetracam$time,tetracam$type,tetracam[band],asd[band])
names(df)<-c("filename","time","type","tetracam","asd")
df<-df[order(df$time),]
return(df)
}
for (band in bands){
single_band_df <- fun1(band, tetracam, asd)
}
Related
This is all about a code in R.
I have seperated a big data file "All_data.csv" in smaller data of individuals in a particular year.
So the list looks like this:
All individuals is a list of 25 individuals. If you would then take the first element of that list you get:
Indivudual 1: dataframe_year1, dataframe_year2.
If you take the second element you get for instance:
Individual 2: dataframe_year1, dataframe_year2, dataframe_year3.
etc. so the lists in the lists differ in their length.
Now I want to do a (analysis) function on the dataframes, I do not need to store the output again in the list per se.
I solved it with doing an lapply on the list All_data, with a function defined by myself which also calls lapply again and then my analysis function. But I was wondering if there was another way. Because it seems a bit inefficient to do.
split <- function (All_data)
{
#function that splits files by date and individual
#returns list of individuals and within that list is another list of dataframes. Called All_individuals
}
Make_analysis <- function (All_individuals)
{
Listfiles <- split (All_individuals)
HRE <- lapply (Listfiles, Doall)
}
Analysis <- function (files)
{
...
}
function calls:
lapply (All_data, Make_analysis)
Could anyone help?
Also is this the best way to go if I would want to parallise the analysis with RSlurm to run it on a HPC? Then I could change lapply with slurm map right?
My function in itself works but it seems very inefficient. Would like some tips on how to make code more efficient. Also on how to parallise it with Rslurm.
I almost finish a messy code to apply several statistical methods/test to 11 data frames from different watersheds with physico-chemical parameters as variables. I reach the goal, but I need to do this functional.
So to start i made a function to compute correlation, and save the results as .txt tables and .pdf images.
It works great when run the function to one dataframe at the time (for that you should import each dataframe separately using read.table, which is not written in the code below).
As i want it functional, made a list of the 11 dataframes and use lapply to run the function to each one. It works in the sense that gives me one list (corr) containing the correlation results of each dataframe.
Here comes the issues:
The list cor with correlation results for each dataframe looks like has values instead of data frames, so i dont know how to access or save them (see the corr list in the Environment/Data window). Well, until here, at least looks like correlation results exists somewhere.
The second problem is that when i run corr<-lapply(PQ_data, cor_PQ), which has a line to save the outputs as tables (.txt) and images (.pdf) using part of the name of the original dataframe computed (e.g first element of PQ_data is "AgIX_E_PQ" so table and plot of cor_PQ(PQ_data[["AgIX_E_PQ"]] should get the names "mCorAgIX_E_PQ.txt" and "CorAgIX_E_PQ.pdf" respectively), im getting just one output (mCorX[[I]].txt and CorX[[i]].pdf) with the last dataframe correlation result. That is, tables and images for each dataframe correlation result are overwritten into this generics mCorX[[I]].txt, CorX[[i]].pdf files.
Now i guess have to define 'i' or something to avoid this. Should i define cor_PQ function for PQ_data instead X?
If anyone can see where im failing, i will appreciate any help to solve this, please.
My data: PQ_data /save it in your workspace and fix setwd with it.
My code:
rm(list=ls(all=TRUE))
cat("\014")
setwd("C:/Users/Sol/Documents/ProyectoTÃtulo/CalidadAgua/Matrices/Regs") #my workspace
PQ_files<-list.files(path="C:/Users/Sol/Documents/ProyectoTÃtulo/CalidadAgua/Matrices/Regs",
pattern="\\_PQ.txt") #my list of 14 dataframes in my workspace.
PQ_data<-lapply(PQ_files, read.table) #read tables of the 14 dataframes in the list.
names(PQ_data)<-gsub("\\_PQ.txt","", PQ_files) #name the 14 dataframes with their original names.
#FUNCTION TO COMPUTE CORRELATIONS, SAVE TABLES AND PLOTS.
cor_PQ<-function(X) {
corPQ<-cor(X, use="pairwise.complete.obs")
outputname.txt<-paste0("mCor",deparse(substitute(X)),".txt")
write.table(corPQ, file=outputname.txt)
outputname.pdf<-paste0("Cor",deparse(substitute(X)),".pdf")
pdf(outputname.pdf)
plot(X)
dev.off()
return(corPQ)
}
corr<-lapply(PQ_data, cor_PQ)
After this, as i said, a get a list called "corr" with 11 elements containing correlation results from each dataframe in my list (PQ_data), but i cant access them as tables when i pin the "corr" list in my environment/data window (they dont show the blue R arrow to expand the element).
`
And i get only 2 output files called mCorX[[I]].txt and CorX[[i]].pdf showing only the last dataframe correlation result because the write.table and .pdf functions overwrite the results of the 10 previous calculations.
Again, i will appreciate any help. I really need a push to catch the idea.
Thanks!!!
lapply doesn't send names of the list to the function. So although the function works for individual files it doesn't work with list of files. Also since there are no names to the files all the files generated are given the same name, hence all the new files overwrite the previously existing files and in the end you get output with only 1 file which is the last element in your list. You can use the below function where we send the names as different parameter to assign the name to the files.
cor_PQ<-function(X, Y) {
corPQ<-cor(X, use="pairwise.complete.obs")
outputname.txt<-paste0("mCor",Y,".txt")
write.table(corPQ, file= outputname.txt)
outputname.pdf<-paste0("Cor",Y,".pdf")
pdf(outputname.pdf)
plot(X)
dev.off()
return(corPQ)
}
Now use Map to apply the same function.
Map(cor_PQ, PQ_data, names(PQ_data))
We can also use imap from purrr to apply this function.
purrr::imap(PQ_data, cor_PQ)
I have the following data frames. These already exist and they are identical but with different contents and row count. I want to assign dataframe FX_nyear to Astar in each iteration.
FX_3year
FX_4year
FX_5year
...
and I want to run some complex analysis etc. I do not want to use lapply. Just a simple For loop as shown below:
for(n in 3:n)
{ Astar = assign(paste("FX_",n,"year",sep="")) }
While I can get Astar named to "FX_3year" using only paste, I am having trouble setting Astar to the actual pre-existing data frame FX_3year.
I know this is a very basic question and variants of this have been asked in the past, but I cannot get it to work.
You can use get()
for(i in 3:n){
assign("Astar",get(paste("FX_",i,"year",sep="")))
}
I have multiple csv-files in one folder. I want to load each csv-file in this folder into one separate data frame. Next, I want to extract certain elements from this data frame into a matrix and calculate the mean of all these matrixes.
setwd("D:\\data")
group_1<-list.files()
a<-length(group_1)
mferg_mean<-data.frame
for(i in 1:a)
{
assign(paste0("mferg_",i),read.csv(group_1[i],header=FALSE,sep=";",quote="",dec=",",col.names=1:90))
}
As there are 11 csv-files in the folder I now have the data frames
mferg_1
to
mferg_11
How can I address each data frame in this loop? As mentioned, I want to extract certain elements from each data frame to a matrix. I would imagine it something like this:
assign(paste0("mferg_matrix_",i),mferg_i[1:5,1:10])
But this obviously does not work because R does not recognize mferg_i in the loop. How can I address this data frame?
This is not something you should probably be using assign for in the first place. Working with a bunch of different data.frames in R is a mess, but working with a list of data.frames is much easier. Try reading your data with
group_1<-list.files()
mferg <- lapply(group_1, function(filename) {
read.csv(filename,header=FALSE,sep=";",quote="",dec=",",col.names=1:90))
})
and you get each each value with mferg[[1]], mferg[[1]], etc. And then you can create a list of extractions with
mferg_matrix <- lapply(mferg, function(x) x[1:5, 1:10])
This is the more R-like way to do things.
But technically you can use get to retrieve values like you use assign to create them. For example
assign(paste0("mferg_matrix_",i),get(paste0("mferg_",i))[1:5,1:10])
but again, this is probably not a smart strategy in the long run.
this question might be very simple, but I do not find a good way to solve it:
I have a dataset with many subgroups which need to be analysed all-together and on their own. Therefore, I want to use subsets for the groups and use them for the later analysis. As well, the defintion of the subsets as the analysis should be partly done with loops in order to save space and to ensure that the same analysis has been done with all subgroups.
Here is an example of my code using an example dataframe from the boot package:
data(aids)
qlist <- c("1","2","3","4")
for (i in length(qlist)) {
paste("aids.sub.",qlist[i],sep="") <- subset(aids, quarter==qlist[i])
}
The variable which contains the subgroups in my dataset is stored as a string, therefore I added the qlist part which would be not required otherwise.
Make a list of the subsets with lapply:
lapply(qlist, function(x) subset(aids, quarter==x))
Equivalently, avoiding the subset():
lapply(qlist, function(x) aids[aids$quarter==x,])
It is likely the case that using a list will make the subsequent code easier to write and understand. You can subset the list to get a single data frame (just as you can use one of the subsets, as created below). But you can also iterate over it (using for or lapply) without having to construct variable names.
To do the job as you are asking, use assign:
for (i in qlist) {
assign(paste("aids.sub.",i,sep=""), subset(aids, quarter==i))
}
Note the removal of the length() function, and that this is iterating directly over qlist.