Formatting multiple files at once in R - r

Question

I'm quite new to R so I hope this question will still be interesting. I created a for loop which produced 11 csv files. Here's the code I used for that in case that could help clarify the question:
for (i in seq(0, 1, by = 0.1))
{collar$results2<-mutate(collar,results2 = case_when( (probability > i & results1 == "POSITIVE") | (probability < i & results1 == "NEGATIVE") ~ TRUE, TRUE ~ FALSE) )
as.character(collar$results2)
collaraccuracy1=paste('collar41361_41365', i, 'csv', sep = '.')
write.csv(collar,collaraccuracy1)}
As you can see, all files created have the following format: collar41361_41365.i.csv, with ``i` ranging from 0 to 1 every 0.1, like so:
[1] "collar41361_41365.0.csv"
[1] "collar41361_41365.0.1.csv"
[1] "collar41361_41365.0.2.csv"
[1] "collar41361_41365.0.3.csv"
[1] "collar41361_41365.0.4.csv"
[1] "collar41361_41365.0.5.csv"
[1] "collar41361_41365.0.6.csv"
[1] "collar41361_41365.0.7.csv"
[1] "collar41361_41365.0.8.csv"
[1] "collar41361_41365.0.9.csv"
[1] "collar41361_41365.1.csv"
Now, I'd like to format all the files at one since they have the same structure (10 columns, 240 rows and same column header) and same name format.
See below the code with the actions I've been trying take over this 11 files. I've used Sys.glob as this was mentionned to be the best way to perform the task in another post. I've previously coded this actions for a single file and it worked. I now want to apply the code for all 11 files at once:
#1) Reading multiple files at one. Now, this will only work for the files with a decimal value of i in their name -which is fine-. If I was reading files with i=0 or i=1, then we'll have the pattern "collar41361_41365.*.csv". Am I right?
collaraccuracy<-lapply(Sys.glob("collar41361_41365.***.csv"), read.csv)
#2) Select only the columns with header "observed","predicted","probability","results1","results2.results2"
collaraccuracy<-fread("collar41361_41365.***.csv",select=c("observed","predicted","probability","results1","results2.results2"),stringsAsFactors = F)
#3) Rename column "results2.results2" to "results2"
colnames(collaraccuracy)<-c("observed","predicted","probability","results1","results2")
#4) Create 6th column "results" by merging columns "results1" and "results2"
collaraccuracy$results <- paste(collaraccuracy$results2,
collaraccuracy$results1,sep="_")
#5) End of the formatting. Write new formated csv files with the pattern "collar41361_by_41365.i.csv"
collaraccuracy2=paste('collar41361_by_41365', i, 'csv', sep = '.')
write.csv(collaraccuracy,collaraccuracy2)
As you can see, I've 5 different actions to compute the should end up with eventually 9 files for i values of (0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9)
I'm specially concerned with the syntax in actions 1) and 2), but this is the best I can do so far.
Any tips to formulate this? Any help is appreciated!
P.S. Update: I've tried to create a function and applying it to the rest of the files with lapply:
collarcolumns<-function(collaraccuracy1)
{collaraccuracy1<-fread(("collar41361_41365.1.csv"),select=c("observed","predicted","probability","results1","results2.results2"),stringsAsFactors = F)
colnames(collaraccuracy1)<-c("observed","predicted","probability","results1","results2")
collaraccuracy1$results <- paste(collaraccuracy1$results2, collaraccuracy1$results1,sep="_")
collaraccuracy2=paste('collar41361_by_41365', i, 'csv', sep = '.')
write.csv(collaraccuracy1,collaraccuracy2)}
lapply(Sys.glob("collar41361_41365.*.csv"), collarcolumns)
And got 11 "NULL"s printed by R.. Was I on the right track?

Answer 1

Taking a step back, it sounds like you want to do the following for each i:
Add a column results2 that checks whether the predicted value matches the observed value with probability i.
Add a column results that concatenates results1 and results2.
The reason you're seeing strange column names like results2.results2 is that the original for loop is redundant; you don't need both an assignment statement (collar$results2 <- ...) and mutate. We can strip the whole thing down to one loop, like this:
for(i in seq(0, 1, by = 0.1)) {
collar.temp = collar %>%
mutate(results2 = case_when((probability > i & results1 == "POSITIVE") |
(probability < i & results1 == "NEGATIVE") ~ T,
T ~ F)) %>%
mutate(results = paste(results1, results2, sep = "_"))
collaraccuracy1 = paste('collar41361_41365', i, 'csv', sep = '.')
write.csv(collar.temp, collaraccuracy1)
}
Taking yet a further step back, are you sure you want 11 separate tables? It looks to me like you're effectively checking how accurate the predictions are at various "confidence" cutoffs. One way to put the data into a tidy format would be like this, where cutoff is its own column:
collar.tidy = do.call(
"bind_rows",
lapply(
seq(0, 1, by = 0.1),
function(x) {
collar %>%
mutate(cutoff = x,
results2 = case_when((probability > x & results1 == "POSITIVE") |
(probability < x & results1 == "NEGATIVE") ~ T,
T ~ F)) %>%
mutate(results = paste(results1, results2, sep = "_"))
}
)
)
See here for a great introduction to tidy data. You may think of other ways to tidy up this dataset; for example, it's not clear to me whether the results column that concatenates two other columns is strictly necessary.