I'm looking for the (1) name and (2) a (cleaner) method in R (base and data.table preferred) of the following.
Input
> d1
id x y
1 1 1 NA
2 2 NA 3
3 3 4 NA
> d2
id x y z
1 4 NA 30 a
2 3 20 2 b
3 2 14 NA c
4 1 15 97 d
(note that the actual data.frames have hundreds of columns)
Expected output:
> d1
id x y z
1 1 1 97 d
2 2 14 3 c
3 3 4 2 b
Data and current solution:
d1 <- data.frame(id = 1:3, x = c(1, NA, 4), y = c(NA, 3, NA))
d2 <- data.frame(id = 4:1, x = c(NA, 20, 14, 15), y = c(30, 2, NA, 97), z = letters[1:4])
for (col in setdiff(names(d1), "id")) {
# If missing look in d2
missing <- is.na(d1[[col]])
d1[missing, col] <- d2[match(d1$id[missing], d2$id), col]
}
for (col in setdiff(names(d2), names(d1))) {
# If column missing then add
d1[[col]] <- d2[match(d1$id, d2$id), col]
}
PS:
Likely this questions has been asked before but I'm lacking in vocabulary to search it.
Assuming you are working with 2 data.frames, here is a base solution
#expand d1 to have the same columns as d2
d <- merge(d1, d2[, c("id", setdiff(names(d2), names(d1))), drop=FALSE],
by="id", all.x=TRUE, all.y=FALSE)
#make sure that d2 also have same number of columns as d1
d2 <- merge(d2, d1[, c("id", setdiff(names(d1), names(d2))), drop=FALSE],
by="id", all.x=TRUE, all.y=FALSE)
#align rows and columns to match those in d1
mask <- d2[match(d1$id, d2$id), names(d)]
#replace NAs with those mask
replace(d, is.na(d), mask[is.na(d)])
If you dont mind, we can rewrite your question into a general matrix-coalesce question (i.e. any number of matrices, columns, rows) which seems like it has not been asked before.
edit:
Another base R solution is a hack of coalesce1a from How to implement coalesce efficiently in R
coalesce.mat <- function(...) {
ans <- ..1
for (elt in list(...)[-1]) {
rn <- match(ans$id, elt$id)
ans[is.na(ans)] <- elt[rn, names(ans)][is.na(ans)]
}
ans
}
allcols <- Reduce(union, lapply(list(d1, d2), names))
do.call(coalesce.mat,
lapply(list(d1, d2), function(x) {
x[, setdiff(allcols, names(x))] <- NA
x
}))
edit:
a possible data.table solution using coalesce1a from How to implement coalesce efficiently in R by Martin Morgan.
coalesce1a <- function(...) {
ans <- ..1
for (elt in list(...)[-1]) {
i <- which(is.na(ans))
ans[i] <- elt[i]
}
ans
}
setDT(d1)
setDT(d2)
#melt into long formats and full outer join the 2
mdt <- merge(melt(d1, id.vars="id"), melt(d2, id.vars="id"), by=c("id","variable"), all=TRUE)
#perform a coalesce on vectors
mdt[, value := do.call(coalesce1a, .SD), .SDcols=grep("value", names(mdt), value=TRUE)]
#pivot into original format and subset to those in d1
dcast.data.table(mdt, id ~ variable, value.var="value")[
d1, .SD, on=.(id)]
Here is a possibility using dplyr::left_join:
left_join(d1, d2, by = "id") %>%
mutate(
x = ifelse(!is.na(x.x), x.x, x.y),
y = ifelse(!is.na(y.x), y.x, y.y)) %>%
select(id, x, y, z)
# id x y z
#1 1 1 97 d
#2 2 14 3 c
#3 3 4 2 b
We can use data.table with coalesce from dplyr. Create a vector of column names that are common ('nm1') and difference ('nm2') in both datasets. Convert the first dataset to 'data.table' (setDT(d1)), join on the 'id' column, assign (:=) the coalesced columns of the first and second (with prefix i. - if there are common columns) to update the values in the first dataset
library(data.table)
nm1 <- setdiff(intersect(names(d1), names(d2)), 'id')
nm2 <- setdiff(names(d2), names(d1))
setDT(d1)[d2, c(nm1, nm2) := c(Map(dplyr::coalesce, mget(nm1),
mget(paste0("i.", nm1))), mget(nm2)), on = .(id)]
d1
# id x y z
#1: 1 1 97 d
#2: 2 14 3 c
#3: 3 4 2 b
Related
I have several data frames that I want to combine by row. In the resulting single data frame, I want to create a new variable identifying which data set the observation came from.
# original data frames
df1 <- data.frame(x = c(1, 3), y = c(2, 4))
df2 <- data.frame(x = c(5, 7), y = c(6, 8))
# desired, combined data frame
df3 <- data.frame(x = c(1, 3, 5, 7), y = c(2, 4, 6, 8),
source = c("df1", "df1", "df2", "df2")
# x y source
# 1 2 df1
# 3 4 df1
# 5 6 df2
# 7 8 df2
How can I achieve this?
Thanks in advance!
It's not exactly what you asked for, but it's pretty close. Put your objects in a named list and use do.call(rbind...)
> do.call(rbind, list(df1 = df1, df2 = df2))
x y
df1.1 1 2
df1.2 3 4
df2.1 5 6
df2.2 7 8
Notice that the row names now reflect the source data.frames.
Update: Use cbind and rbind
Another option is to make a basic function like the following:
AppendMe <- function(dfNames) {
do.call(rbind, lapply(dfNames, function(x) {
cbind(get(x), source = x)
}))
}
This function then takes a character vector of the data.frame names that you want to "stack", as follows:
> AppendMe(c("df1", "df2"))
x y source
1 1 2 df1
2 3 4 df1
3 5 6 df2
4 7 8 df2
Update 2: Use combine from the "gdata" package
> library(gdata)
> combine(df1, df2)
x y source
1 1 2 df1
2 3 4 df1
3 5 6 df2
4 7 8 df2
Update 3: Use rbindlist from "data.table"
Another approach that can be used now is to use rbindlist from "data.table" and its idcol argument. With that, the approach could be:
> rbindlist(mget(ls(pattern = "df\\d+")), idcol = TRUE)
.id x y
1: df1 1 2
2: df1 3 4
3: df2 5 6
4: df2 7 8
Update 4: use map_df from "purrr"
Similar to rbindlist, you can also use map_df from "purrr" with I or c as the function to apply to each list element.
> mget(ls(pattern = "df\\d+")) %>% map_df(I, .id = "src")
Source: local data frame [4 x 3]
src x y
(chr) (int) (int)
1 df1 1 2
2 df1 3 4
3 df2 5 6
4 df2 7 8
Another approach using dplyr:
df1 <- data.frame(x = c(1,3), y = c(2,4))
df2 <- data.frame(x = c(5,7), y = c(6,8))
df3 <- dplyr::bind_rows(list(df1=df1, df2=df2), .id = 'source')
df3
Source: local data frame [4 x 3]
source x y
(chr) (dbl) (dbl)
1 df1 1 2
2 df1 3 4
3 df2 5 6
4 df2 7 8
I'm not sure if such a function already exists, but this seems to do the trick:
bindAndSource <- function(df1, df2) {
df1$source <- as.character(match.call())[[2]]
df2$source <- as.character(match.call())[[3]]
rbind(df1, df2)
}
results:
bindAndSource(df1, df2)
1 1 2 df1
2 3 4 df1
3 5 6 df2
4 7 8 df2
Caveat: This will not work in *aply-like calls
A blend of the other two answers:
df1 <- data.frame(x = 1:3,y = 1:3)
df2 <- data.frame(x = 4:6,y = 4:6)
> foo <- function(...){
args <- list(...)
result <- do.call(rbind,args)
result$source <- rep(as.character(match.call()[-1]),times = sapply(args,nrow))
result
}
> foo(df1,df2,df1)
x y source
1 1 1 df1
2 2 2 df1
3 3 3 df1
4 4 4 df2
5 5 5 df2
6 6 6 df2
7 1 1 df1
8 2 2 df1
9 3 3 df1
If you want to avoid the match.call business, you can always limit yourself to naming the function arguments (i.e. df1 = df1, df2 = df2) and using names(args) to access the names.
Another workaround for this one is using ldply in the plyr package...
df1 <- data.frame(x = c(1,3), y = c(2,4))
df2 <- data.frame(x = c(5,7), y = c(6,8))
list = list(df1 = df1, df2 = df2)
df3 <- ldply(list)
df3
.id x y
df1 1 2
df1 3 4
df2 5 6
df2 7 8
Even though there are already some great answers here, I just wanted to add the one I have been using. It is base R so it might be be less limiting if you want to use it in a package, and it is a little faster than some of the other base R solutions.
dfs <- list(df1 = data.frame("x"=c(1,2), "y"=2),
df2 = data.frame("x"=c(2,4), "y"=4),
df3 = data.frame("x"=2, "y"=c(4,5,7)))
> microbenchmark(cbind(do.call(rbind,dfs),
rep(names(dfs), vapply(dfs, nrow, numeric(1)))), times = 1001)
Unit: microseconds
min lq mean median uq max neval
393.541 409.083 454.9913 433.422 453.657 6157.649 1001
The first part, do.call(rbind, dfs) binds the rows of data frames into a single data frame. The vapply(dfs, nrow, numeric(1)) finds how many rows each data frame has which is passed to rep in rep(names(dfs), vapply(dfs, nrow, numeric(1))) to repeat the name of the data frame once for each row of the data frame. cbind puts them all together.
This is similar to a previously posted solution, but about 2x faster.
> microbenchmark(do.call(rbind,
lapply(names(dfs), function(x) cbind(dfs[[x]], source = x))),
times = 1001)
Unit: microseconds
min lq mean median uq max neval
844.558 870.071 1034.182 896.464 1210.533 8867.858 1001
I am not 100% certain, but I believe the speed up is due to making a single call to cbind rather than one per data frame.
Here is one option using Map. First, I create a named list of dataframes. Then, I can cbind the names to each dataframe. Then, use unname to remove the row names. Finally, rbind all the dataframes together.
# original data frames
df1 <- data.frame(x = c(1, 3), y = c(2, 4))
df2 <- data.frame(x = c(5, 7), y = c(6, 8))
df.list <- Hmisc::llist(df1, df2)
do.call(rbind, unname(Map(cbind, source = names(df.list), df.list)))
Output
source x y
1 df1 1 2
2 df1 3 4
3 df2 5 6
4 df2 7 8
I have several data frames that I want to combine by row. In the resulting single data frame, I want to create a new variable identifying which data set the observation came from.
# original data frames
df1 <- data.frame(x = c(1, 3), y = c(2, 4))
df2 <- data.frame(x = c(5, 7), y = c(6, 8))
# desired, combined data frame
df3 <- data.frame(x = c(1, 3, 5, 7), y = c(2, 4, 6, 8),
source = c("df1", "df1", "df2", "df2")
# x y source
# 1 2 df1
# 3 4 df1
# 5 6 df2
# 7 8 df2
How can I achieve this?
Thanks in advance!
It's not exactly what you asked for, but it's pretty close. Put your objects in a named list and use do.call(rbind...)
> do.call(rbind, list(df1 = df1, df2 = df2))
x y
df1.1 1 2
df1.2 3 4
df2.1 5 6
df2.2 7 8
Notice that the row names now reflect the source data.frames.
Update: Use cbind and rbind
Another option is to make a basic function like the following:
AppendMe <- function(dfNames) {
do.call(rbind, lapply(dfNames, function(x) {
cbind(get(x), source = x)
}))
}
This function then takes a character vector of the data.frame names that you want to "stack", as follows:
> AppendMe(c("df1", "df2"))
x y source
1 1 2 df1
2 3 4 df1
3 5 6 df2
4 7 8 df2
Update 2: Use combine from the "gdata" package
> library(gdata)
> combine(df1, df2)
x y source
1 1 2 df1
2 3 4 df1
3 5 6 df2
4 7 8 df2
Update 3: Use rbindlist from "data.table"
Another approach that can be used now is to use rbindlist from "data.table" and its idcol argument. With that, the approach could be:
> rbindlist(mget(ls(pattern = "df\\d+")), idcol = TRUE)
.id x y
1: df1 1 2
2: df1 3 4
3: df2 5 6
4: df2 7 8
Update 4: use map_df from "purrr"
Similar to rbindlist, you can also use map_df from "purrr" with I or c as the function to apply to each list element.
> mget(ls(pattern = "df\\d+")) %>% map_df(I, .id = "src")
Source: local data frame [4 x 3]
src x y
(chr) (int) (int)
1 df1 1 2
2 df1 3 4
3 df2 5 6
4 df2 7 8
Another approach using dplyr:
df1 <- data.frame(x = c(1,3), y = c(2,4))
df2 <- data.frame(x = c(5,7), y = c(6,8))
df3 <- dplyr::bind_rows(list(df1=df1, df2=df2), .id = 'source')
df3
Source: local data frame [4 x 3]
source x y
(chr) (dbl) (dbl)
1 df1 1 2
2 df1 3 4
3 df2 5 6
4 df2 7 8
I'm not sure if such a function already exists, but this seems to do the trick:
bindAndSource <- function(df1, df2) {
df1$source <- as.character(match.call())[[2]]
df2$source <- as.character(match.call())[[3]]
rbind(df1, df2)
}
results:
bindAndSource(df1, df2)
1 1 2 df1
2 3 4 df1
3 5 6 df2
4 7 8 df2
Caveat: This will not work in *aply-like calls
A blend of the other two answers:
df1 <- data.frame(x = 1:3,y = 1:3)
df2 <- data.frame(x = 4:6,y = 4:6)
> foo <- function(...){
args <- list(...)
result <- do.call(rbind,args)
result$source <- rep(as.character(match.call()[-1]),times = sapply(args,nrow))
result
}
> foo(df1,df2,df1)
x y source
1 1 1 df1
2 2 2 df1
3 3 3 df1
4 4 4 df2
5 5 5 df2
6 6 6 df2
7 1 1 df1
8 2 2 df1
9 3 3 df1
If you want to avoid the match.call business, you can always limit yourself to naming the function arguments (i.e. df1 = df1, df2 = df2) and using names(args) to access the names.
Another workaround for this one is using ldply in the plyr package...
df1 <- data.frame(x = c(1,3), y = c(2,4))
df2 <- data.frame(x = c(5,7), y = c(6,8))
list = list(df1 = df1, df2 = df2)
df3 <- ldply(list)
df3
.id x y
df1 1 2
df1 3 4
df2 5 6
df2 7 8
Even though there are already some great answers here, I just wanted to add the one I have been using. It is base R so it might be be less limiting if you want to use it in a package, and it is a little faster than some of the other base R solutions.
dfs <- list(df1 = data.frame("x"=c(1,2), "y"=2),
df2 = data.frame("x"=c(2,4), "y"=4),
df3 = data.frame("x"=2, "y"=c(4,5,7)))
> microbenchmark(cbind(do.call(rbind,dfs),
rep(names(dfs), vapply(dfs, nrow, numeric(1)))), times = 1001)
Unit: microseconds
min lq mean median uq max neval
393.541 409.083 454.9913 433.422 453.657 6157.649 1001
The first part, do.call(rbind, dfs) binds the rows of data frames into a single data frame. The vapply(dfs, nrow, numeric(1)) finds how many rows each data frame has which is passed to rep in rep(names(dfs), vapply(dfs, nrow, numeric(1))) to repeat the name of the data frame once for each row of the data frame. cbind puts them all together.
This is similar to a previously posted solution, but about 2x faster.
> microbenchmark(do.call(rbind,
lapply(names(dfs), function(x) cbind(dfs[[x]], source = x))),
times = 1001)
Unit: microseconds
min lq mean median uq max neval
844.558 870.071 1034.182 896.464 1210.533 8867.858 1001
I am not 100% certain, but I believe the speed up is due to making a single call to cbind rather than one per data frame.
Here is one option using Map. First, I create a named list of dataframes. Then, I can cbind the names to each dataframe. Then, use unname to remove the row names. Finally, rbind all the dataframes together.
# original data frames
df1 <- data.frame(x = c(1, 3), y = c(2, 4))
df2 <- data.frame(x = c(5, 7), y = c(6, 8))
df.list <- Hmisc::llist(df1, df2)
do.call(rbind, unname(Map(cbind, source = names(df.list), df.list)))
Output
source x y
1 df1 1 2
2 df1 3 4
3 df2 5 6
4 df2 7 8
I have several data frames that I want to combine by row. In the resulting single data frame, I want to create a new variable identifying which data set the observation came from.
# original data frames
df1 <- data.frame(x = c(1, 3), y = c(2, 4))
df2 <- data.frame(x = c(5, 7), y = c(6, 8))
# desired, combined data frame
df3 <- data.frame(x = c(1, 3, 5, 7), y = c(2, 4, 6, 8),
source = c("df1", "df1", "df2", "df2")
# x y source
# 1 2 df1
# 3 4 df1
# 5 6 df2
# 7 8 df2
How can I achieve this?
Thanks in advance!
It's not exactly what you asked for, but it's pretty close. Put your objects in a named list and use do.call(rbind...)
> do.call(rbind, list(df1 = df1, df2 = df2))
x y
df1.1 1 2
df1.2 3 4
df2.1 5 6
df2.2 7 8
Notice that the row names now reflect the source data.frames.
Update: Use cbind and rbind
Another option is to make a basic function like the following:
AppendMe <- function(dfNames) {
do.call(rbind, lapply(dfNames, function(x) {
cbind(get(x), source = x)
}))
}
This function then takes a character vector of the data.frame names that you want to "stack", as follows:
> AppendMe(c("df1", "df2"))
x y source
1 1 2 df1
2 3 4 df1
3 5 6 df2
4 7 8 df2
Update 2: Use combine from the "gdata" package
> library(gdata)
> combine(df1, df2)
x y source
1 1 2 df1
2 3 4 df1
3 5 6 df2
4 7 8 df2
Update 3: Use rbindlist from "data.table"
Another approach that can be used now is to use rbindlist from "data.table" and its idcol argument. With that, the approach could be:
> rbindlist(mget(ls(pattern = "df\\d+")), idcol = TRUE)
.id x y
1: df1 1 2
2: df1 3 4
3: df2 5 6
4: df2 7 8
Update 4: use map_df from "purrr"
Similar to rbindlist, you can also use map_df from "purrr" with I or c as the function to apply to each list element.
> mget(ls(pattern = "df\\d+")) %>% map_df(I, .id = "src")
Source: local data frame [4 x 3]
src x y
(chr) (int) (int)
1 df1 1 2
2 df1 3 4
3 df2 5 6
4 df2 7 8
Another approach using dplyr:
df1 <- data.frame(x = c(1,3), y = c(2,4))
df2 <- data.frame(x = c(5,7), y = c(6,8))
df3 <- dplyr::bind_rows(list(df1=df1, df2=df2), .id = 'source')
df3
Source: local data frame [4 x 3]
source x y
(chr) (dbl) (dbl)
1 df1 1 2
2 df1 3 4
3 df2 5 6
4 df2 7 8
I'm not sure if such a function already exists, but this seems to do the trick:
bindAndSource <- function(df1, df2) {
df1$source <- as.character(match.call())[[2]]
df2$source <- as.character(match.call())[[3]]
rbind(df1, df2)
}
results:
bindAndSource(df1, df2)
1 1 2 df1
2 3 4 df1
3 5 6 df2
4 7 8 df2
Caveat: This will not work in *aply-like calls
A blend of the other two answers:
df1 <- data.frame(x = 1:3,y = 1:3)
df2 <- data.frame(x = 4:6,y = 4:6)
> foo <- function(...){
args <- list(...)
result <- do.call(rbind,args)
result$source <- rep(as.character(match.call()[-1]),times = sapply(args,nrow))
result
}
> foo(df1,df2,df1)
x y source
1 1 1 df1
2 2 2 df1
3 3 3 df1
4 4 4 df2
5 5 5 df2
6 6 6 df2
7 1 1 df1
8 2 2 df1
9 3 3 df1
If you want to avoid the match.call business, you can always limit yourself to naming the function arguments (i.e. df1 = df1, df2 = df2) and using names(args) to access the names.
Another workaround for this one is using ldply in the plyr package...
df1 <- data.frame(x = c(1,3), y = c(2,4))
df2 <- data.frame(x = c(5,7), y = c(6,8))
list = list(df1 = df1, df2 = df2)
df3 <- ldply(list)
df3
.id x y
df1 1 2
df1 3 4
df2 5 6
df2 7 8
Even though there are already some great answers here, I just wanted to add the one I have been using. It is base R so it might be be less limiting if you want to use it in a package, and it is a little faster than some of the other base R solutions.
dfs <- list(df1 = data.frame("x"=c(1,2), "y"=2),
df2 = data.frame("x"=c(2,4), "y"=4),
df3 = data.frame("x"=2, "y"=c(4,5,7)))
> microbenchmark(cbind(do.call(rbind,dfs),
rep(names(dfs), vapply(dfs, nrow, numeric(1)))), times = 1001)
Unit: microseconds
min lq mean median uq max neval
393.541 409.083 454.9913 433.422 453.657 6157.649 1001
The first part, do.call(rbind, dfs) binds the rows of data frames into a single data frame. The vapply(dfs, nrow, numeric(1)) finds how many rows each data frame has which is passed to rep in rep(names(dfs), vapply(dfs, nrow, numeric(1))) to repeat the name of the data frame once for each row of the data frame. cbind puts them all together.
This is similar to a previously posted solution, but about 2x faster.
> microbenchmark(do.call(rbind,
lapply(names(dfs), function(x) cbind(dfs[[x]], source = x))),
times = 1001)
Unit: microseconds
min lq mean median uq max neval
844.558 870.071 1034.182 896.464 1210.533 8867.858 1001
I am not 100% certain, but I believe the speed up is due to making a single call to cbind rather than one per data frame.
Here is one option using Map. First, I create a named list of dataframes. Then, I can cbind the names to each dataframe. Then, use unname to remove the row names. Finally, rbind all the dataframes together.
# original data frames
df1 <- data.frame(x = c(1, 3), y = c(2, 4))
df2 <- data.frame(x = c(5, 7), y = c(6, 8))
df.list <- Hmisc::llist(df1, df2)
do.call(rbind, unname(Map(cbind, source = names(df.list), df.list)))
Output
source x y
1 df1 1 2
2 df1 3 4
3 df2 5 6
4 df2 7 8
I am looping through different data.tables and the variables in the data.table. But I'm having trouble referencing the variables inside of the for loop
dt1 <- data.table(a1 = c(1,2,3), a2 = c(4,5,2))
dt2 <- data.table(a1 = c(1,43,1), a2 = c(52,4,1))
For each datatable, I want to find the average of each variable for observations where that variable != 1. Below is my attempt which doesn't work:
dtname = 'dt'
ind = c('1', '2')
for (d in ind) {
df <- get(paste0('dt', d, sep=''))
for (v in ind) {
varname <- paste0('a', v, sep='')
df1 <- df %>%
filter(varname!=1) %>%
summarise(varname = mean(varname))
print(df1)
}
}
The desired output is to take and print the average of a1 = c(2,3) in dt1, the average of a2 = (4,5,2) in dt1, the average of a1 = c(43) in dt2, the average of a2 = c(54,4) in dt2.
What am I doing wrong here? In general, how should I reference a variable inside of a for loop (varname) that is pieced together by using the looping index (v) and something else?
For a purely data.table way, I would combine the different data.tables and compute the averages:
# Concatenate the data.tables:
all_dt <- rbind("dt1" = dt1, "dt2" = dt2, idcol = "origin")
all_dt
# origin a1 a2
# 1: dt1 1 4
# 2: dt1 2 5
# 3: dt1 3 2
# 4: dt2 1 52
# 5: dt2 43 4
# 6: dt2 1 1
# Melt so that "a1" and "a2" are labels in a group column:
all_dt <- melt(all_dt, id.vars="origin")
all_dt
# origin variable value
# 1: dt1 a1 1
# 2: dt1 a1 2
# 3: dt1 a1 3
# 4: dt2 a1 1
# 5: dt2 a1 43
# 6: dt2 a1 1
# 7: dt1 a2 4
# 8: dt1 a2 5
# 9: dt1 a2 2
# 10: dt2 a2 52
# 11: dt2 a2 4
# 12: dt2 a2 1
# Compute averages by each data.table and column group, ignoring 1s:
all_dt[value != 1, .(mean = mean(value)), by = .(origin, variable)]
# origin variable mean
# 1: dt1 a1 2.500000
# 2: dt2 a1 43.000000
# 3: dt1 a2 3.666667
# 4: dt2 a2 28.000000
I figured out a solution based on the comments of #Amar and #Scott Richie
for (d in ind) {
df <- get(paste0('dt', d, sep=''))
for (v in ind) {
varname <- paste0('a', v, sep='')
df1 <- df[eval(as.name(varname))!=1, .(mean =
mean(eval(as.name(varname))))]
print(df1)
}
}
Thanks EVERYONE!
Would go for a vectorised approach. You are using R!
One possible way:
require(dplyr)
dt1[dt1==1] <- NA #replace 1 with NA
dt1 %>% summarise_all(mean, na.rm = TRUE) #mean of all columns.
a1 a2
1 2.5 3.666667
It is not very clear what you are trying to do, but if you want to replace all of the rows in the dataframe with the mean of the previous data frame's columns, I would suggest using a dataframe type instead as it is easier to index. Here is code that should work:
dt1 <- data.frame(a1 = c(1,2,3), a2 = c(4,5,2))
dt2 <- data.frame(a1 = c(1,43,1), a2 = c(52,4,1))
dtname = 'dt'
ind = c('1', '2')
for (d in ind){
df <- get(paste0('dt', d, sep=''))
for (i in 1:nrow(df)){
for (j in 1:ncol(df)){
if (df[i,j] !=1){
df[,j]<- mean(df[,j])
}
}
print(df)
}
}
The reason your code was not working before was because the variables were being treated like strings, not actual variables. You can see this by printing the data type of variances:
dtname = 'dt'
ind = c('1', '2')
for (d in ind) {
df <- get(paste0('dt', d, sep=''))
for (v in ind) {
varname <- paste0('a', v, sep='')
print(class(varname))
}
}
Which just returns "character"
Another solution using variable names and the dataframe type would be to index the df as follows:
df[["varname"]]
Here are two helpful links for this kind of operation:
* link 1: How to find the mean of a column
* link 2: Data frames
I want to create a for loop by variable names.
Each time, I calculte the max between each two variables, and define a new one in data df. New variables look like this:var1_1, var1_2... Here is my code:
df=data.frame(matrix(c(1:6), nrow = 2))
colnames(df) = c("x", "y", "z")
for(i in length(names(df))-1){
df = df %>% mutate(paste0("var", i, "_", i+1) = max(names(df)[i], names(df)[i+1]))
}
But there gives error.
Expected output:
>df
x y z var1_2 var1_3 var2_3
1 3 5 3 5 5
2 4 6 4 6 6
One way via base R,
m1 <- sapply(combn(names(df),2, simplify = FALSE), function(i) do.call(pmax, df[i]))
nms <- combn(ncol(m1), 2, function(i) paste0('Var', i[1], '_', i[2]))
cbind(df, setNames(data.frame(m1), nms))
# x y z Var1_2 Var1_3 Var2_3
#1 1 3 5 3 5 5
#2 2 4 6 4 6 6
If you really want to use a Loop you can try:
ind<-combn(3,2)
for(i in 1:dim(df)[2]){
i <- ind[,i]
name <- paste0("var", i[1], "_", i[2])
val <- names(df)[i[ifelse(sum(df[,i[1]]) > sum(df[,i[2]]),1,2)]]
df <- mutate_(df, .dots= setNames(list(val),name))
}