Name columns with Dataframe name in a list of Dataframes - r

Objective: Change colname of dataframes in a list of dataframes to the name of each dataframe.
I have some issues when dealing with list and dataframes regarding its name. I have prepared this example to clarify. Hope it is not a mess.
Data:
df1 <- data.frame(A = 1, B = 2, C = 3)
df2 <- data.frame(A = 3, B = 3, C = 2)
dfList <- list(df1,df2)
Output:
> str(dfList)
List of 2
$ :'data.frame': 1 obs. of 3 variables:
..$ A: num 1
..$ B: num 2
..$ C: num 3
$ :'data.frame': 1 obs. of 3 variables:
..$ A: num 3
..$ B: num 3
..$ C: num 2
> names(dfList)
NULL
> names(dfList$df1)
NULL
> names(dfList$df2)
NULL
Manually Input names:
names(dfList) <- c("df1", "df2")
dfList <- lapply(dfList, setNames, c("A", "B", "C"))
Which yields:
> str(dfList)
List of 2
$ df1:'data.frame': 1 obs. of 3 variables:
..$ A: num 1
..$ B: num 2
..$ C: num 3
$ df2:'data.frame': 1 obs. of 3 variables:
..$ A: num 3
..$ B: num 3
..$ C: num 2
> names(dfList)
[1] "df1" "df2"
> names(dfList$df1)
[1] "A" "B" "C"
> names(dfList$df2)
[1] "A" "B" "C"
Desired Solution:
WishedList <- dfList
WishedList[[1]] <- setNames(WishedList[[1]], c("A", "B", "df1"))
WishedList[[2]] <- setNames(WishedList[[2]], c("A", "B", "df2"))
Output solution:
> str(WishedList)
List of 2
$ df1:'data.frame': 1 obs. of 3 variables:
..$ A : num 1
..$ B : num 2
..$ df1: num 3
$ df2:'data.frame': 1 obs. of 3 variables:
..$ A : num 3
..$ B : num 3
..$ df2: num 2
> names(WishedList)
[1] "df1" "df2"
> names(WishedList$df1)
[1] "A" "B" "df1"
> names(WishedList$df2)
[1] "A" "B" "df2"
MyTry:
TryList1 <- lapply(dfList, function(x) setNames(x, c("A", "B", quote(x))))
str(TryList1)
List of 2
$ df1:'data.frame': 1 obs. of 3 variables:
..$ A: num 1
..$ B: num 2
..$ x: num 3
$ df2:'data.frame': 1 obs. of 3 variables:
..$ A: num 3
..$ B: num 3
..$ x: num 2
Doubts:
1) Why when creating the file the names both of the dataframes and of the cols of the dataframes are not included in the list?
2) quote(x) with a single dataframe works. Why not in the list?
> df1 <- data.frame(A = 1, B = 2, C = 3)
> df1 <- setNames(df1, c("A", "B", quote(df1)))
> names(df1)
[1] "A" "B" "df1"
Thank you very much!

Here's a slightly different approach:
df1 <- data.frame(A = 1, B = 2, C = 3)
df2 <- data.frame(A = 3, B = 3, C = 2)
dfList <- list(df1,df2)
names(dfList) <- c("df1", "df2")
Map(function(df, dfn) {names(df)[3] <- dfn; df}, dfList, names(dfList))
#$df1
# A B df1
#1 1 2 3
#
#$df2
# A B df2
#1 3 3 2
You could alternatively use setNames(df, c("A", "B", dfn)) inside the mapply function.
A note on OP's trial: The documentation for quote states:
quote simply returns its argument.
That's why when you use quote(x) inside lapply, it simply returns the character x.

We can lapply() over names(dfList) instead of dfList:
lapply(names(dfList), function(dfn) {
df <- dfList[[dfn]]
names(df)[3] <- dfn
df
})
# [[1]]
# A B df1
# 1 1 2 3
#
# [[2]]
# A B df2
# 1 3 3 2
There's a convenience function in purrr that maps over a list and its names simultaneously:
library(purrr)
imap(dfList, ~ {
names(.x)[3] <- .y
.x
})
# $df1
# A B df1
# 1 1 2 3
#
# $df2
# A B df2
# 1 3 3 2
Or if you're after a short one-liner and don't mind hard-coding "A" and "B":
imap(dfList, ~ setNames(.x, c("A", "B", .y)))
(NB: Essentially those are just variations around Docendo discimus' answer).
Also, not your expected output but maybe of interest for you:
dplyr::bind_rows(dfList, .id = "origin")
# origin A B C
# 1 df1 1 2 3
# 2 df2 3 3 2
Or:
bind_rows(map(dfList, select, -C), .id = "C")
# C A B
# 1 df1 1 2
# 2 df2 3 3

Related

Combining two sets of variable in R

I am very new to R software and appreciate if you can provide some suggestions to combine variables (antibiotic) within common variable (antibiotic_date).
My original data looks like this (3X3 table);
id: 1
antibiotic: a, b, c
antibiotic_date: 2018-01-20, 2018-01-20, 2018-03-04
Is it possible to transform the above date to (3X table);
id: 1
antibiotic: a b, c
antibiotic_date: 2018-01-20, 2018-03-04
Thank you very much for your help.
Looks like you have
df
# id antibiotic antibiotic_date
# 1 1 a 2018-01-20
# 2 1 b 2018-01-20
# 3 1 c 2018-03-04
Use unique in aggregate.
(res1 <- aggregate(. ~ antibiotic_date, df, unique))
# antibiotic_date id antibiotic
# 1 2018-01-20 1 a, b
# 2 2018-03-04 1 c
Where
str(res1)
# 'data.frame': 2 obs. of 3 variables:
# $ antibiotic_date: chr "2018-01-20" "2018-03-04"
# $ id : chr "1" "1"
# $ antibiotic :List of 2
# ..$ : chr "a" "b"
# ..$ : chr "c"
If you need a string rather than a vector of length > 1 make it toString,
(res2 <- aggregate(. ~ antibiotic_date, df, \(x) toString(unique(x))))
# antibiotic_date id antibiotic
# 1 2018-01-20 1 a, b
# 2 2018-03-04 1 c
where:
str(res2)
# 'data.frame': 2 obs. of 3 variables:
# $ antibiotic_date: chr "2018-01-20" "2018-03-04"
# $ id : chr "1" "1"
# $ antibiotic : chr "a, b" "c"
Or paste,
(res3 <- aggregate(. ~ antibiotic_date, df, \(x) paste(unique(x), collapse=' ')))
# antibiotic_date id antibiotic
# 1 2018-01-20 1 a b
# 2 2018-03-04 1 c
where:
str(res3)
# 'data.frame': 2 obs. of 3 variables:
# $ antibiotic_date: chr "2018-01-20" "2018-03-04"
# $ id : chr "1" "1"
# $ antibiotic : chr "a b" "c"
You can also wrap a sort around it, if needed, e.g. sort(toString(unique(.))).
Data:
df <- structure(list(id = c(1, 1, 1), antibiotic = c("a", "b", "c"),
antibiotic_date = c("2018-01-20", "2018-01-20", "2018-03-04"
)), class = "data.frame", row.names = c(NA, -3L))

Renaming nested lists with lapply

I have a nested list:
my_list <- list(A = 1, B = 2, C = 3)
my_nested_list <- list(D = my_list, E = my_list, F = my_list)
I want to change the names of the inner most elements to the following:
my_names <- c("X", "Y", "Z")
So, A, B, C should becoming X, Y, Z.
Here is my attempt:
name_changer <- lapply(my_nested_list, FUN = function(x){
lapply(x, FUN = function(y){
names(y) <- my_names
})
})
Why does this not work?
In your example you have two lapply loops when you only need one.
Your original second loop tries to apply the 3-element "my_names" vector to each individual element of "my_list", which won't work, since the lengths don't match. But you don't need the second loop at all:
my_list <- list(A = 1, B = 2, C = 3)
my_nested_list <- list(D = my_list, E = my_list, F = my_list)
my_names <- c("X", "Y", "Z")
name_changer <- lapply(my_nested_list, FUN = function(x){
names(x) <- my_names
return(x)
})
str(name_changer)
List of 3
$ D:List of 3
..$ X: num 1
..$ Y: num 2
..$ Z: num 3
$ E:List of 3
..$ X: num 1
..$ Y: num 2
..$ Z: num 3
$ F:List of 3
..$ X: num 1
..$ Y: num 2
..$ Z: num 3
You can use setNames as well -
lapply(my_nested_list, setNames, my_names)
#$D
#$D$X
#[1] 1
#$D$Y
#[1] 2
#$D$Z
#[1] 3
#$E
#$E$X
#[1] 1
#$E$Y
#[1] 2
#$E$Z
#[1] 3
#$F
#$F$X
#[1] 1
#$F$Y
#[1] 2
#$F$Z
#[1] 3

bind list of lists based on the lists location

I have some data similar to mainList below.
List of 2
$ :List of 3
..$ :List of 1
.. ..$ :'data.frame': 3 obs. of 2 variables:
.. .. ..$ col1: chr [1:3] "1" "2" "3"
.. .. ..$ col2: chr [1:3] "a" "b" "c"
..$ :List of 1
.. ..$ :'data.frame': 3 obs. of 2 variables:
.. .. ..$ col1: chr [1:3] "3" "7" "4"
.. .. ..$ col2: chr [1:3] "e" "d" "g"
..$ :List of 1
.. ..$ :'data.frame': 3 obs. of 2 variables:
.. .. ..$ col1: chr [1:3] "2" "7" "4"
.. .. ..$ col2: chr [1:3] "l" "o" "i"
$ :List of 3
..$ :List of 1
.. ..$ :'data.frame': 3 obs. of 2 variables:
.. .. ..$ col1: chr [1:3] "8" "3" "4"
.. .. ..$ col2: chr [1:3] "r" "t" "q"
..$ :List of 1
.. ..$ :'data.frame': 3 obs. of 2 variables:
.. .. ..$ col1: chr [1:3] "7" "5" "2"
.. .. ..$ col2: chr [1:3] "h" "w" "p"
..$ :List of 1
.. ..$ :'data.frame': 3 obs. of 2 variables:
.. .. ..$ col1: chr [1:3] "9" "3" "6"
.. .. ..$ col2: chr [1:3] "x" "y" "z"
I want to merge, or bind the lists based on the lists location in the list of lists.
That is, I want to merge splt1 with splt11, and then merge splt2 with splt22 and finally splt3 with splt33.
So it would take the first data frame from the first List of 3 and merge it with the first data frame from the second List of 3.
This does not get what I want
mainList %>%
map(., ~bind_rows(., .id = "split"))
Since all of the splits are merged into a single data frame (I want them kept separate).
Data:
splt1 <- list(
data.frame(
col1 = c("1", "2", "3"),
col2 = c("a", "b", "c")
)
)
splt2 <- list(
data.frame(
col1 = c("3", "7", "4"),
col2 = c("e", "d", "g")
)
)
splt3 <- list(
data.frame(
col1 = c("2", "7", "4"),
col2 = c("l", "o", "i")
)
)
nestList1 <- list(
splt1,
splt2,
splt3
)
splt11 <- list(
data.frame(
col1 = c("8", "3", "4"),
col2 = c("r", "t", "q")
)
)
splt22 <- list(
data.frame(
col1 = c("7", "5", "2"),
col2 = c("h", "w", "p")
)
)
splt33 <- list(
data.frame(
col1 = c("9", "3", "6"),
col2 = c("x", "y", "z")
)
)
nestList2 <- list(
splt11,
splt22,
splt33
)
mainList <- list(
nestList1,
nestList2
)
EDIT:
Screenshot of the lists:
I am trying to bind together all of the split's, i.e.
split1 will contain the results from 08001, 08003, 08005 ... 0801501 for each of the lists in catalunya_madrid.
split2 will contain the same results 08001, 08003, 08005 ... 0801501
and so on.
EDIT2:
# Function to invert the list structure
invertListStructure <- function(ll) {
nms <- unique(unlist(lapply(ll, function(X) names(X))))
ll <- lapply(ll, function(X) setNames(X[nms], nms))
ll <- apply(do.call(rbind, ll), 2, as.list)
lapply(ll, function(X) X[!sapply(X, is.null)])
}
invertedList <- map(analysis, ~invertListStructure(.) %>%
map(., ~bind_rows(.x, .id = "MITMA")))
You can use purrr::transpose() to group list elements with the same location (i.e. the first element in list 1 with the first element in list 2 and list 3 and so on) for any number of lists. In your case, transpose will convert 592 lists of 216 into 216 lists of 592, each properly titled. With transpose, l[[x]][[y]] becomes l[[y]][[x]].
library(tidyverse)
mainList %>% purrr::transpose() %>%
map(function(x) {
flatten(x) %>% bind_rows(.id = 'id')
})
# $splt1
# id col1 col2
# 1 1 1 a
# 2 1 2 b
# 3 1 3 c
# 4 2 8 r
# 5 2 3 t
# 6 2 4 q
#
# $splt2
# id col1 col2
# 1 1 3 e
# 2 1 7 d
# 3 1 4 g
# 4 2 7 h
# 5 2 5 w
# 6 2 2 p
#
# $splt3
# id col1 col2
# 1 1 2 l
# 2 1 7 o
# 3 1 4 i
# 4 2 9 x
# 5 2 3 y
# 6 2 6 z
Note that you only need to flatten if the data.frame is in a list of length 1, by itself. If you have a list of data.frames (as opposed to a list of lists, each of which contains one data.frame, as in your example data), you can ignore the flatten() command and just bind the rows.
Your example dataset doesn't quite match your actual data, but if you make a list of two mainLists, it's closer. These types of operations are heavily dependent on the structure of the data, though, so I can't be sure this is what you need. All you need to do here is add a subscript.
mainList2 <- list(mainList, mainList) # First is Madrid, second is Valencia
# Operations are done on Madrid only
mainList2[[1]] %>%
transpose() %>%
map(function(x) {
flatten(x) %>% bind_rows(.id = 'id')
})
If you want to do this for both elements in mainList2, you can wrap the whole thing in map.
mainList2 %>% map(function(x) {
transpose(x) %>%
map(function(x) {
flatten(x) %>% bind_rows(.id = 'id')
})
})
You can combine the pairs in following way :
Map(rbind, unlist(mainList[[1]], recursive = FALSE),
unlist(mainList[[2]], recursive = FALSE))
Or using purrr you can also add an id column easily.
library(purrr)
map2(mainList[[1]] %>% flatten,
mainList[[2]] %>% flatten, dplyr::bind_rows, .id = 'id')
#[[1]]
# id col1 col2
#1 1 1 a
#2 1 2 b
#3 1 3 c
#4 2 8 r
#5 2 3 t
#6 2 4 q
#[[2]]
# id col1 col2
#1 1 3 e
#2 1 7 d
#3 1 4 g
#4 2 7 h
#5 2 5 w
#6 2 2 p
#[[3]]
# id col1 col2
#1 1 2 l
#2 1 7 o
#3 1 4 i
#4 2 9 x
#5 2 3 y
#6 2 6 z

Bind_rows() error: "Argument 1 must have names" // Occurs after tidyverse update

After a global tidyverse update, I have noted a change of behaviour in my code and after many researches I am desperately unable to solve the issue. Basically I need to convert a list of elements (including lists) to a dataframe.
Here is a reprex:
Data
x <- list(
col1 = list("a", "b", "c", NA),
col2 = list(1, 2, 3, 4),
col3 = list("value1", "value2", "value1", c("value1", "value2")))
Expected behaviour and output (before tidyverse update):
x <- data.frame((sapply(x, c)))
x <- purrr::map_df(x, function(x) sapply(x, function(x) unlist(x))) %>% as.data.frame()
> x
# col1 col2 col3
# 1 a 1 value1
# 2 b 2 value2
# 3 c 3 value1
# 4 <NA> 4 value1, value2
> str(x)
# 'data.frame': 4 obs. of 3 variables:
# $ col1: chr "a" "b" "c" NA
# $ col2: num 1 2 3 4
# $ col3:List of 4
# ..$ : chr "value1"
# ..$ : chr "value2"
# ..$ : chr "value1"
# ..$ : chr "value1" "value2"
Problem encountered after update
x <- data.frame((sapply(x, c)))
x <- purrr::map_df(x, function(x) sapply(x, function(x) unlist(x)))
# Error: Argument 1 must have names.
# Run `rlang::last_error()` to see where the error occurred.
# In addition: Warning message:
# Outer names are only allowed for unnamed scalar atomic inputs
> rlang::last_error()
# <error/rlang_error>
# Argument 1 must have names.
# Backtrace:
# 1. purrr::map_df(x, function(x) sapply(x, function(x) unlist(x)))
# 2. dplyr::bind_rows(res, .id = .id)
# Run `rlang::last_trace()` to see the full context.
This error seems to be well known, and I have explored many options with the purrr::flatten_() family, and others found on Stackoverflow but was not able to solve.
Thank you if any help is providable!
The first part of your attempt gives you a list for every column irrespective of it's length.
x <- data.frame((sapply(x, c)))
str(x)
#'data.frame': 4 obs. of 3 variables:
# $ col1:List of 4
# ..$ : chr "a"
# ..$ : chr "b"
# ..$ : chr "c"
# ..$ : logi NA
# $ col2:List of 4
# ..$ : num 1
# ..$ : num 2
# ..$ : num 3
# ..$ : num 4
# $ col3:List of 4
# ..$ : chr "value1"
# ..$ : chr "value2"
# ..$ : chr "value1"
# ..$ : chr "value1" "value2"
You can unlist the above for columns with only 1 element.
x[] <- lapply(x, function(p) if(max(lengths(p)) == 1) unlist(p) else p)
x
# col1 col2 col3
#1 a 1 value1
#2 b 2 value2
#3 c 3 value1
#4 <NA> 4 value1, value2
str(x)
#'data.frame': 4 obs. of 3 variables:
# $ col1: chr "a" "b" "c" NA
# $ col2: num 1 2 3 4
# $ col3:List of 4
# ..$ : chr "value1"
# ..$ : chr "value2"
# ..$ : chr "value1"
# ..$ : chr "value1" "value2"
One option utilizing dplyr, tibble and purrr could be:
imap_dfc(x, ~ tibble(!!.y := .x)) %>%
mutate(across(where(~ all(lengths(.) == 1)), ~ unlist(.)))
col1 col2 col3
<chr> <dbl> <list>
1 a 1 <chr [1]>
2 b 2 <chr [1]>
3 c 3 <chr [1]>
4 <NA> 4 <chr [2]>
no tidyverse solution, but it seems to work..
library( rlist )
as.data.frame( rlist::list.cbind( x ) )
# col1 col2 col3
# 1 a 1 value1
# 2 b 2 value2
# 3 c 3 value1
# 4 NA 4 value1, value2

Why are my data.frame columns lists?

I have a data.frame
'data.frame': 4 obs. of 2 variables:
$ name:List of 4
..$ : chr "a"
..$ : chr "b"
..$ : chr "c"
..$ : chr "d"
$ tvd :List of 4
..$ : num 0.149
..$ : num 0.188
..$ : num 0.161
..$ : num 0.187
structure(list(name = list("a", "b", "c",
"d"), tvd = list(0.148831029536996, 0.187699857380692,
0.161428147003292, 0.18652668961466)), .Names = c("name",
"tvd"), row.names = c(NA, -4L), class = "data.frame")
It appears that as.data.frame(lapply(z,unlist)) converts it to the usual
'data.frame': 4 obs. of 2 variables:
$ name: Factor w/ 4 levels "a",..: 4 1 2 3
$ tvd : num 0.149 0.188 0.161 0.187
However, I wonder if I could do better.
I create my ugly data frame like this:
as.data.frame(do.call(rbind,lapply(my.list, function (m)
list(name = ...,
tvd = ...))))
I wonder if it is possible to modify this expressing so that it would produce the normal data table.
It looks like you're just trying to tear down your original data then re-assemble it? If so, here are a few cool things to look at. Assume df is your data.
A data.frame is just a list in disguise. To see this, compare df[[1]] to df$name in your data. [[ is used for list indexing, as well as $. So we are actually viewing a list item when we use df$name on a data frame.
> is.data.frame(df) # df is a data frame
# [1] TRUE
> is.list(df) # and it's also a list
# [1] TRUE
> x <- as.list(df) # as.list() can be more useful than unlist() sometimes
# take a look at x here, it's a bit long
> (y <- do.call(cbind, x)) # reassemble to matrix form
# name tvd
# [1,] "a" 0.148831
# [2,] "b" 0.1876999
# [3,] "c" 0.1614281
# [4,] "d" 0.1865267
> as.data.frame(y) # back to df
# name tvd
# 1 a 0.148831
# 2 b 0.1876999
# 3 c 0.1614281
# 4 d 0.1865267
I recommend doing
do.call(rbind,lapply(my.list, function (m)
data.frame(name = ...,
tvd = ...)))
rather than trying to convert a list of lists into a data.frame

Resources