R provides two different methods for accessing the elements of a list or data.frame: [] and [[]].
What is the difference between the two, and when should I use one over the other?
The R Language Definition is handy for answering these types of questions:
http://cran.r-project.org/doc/manuals/R-lang.html#Indexing
R has three basic indexing operators, with syntax displayed by the following examples
x[i]
x[i, j]
x[[i]]
x[[i, j]]
x$a
x$"a"
For vectors and matrices the [[ forms are rarely used, although they have some slight semantic differences from the [ form (e.g. it drops any names or dimnames attribute, and that partial matching is used for character indices). When indexing multi-dimensional structures with a single index, x[[i]] or x[i] will return the ith sequential element of x.
For lists, one generally uses [[ to select any single element, whereas [ returns a list of the selected elements.
The [[ form allows only a single element to be selected using integer or character indices, whereas [ allows indexing by vectors. Note though that for a list, the index can be a vector and each element of the vector is applied in turn to the list, the selected component, the selected component of that component, and so on. The result is still a single element.
The significant differences between the two methods are the class of the objects they return when used for extraction and whether they may accept a range of values, or just a single value during assignment.
Consider the case of data extraction on the following list:
foo <- list( str='R', vec=c(1,2,3), bool=TRUE )
Say we would like to extract the value stored by bool from foo and use it inside an if() statement. This will illustrate the differences between the return values of [] and [[]] when they are used for data extraction. The [] method returns objects of class list (or data.frame if foo was a data.frame) while the [[]] method returns objects whose class is determined by the type of their values.
So, using the [] method results in the following:
if( foo[ 'bool' ] ){ print("Hi!") }
Error in if (foo["bool"]) { : argument is not interpretable as logical
class( foo[ 'bool' ] )
[1] "list"
This is because the [] method returned a list and a list is not valid object to pass directly into an if() statement. In this case we need to use [[]] because it will return the "bare" object stored in 'bool' which will have the appropriate class:
if( foo[[ 'bool' ]] ){ print("Hi!") }
[1] "Hi!"
class( foo[[ 'bool' ]] )
[1] "logical"
The second difference is that the [] operator may be used to access a range of slots in a list or columns in a data frame while the [[]] operator is limited to accessing a single slot or column. Consider the case of value assignment using a second list, bar():
bar <- list( mat=matrix(0,nrow=2,ncol=2), rand=rnorm(1) )
Say we want to overwrite the last two slots of foo with the data contained in bar. If we try to use the [[]] operator, this is what happens:
foo[[ 2:3 ]] <- bar
Error in foo[[2:3]] <- bar :
more elements supplied than there are to replace
This is because [[]] is limited to accessing a single element. We need to use []:
foo[ 2:3 ] <- bar
print( foo )
$str
[1] "R"
$vec
[,1] [,2]
[1,] 0 0
[2,] 0 0
$bool
[1] -0.6291121
Note that while the assignment was successful, the slots in foo kept their original names.
Double brackets accesses a list element, while a single bracket gives you back a list with a single element.
lst <- list('one','two','three')
a <- lst[1]
class(a)
## returns "list"
a <- lst[[1]]
class(a)
## returns "character"
From Hadley Wickham:
My (crappy looking) modification to show using tidyverse / purrr:
[] extracts a list, [[]] extracts elements within the list
alist <- list(c("a", "b", "c"), c(1,2,3,4), c(8e6, 5.2e9, -9.3e7))
str(alist[[1]])
chr [1:3] "a" "b" "c"
str(alist[1])
List of 1
$ : chr [1:3] "a" "b" "c"
str(alist[[1]][1])
chr "a"
Just adding here that [[ also is equipped for recursive indexing.
This was hinted at in the answer by #JijoMatthew but not explored.
As noted in ?"[[", syntax like x[[y]], where length(y) > 1, is interpreted as:
x[[ y[1] ]][[ y[2] ]][[ y[3] ]] ... [[ y[length(y)] ]]
Note that this doesn't change what should be your main takeaway on the difference between [ and [[ -- namely, that the former is used for subsetting, and the latter is used for extracting single list elements.
For example,
x <- list(list(list(1), 2), list(list(list(3), 4), 5), 6)
x
# [[1]]
# [[1]][[1]]
# [[1]][[1]][[1]]
# [1] 1
#
# [[1]][[2]]
# [1] 2
#
# [[2]]
# [[2]][[1]]
# [[2]][[1]][[1]]
# [[2]][[1]][[1]][[1]]
# [1] 3
#
# [[2]][[1]][[2]]
# [1] 4
#
# [[2]][[2]]
# [1] 5
#
# [[3]]
# [1] 6
To get the value 3, we can do:
x[[c(2, 1, 1, 1)]]
# [1] 3
Getting back to #JijoMatthew's answer above, recall r:
r <- list(1:10, foo=1, far=2)
In particular, this explains the errors we tend to get when mis-using [[, namely:
r[[1:3]]
Error in r[[1:3]] : recursive indexing failed at level 2
Since this code actually tried to evaluate r[[1]][[2]][[3]], and the nesting of r stops at level one, the attempt to extract through recursive indexing failed at [[2]], i.e., at level 2.
Error in r[[c("foo", "far")]] : subscript out of bounds
Here, R was looking for r[["foo"]][["far"]], which doesn't exist, so we get the subscript out of bounds error.
It probably would be a bit more helpful/consistent if both of these errors gave the same message.
Being terminological, [[ operator extracts the element from a list whereas [ operator takes subset of a list.
To help newbies navigate through the manual fog, it might be helpful to see the [[ ... ]] notation as a collapsing function - in other words, it is when you just want to 'get the data' from a named vector, list or data frame. It is good to do this if you want to use data from these objects for calculations. These simple examples will illustrate.
(x <- c(x=1, y=2)); x[1]; x[[1]]
(x <- list(x=1, y=2, z=3)); x[1]; x[[1]]
(x <- data.frame(x=1, y=2, z=3)); x[1]; x[[1]]
So from the third example:
> 2 * x[1]
x
1 2
> 2 * x[[1]]
[1] 2
Both of them are ways of subsetting.
The single bracket will return a subset of the list, which in itself will be a list. i.e., It may or may not contain more than one elements.
On the other hand, a double bracket will return just a single element from the list.
-Single bracket will give us a list. We can also use single bracket if we wish to return multiple elements from the list.
Consider the following list:
>r<-list(c(1:10),foo=1,far=2);
Now, please note the way the list is returned when I try to display it.
I type r and press enter.
>r
#the result is:-
[[1]]
[1] 1 2 3 4 5 6 7 8 9 10
$foo
[1] 1
$far
[1] 2
Now we will see the magic of single bracket:
>r[c(1,2,3)]
#the above command will return a list with all three elements of the actual list r as below
[[1]]
[1] 1 2 3 4 5 6 7 8 9 10
$foo
[1] 1
$far
[1] 2
which is exactly the same as when we tried to display value of r on screen, which means the usage of single bracket has returned a list, where at index 1 we have a vector of 10 elements, then we have two more elements with names foo and far.
We may also choose to give a single index or element name as input to the single bracket.
e.g.,:
> r[1]
[[1]]
[1] 1 2 3 4 5 6 7 8 9 10
In this example, we gave one index "1" and in return got a list with one element(which is an array of 10 numbers)
> r[2]
$foo
[1] 1
In the above example, we gave one index "2" and in return got a list with one element:
> r["foo"];
$foo
[1] 1
In this example, we passed the name of one element and in return a list was returned with one element.
You may also pass a vector of element names like:
> x<-c("foo","far")
> r[x];
$foo
[1] 1
$far
[1] 2
In this example, we passed an vector with two element names "foo" and "far".
In return we got a list with two elements.
In short, a single bracket will always return you another list with number of elements equal to the number of elements or number of indices you pass into the single bracket.
In contrast, a double bracket will always return only one element.
Before moving to double bracket a note to be kept in mind.
NOTE:THE MAJOR DIFFERENCE BETWEEN THE TWO IS THAT SINGLE BRACKET RETURNS YOU A LIST WITH AS MANY ELEMENTS AS YOU WISH WHILE A DOUBLE BRACKET WILL NEVER RETURN A LIST. RATHER A DOUBLE BRACKET WILL RETURN ONLY A SINGLE ELEMENT FROM THE LIST.
I will site a few examples. Please keep a note of the words in bold and come back to it after you are done with the examples below:
Double bracket will return you the actual value at the index.(It will NOT return a list)
> r[[1]]
[1] 1 2 3 4 5 6 7 8 9 10
>r[["foo"]]
[1] 1
for double brackets if we try to view more than one elements by passing a vector it will result in an error just because it was not built to cater to that need, but just to return a single element.
Consider the following
> r[[c(1:3)]]
Error in r[[c(1:3)]] : recursive indexing failed at level 2
> r[[c(1,2,3)]]
Error in r[[c(1, 2, 3)]] : recursive indexing failed at level 2
> r[[c("foo","far")]]
Error in r[[c("foo", "far")]] : subscript out of bounds
For yet another concrete use case, use double brackets when you want to select a data frame created by the split() function. If you don't know, split() groups a list/data frame into subsets based on a key field. It's useful if when you want to operate on multiple groups, plot them, etc.
> class(data)
[1] "data.frame"
> dsplit<-split(data, data$id)
> class(dsplit)
[1] "list"
> class(dsplit['ID-1'])
[1] "list"
> class(dsplit[['ID-1']])
[1] "data.frame"
Please refer the below-detailed explanation.
I have used Built-in data frame in R, called mtcars.
> mtcars
mpg cyl disp hp drat wt ...
Mazda RX4 21.0 6 160 110 3.90 2.62 ...
Mazda RX4 Wag 21.0 6 160 110 3.90 2.88 ...
Datsun 710 22.8 4 108 93 3.85 2.32 ...
............
The top line of the table is called the header which contains the column names. Each horizontal line afterward denotes a data row, which begins with the name of the row, and then followed by the actual data.
Each data member of a row is called a cell.
single square bracket "[]" operator
To retrieve data in a cell, we would enter its row and column coordinates in the single square bracket "[]" operator. The two coordinates are separated by a comma. In other words, the coordinates begin with row position, then followed by a comma, and ends with the column position. The order is important.
Eg 1:- Here is the cell value from the first row, second column of mtcars.
> mtcars[1, 2]
[1] 6
Eg 2:- Furthermore, we can use the row and column names instead of the numeric coordinates.
> mtcars["Mazda RX4", "cyl"]
[1] 6
Double square bracket "[[]]" operator
We reference a data frame column with the double square bracket "[[]]" operator.
Eg 1:- To retrieve the ninth column vector of the built-in data set mtcars, we write mtcars[[9]].
mtcars[[9]]
[1] 1 1 1 0 0 0 0 0 0 0 0 ...
Eg 2:- We can retrieve the same column vector by its name.
mtcars[["am"]]
[1] 1 1 1 0 0 0 0 0 0 0 0 ...
I've seen the function lapply used in R to extract elements from matrices that exist in a list of matrices.
E.g. I have a list of 3 (2x2) matrices, and I want to extract element [1,2] from each of those 3 matrices.
The code: list1 = lapply(mylist, '[', 1,2) works just fine. It returns a list with those 3 elements.
I am trying to research what this is exactly doing. Google hasn't helped and using ?'[' in the R help isn't too explanatory. I don't see how '[' is a function in R, so the code is not intuitive.
The square brackets are in fact a function whose first argument is the object being subsetted. Subsequent arguments are the index to that subset.
# For example, if M is a matrix
M[1, 2] # extracts the element at row 1, col 2
# is the same as
`[`(M, 1, 2)
# Try them!
Now, Have a look at the arguments to lapply:
args(lapply)
# function (X, FUN, ...)
Everything represented in those dots gets passed on to the function FUN as arguments.
Thus, when FUN="[", the first argument to "[" is the current element of the list (being iterated over), ie, the object being subsetted. While the subsequent arguments are the indexes to "["
Operators in R are just functions.
These are equivalent:
> x <- list(a=1,b=2)
> x[1]
$a
[1] 1
> `[`(x,1)
$a
[1] 1
The backticks are necessary only to prevent interpretation by the parser (e.g. to tell it it's a function name not to start interpreting the [ prematurely).
Being a function, it follows the same object-oriented rules (in this case, S3) as everything else.
> methods(`[`)
[1] [.acf* [.arrow* [.AsIs [.bibentry* [.cluster* [.data.frame [.data.table*
[8] [.Date [.difftime [.envlist* [.factor [.formula* [.fractions* [.getAnywhere*
[15] [.gList* [.gpar* [.gtable* [.hexmode [.idf* [.indexed* [.insensitive*
[22] [.ITime* [.listof [.noquote [.numeric_version [.octmode [.pdf_doc* [.person*
[29] [.POSIXct [.POSIXlt [.quoted* [.raster* [.roman* [.shingle* [.simple.list
[36] [.split* [.terms* [.trellis* [.ts* [.tskernel* [.uneval* [.unit*
[43] [.unit.arithmetic* [.unit.list* [.vpPath*
Non-visible functions are asterisked
+, =, etc. and other operators all work this way as well.
I have the following list
test_list=list(list(a=1,b=2),list(a=3,b=4))
and I want to extract all elements with list element name a.
I can do this via
sapply(test_list,`[[`,"a")
which gives me the correct result
#[1] 1 3
When I try the same with Rs dollar operator $, I get NULL
sapply(test_list,`$`,"a")
#[[1]]
#NULL
#
#[[2]]
#NULL
However, if I use it on a single element of test_list it works as expected
`$`(test_list[[1]],"a")
#[1] 1
Am I missing something obvious here?
evaluation vs. none
[[ evaluates its argument whereas $ does not. L[[a]] gets the component of L whose name is held in the variable a. $ just passes the argument name itself as a character string so L$a finds the "a" component of L. a is not regarded as a variable holding the component name -- just a character string.
Below L[[b]] returns the component of L named "a" because the variable b has the value "a" whereas L$b returns the componet of L named "b" because with that syntax b is not regarded as a variable but is regarded as a character string which itself is passed.
L <- list(a = 1, b = 2)
b <- "a"
L[[b]] # same as L[["a"]] since b holds a
## [1] 1
L$b # same as L[["b"]] since b is regarded as a character string to be passed
## [1] 2
sapply
Now that we understand the key difference bewteen $ and [[ to see what is going on with sapply consider this example. We have made each element of test_list into a "foo" object and defined our own $.foo and [[.foo methods which simply show what R is passing to the method via the name argument:
foo_list <- test_list
class(foo_list[[1]]) <- class(foo_list[[2]]) <- "foo"
"$.foo" <- "[[.foo" <- function(x, name) print(name)
result <- sapply(foo_list, "$", "a")
## "..."
## "..."
result2 <- sapply(foo_list, "[[", "a")
## [1] "a"
## [1] "a"
What is happening in the first case is that sapply is calling whatever$... and ... is not evaluated so it would be looking for a list component which is literally named "..." and, of course, there is no such component so whatever$... is NULL hence the NULLs shown in the output in the question. In the second case whatever[[[...]] evaluates to whatever[["a"]] hence the observed result.
From what I've been able to determine it's a combination of two things.
First, the second element of $ is matched but not evaluated so it cannot be a variable.
Secondly, when arguments are passed to functions they are assigned to the corresponding variables in the function call. When passed to sapply "a" is assigned to a variable and therefore will no longer work with $. We can see this by occurring by running
sapply("a", print)
[1] "a"
a
"a"
This can lead to peculiar results like this
sapply(test_list, function(x, a) {`$`(x, a)})
[1] 1 3
Where despite a being a variable (which hasn't even been assigned) $ matches it to the names of the elements in the list.
Consider the following situation where I have a list of n matrices (this is just dummy data in the example below) in the object myList
mat <- matrix(1:12, ncol = 3)
myList <- list(mat1 = mat, mat2 = mat, mat3 = mat, mat4 = mat)
I want to select a specific column from each of the matrices and do something with it. This will get me the first column of each matrix and return it as a matrix (lapply() would give me a list either is fine).
sapply(myList, function(x) x[, 1])
What I can't seem able to do is use [ directly as a function in my sapply() or lapply() incantations. ?'[' tells me that I need to supply argument j as the column identifier. So what am I doing wrong that this does't work?
> lapply(myList, `[`, j = 1)
$mat1
[1] 1
$mat2
[1] 1
$mat3
[1] 1
$mat4
[1] 1
Where I would expect this:
$mat1
[1] 1 2 3 4
$mat2
[1] 1 2 3 4
$mat3
[1] 1 2 3 4
$mat4
[1] 1 2 3 4
I suspect I am getting the wrong [ method but I can't work out why? Thoughts?
I think you are getting the 1 argument form of [. If you do lapply(myList, `[`, i =, j = 1) it works.
After two pints of Britain's finest ale and a bit of cogitation, I realise that this version will work:
lapply(myList, `[`, , 1)
i.e. don't name anything and treat it like I had done mat[ ,1]. Still don't grep why naming j doesn't work...
...actually, having read ?'[' more closely, I notice the following section:
Argument matching:
Note that these operations do not match their index arguments in
the standard way: argument names are ignored and positional
matching only is used. So ‘m[j=2,i=1]’ is equivalent to ‘m[2,1]’
and *not* to ‘m[1,2]’.
And that explains my quandary above. Yeah for actually reading the documentation.
It's because [ is a .Primitive function. It has no j argument. And there is no [.matrix method.
> `[`
.Primitive("[")
> args(`[`)
NULL
> methods(`[`)
[1] [.acf* [.AsIs [.bibentry* [.data.frame
[5] [.Date [.difftime [.factor [.formula*
[9] [.getAnywhere* [.hexmode [.listof [.noquote
[13] [.numeric_version [.octmode [.person* [.POSIXct
[17] [.POSIXlt [.raster* [.roman* [.SavedPlots*
[21] [.simple.list [.terms* [.ts* [.tskernel*
Though this really just begs the question of how [ is being dispatched on matrix objects...