Scope of Aggregation Functions when nesting apply(within()) - r

Question

Edited original post to clarify question
Background
I'm learning R and saw this scenario and don't understand how R handles (what I'll call) implied context transitions. The script I am trying to understand simply iterates through each row of a matrix and prints the index of the column(s) within that row that contain the minimum value of that row. What I don't understand is how R handles the context transition as different functions are applied to the dependent variable x:
x (when defined as an argument to function(x)) is an atomic vector because of the apply() function with a MARGIN = 1 argument
The which() function then iterates over the individual elements within the atomic vector x to see which ones == min(x)
This is the part that truly confuses me: Despite the fact which() is iterating over elements of atomic vector x, you can call min(x) within the which() function and R somehow switches x to be defined as the entire atomic vector again for calculating the min() across the vector vs. within the scope of a single element
Example Data Matrix
a <- matrix (c(5, 2, 7, 1, 2, 8, 4, 5, 6), 3, 3)
[,1] [,2] [,3]
[1,] 5 1 4
[2,] 2 2 5
[3,] 7 8 6
This is the script that returns the column indexes that I am struggling to understand
apply (a, 1, function(x) which(x == min(x)))
My question:
Within the which() function, why does min(x) return the minimum of the atomic vector (as is desired) and not the minimum within the scope of an individual element within that vector, since which() is iterating over each individual element within the atomic vector x?

Answer 1

Edit: discussion about which and x:
the first comment on your question is incorrect:
x is anonymous function, lambda
x is just a variable, nothing fancy. function(x) declares it as the first (and only) argument of the anonymous function, and then every reference to x after that is referencing what is passed to this anonymous function;
the code uses an anonymous function; normally, almost everything you do in R is using named functions (e.g., mean, min). In some cases (e.g., in apply and related functions), it makes sense to define a whole function as an argument and not name it, as in
## anonymous (unnamed) function
apply(m, 1, function(x) which(x == min(x)))
## equivalently, with a named function
myfunc <- function(x) which(x == min(x))
apply(m, 1, myfunc)
In the first case, function(x) which(x == min(x))) is not named, so it is "anonymous". The results between the two apply calls are identical.
Given that context, x is the first argument to the function (myfunc or the anonymous function in your case). With the rest of the apply/MARGIN discussion below,
x (in this case) contains the whole row (when MARGIN=1);
min(x) returns the value of the lowest value within x, and it is always length 1); and
which(x == min(x)) returns the index of that lowest value within x; in this case, it will always be length 1 or more, because we are confident that there is always one element such that it is equal to the minimum of that vector ... however, there is no guarantee that which will find any matches, so the length of which(...)'s return value can be between 0 and the length of the inputs. Examples:
which(11:15 == 13)
# [1] 3
which(11:15 == 1:5)
# integer(0)
which(11:15 == 11:15)
# [1] 1 2 3 4 5
which(11:15 %in% c(12, 14))
# [1] 2 4
apply works one or more dimensions at a time. For now, I'll stick with a 2d matrix, in which case MARGIN= selects rows or columns. (There is a caveat, see below.)
I'm going to use a step-by-step verbose function for trying to show each step. I'll name it anonfunc, but in your mind convert apply(a, 1, anonfunc) later with apply(a, 1, function(x) { ... }) and you will see what I'm intending to do. Also, I have a dematrix function to help show what's being used in the anonfunc.
dematrix <- function(m, label = "") {
if (!is.matrix(m)) m <- matrix(m, nrow = 1)
out <- capture.output(print(m))[-1]
out <- gsub("^[][,0-9]+", "", out)
paste(paste0(c(label, rep(strrep(" ", nchar(label)), length(out) - 1)), out),
collapse = "\n")
}
anonfunc <- function(x) {
message(dematrix(x, "Input: "))
step1 <- x == min(x)
message(dematrix(step1, "Step1: "))
step2 <- which(step1)
message("Step2: ", paste(step2, collapse = ","), "\n#\n")
step2
}
2d arrays
I'm going to modify your sample data a little by adding a column. This helps visualize how many function calls there are and how big the function's input is.
apply(a, 1, anonfunc)
# Input: 5 1 4 11
# Step1: FALSE TRUE FALSE FALSE
# Step2: 2
# #
# Input: 2 2 5 12
# Step1: TRUE TRUE FALSE FALSE
# Step2: 1,2
# #
# Input: 7 8 6 13
# Step1: FALSE FALSE TRUE FALSE
# Step2: 3
# #
# [[1]]
# [1] 2
# [[2]]
# [1] 1 2
# [[3]]
# [1] 3
Our anonymous function is called three times, once for each row. In each call, it is passed a vector of length 4, which is the size of one row in the matrix.
Note that we get a list in return. Normally apply returns a vector or matrix. The return value is actually the dimension of the MARGIN= axes, with an added dimension of the length of the return values. That is, a has dims 3x4; if the return value from each call to the anon-func is length 1, then the return value is "sort of" 3x1, but R simplifies that to a vector of length 3 (this might be construed as inconsistent mathematically, I don't disagree).; if the return value from each anon-func call is length 10, then the output would be a matrix of 3x10.
However, when any of the anon-func returns is of a different length/size/class as the others, then apply will return a list. (This is the same behavior as sapply, and it can be frustrating if it changes when you are not expecting it. There is allegedly a patch in R-devel that allows us to force a list with apply(..., simplify=FALSE).)
If we instead use MARGIN=2, we'll be operating on columns:
apply(a, 2, anonfunc)
# Input: 5 2 7
# Step1: FALSE TRUE FALSE
# Step2: 2
# #
# Input: 1 2 8
# Step1: TRUE FALSE FALSE
# Step2: 1
# #
# Input: 4 5 6
# Step1: TRUE FALSE FALSE
# Step2: 1
# #
# Input: 11 12 13
# Step1: TRUE FALSE FALSE
# Step2: 1
# #
# [1] 2 1 1 1
Now, one call for each column (4 calls) and x is a vector of length 3 (number of rows in the source matrix).
It is possible to operate on more than one axis at a time; while it seems meaningless to do it with a matrix (2d array), it makes more sense with larger-dimensioned arrays.
apply(a, 1:2, anonfunc)
# Input: 5
# Step1: TRUE
# Step2: 1
# #
# Input: 2
# Step1: TRUE
# Step2: 1
# #
# Input: 7
# Step1: TRUE
# Step2: 1
# #
# ...truncated... total of 12 calls to `anonfunc`
# #
# [,1] [,2] [,3] [,4]
# [1,] 1 1 1 1
# [2,] 1 1 1 1
# [3,] 1 1 1 1
From the discussion of output dimensions, the MARGIN=1:2 means the output dimension will be the dimensions of the margin -- 3x4 -- with the dimension/length of the output. Since the output here is always length 1, then that is technically 3x4x1, which in R-speak is a matrix of dim 3x4.
Pics of what each margin uses from a matrix:
3d array
Let's go slightly larger to see some of the "plane" operations.
a3 <- array(1:24, dim = c(3,4,2))
a3
# , , 1
# [,1] [,2] [,3] [,4]
# [1,] 1 4 7 10
# [2,] 2 5 8 11
# [3,] 3 6 9 12
# , , 2
# [,1] [,2] [,3] [,4]
# [1,] 13 16 19 22
# [2,] 14 17 20 23
# [3,] 15 18 21 24
Starting with MARGIN=1. While you have both arrays visible, look at the first Input: and see which "plane" is being used from the original a3 array. It appears transposed, sure ...
For the sake of brevity (too late!), I'll abbreviate the third and subsequent iterations of anonfunc to show just the first line (inner-matrix row) of the verbose output.
apply(a3, 1, anonfunc)
# Input: 1 13
# 4 16
# 7 19
# 10 22
# Step1: TRUE FALSE
# FALSE FALSE
# FALSE FALSE
# FALSE FALSE
# Step2: 1
# #
# Input: 2 14
# 5 17
# 8 20
# 11 23
# Step1: TRUE FALSE
# FALSE FALSE
# FALSE FALSE
# FALSE FALSE
# Step2: 1
# #
# Input: 3 15 ...
# #
# [1] 1 1 1
Similarly, MARGIN=2. I'll show a3 again so you can see which "plane" is being used:
a3
# , , 1
# [,1] [,2] [,3] [,4]
# [1,] 1 4 7 10
# [2,] 2 5 8 11
# [3,] 3 6 9 12
# , , 2
# [,1] [,2] [,3] [,4]
# [1,] 13 16 19 22
# [2,] 14 17 20 23
# [3,] 15 18 21 24
apply(a3, 2, anonfunc)
# Input: 1 13
# 2 14
# 3 15
# Step1: TRUE FALSE
# FALSE FALSE
# FALSE FALSE
# Step2: 1
# #
# Input: 4 16
# 5 17
# 6 18
# Step1: TRUE FALSE
# FALSE FALSE
# FALSE FALSE
# Step2: 1
# #
# Input: 7 19 ...
# Input: 10 22 ...
# #
# [1] 1 1 1 1
MARGIN=3 is not very exciting: anonfunc is only called twice, one for each of the front-facing "planes" (no abbreviation necessary here):
apply(a3, 3, anonfunc)
# Input: 1 4 7 10
# 2 5 8 11
# 3 6 9 12
# Step1: TRUE FALSE FALSE FALSE
# FALSE FALSE FALSE FALSE
# FALSE FALSE FALSE FALSE
# Step2: 1
# #
# Input: 13 16 19 22
# 14 17 20 23
# 15 18 21 24
# Step1: TRUE FALSE FALSE FALSE
# FALSE FALSE FALSE FALSE
# FALSE FALSE FALSE FALSE
# Step2: 1
# #
# [1] 1 1
One can use multiple dimensions here as well, and this is where I think the Input: string becomes a little clarifying:
a3
# , , 1
# [,1] [,2] [,3] [,4]
# [1,] 1 4 7 10
# [2,] 2 5 8 11
# [3,] 3 6 9 12
# , , 2
# [,1] [,2] [,3] [,4]
# [1,] 13 16 19 22
# [2,] 14 17 20 23
# [3,] 15 18 21 24
apply(a3, 2:3, anonfunc)
# Input: 1 2 3
# Step1: TRUE FALSE FALSE
# Step2: 1
# #
# Input: 4 5 6
# Step1: TRUE FALSE FALSE
# Step2: 1
# #
# Input: 7 8 9 ...
# Input: 10 11 12 ...
# Input: 13 14 15 ...
# Input: 16 17 18 ...
# Input: 19 20 21 ...
# Input: 22 23 24 ...
# #
# [,1] [,2]
# [1,] 1 1
# [2,] 1 1
# [3,] 1 1
# [4,] 1 1
And since the dimensions of a3 are 3,4,2, and we're looking at margins 2:3, and each call to anonfunc returns length 1, our returned matrix is 4x2x1 (where the x1 is silently dropped by R).
To visualize what each call of MARGIN= actually uses, see the below pics:

Answer 2

"Lexical scoping looks up symbol values based on how functions were nested when they were created, not how they are nested when they are called. With lexical scoping, you don’t need to know how the function is called to figure out where the value of a variable will be looked up. You just need to look at the function’s definition."**
**Source: http://adv-r.had.co.nz/Functions.html#lexical-scoping