Easiest way to subset within a multidimensional range? - r

Question

I apologize in advance if this has been asked before, or if I have missed something obvious.
I have two data sets, 'olddata' and 'newdata'
set.seed(0)
olddata <- data.frame(x = rnorm(10, 0,5), y = runif(10, 0, 5), z = runif(10,-10,10))
newdata <- data.frame(x = -5:5, z = -5:5)
I create a model from the old data, and want to predict values from the new data
mymodel <- lm(y ~ x+z, data = olddata)
predict.lm(mymodel, newdata)
However, I'd like to restrict the range of variables in 'newdata' to the range of variables on which the model was trained.
of course I could do this:
newnewdata <- subset(newdata,
x < max(olddata$x) & x > min(olddata$x) &
z < max(olddata$z) & z > max(olddata$z))
But this gets intractable over many dimensions. Is there a less repetitive way to do this?

Answer 1

It seems that all the values in your newdata are already within the appropriate ranges, so there's nothing there to subset. If we expand the ranges of newdata:
set.seed(0)
olddata <- data.frame(x = rnorm(10, 0,5), y = runif(10, 0, 5), z = runif(10,-10,10))
newdata <- data.frame(x = -10:10, z = -10:10)
newdata
x z
1 -10 -10
2 -9 -9
3 -8 -8
4 -7 -7
5 -6 -6
6 -5 -5
7 -4 -4
8 -3 -3
9 -2 -2
10 -1 -1
11 0 0
12 1 1
13 2 2
14 3 3
15 4 4
16 5 5
17 6 6
18 7 7
19 8 8
20 9 9
21 10 10
Then all we need to do is identify the ranges for each variable of olddata and then loop through as many iterations of subset as newdata has columns:
ranges <- sapply(olddata, range, na.rm = TRUE)
for(i in 1:ncol(newdata)) {
col_name <- colnames(newdata)[i]
newdata <- subset(newdata,
newdata[,col_name] >= ranges[1, col_name] &
newdata[,col_name] <= ranges[2, col_name])
}
newdata
x z
4 -7 -7
5 -6 -6
6 -5 -5
7 -4 -4
8 -3 -3
9 -2 -2
10 -1 -1
11 0 0
12 1 1
13 2 2
14 3 3
15 4 4
16 5 5
17 6 6

Answer 2

Here is an approach using the *apply family (using SchaunW's newdata):
set.seed(0)
olddata <- data.frame(x = rnorm(10, 0, 5), y = runif(10, 0, 5), z = runif(10,-10,10))
newdata <- data.frame(x = -10:10, z = -10:10)
minmax <- sapply(olddata[-2], range)
newdata[apply(newdata, 1, function(a) all(a > minmax[1,] & a < minmax[2,])), ]
Some care is required because I have assumed the columns of olddata (after dropping the second column) are identical to newdata.
Brevity comes at the cost of speed. After increasing nrow(newdata) to 2000 to emphasis the difference I found:
test replications elapsed relative user.self sys.self user.child sys.child
1 orizon() 100 2.193 27.759 2.191 0.002 0 0
2 SchaunW() 100 0.079 1.000 0.075 0.004 0 0
My guess at the main cause is that repeated subsetting avoids testing whether rows meet the criteria examined after they are excluded.