I need to find the optimal weight (a) to minimize sum of squared errors. I use this as an example as I'll have a more complex problem to work on which requires the use of some other optimization packages but lm. I receive warnings() saying that "In a * y. Recycling array of length 1 in array-vector arithmetic is deprecated. Use c() or as.vector() instead." I tried as.vector(a * y), a[1] * y, a * as.vector(y) but nothing works.
What should I do to get rid of this message?
install.packages("NlcOptim")
library(NlcOptim)
x <- c(1:4)
y <- c(2,4,6,8)
objfun <- function(a) {
return(sum((x-a*y)^2))
}
x0 <- 1
solnl(x0,objfun = objfun)
Here is the error message that I see after typing warnings():
1: In a * y :
Recycling array of length 1 in array-vector arithmetic is deprecated.
Use c() or as.vector() instead.
Actually I would suggest to do nothing about it. R simply does not like to adding one dimensional arrays to numeric vectors.
x <- array(1, dim = 1);
x + c(1,1)
[1] 2 2
Warning message:
In x + c(1, 1) :
Recycling array of length 1 in array-vector arithmetic is deprecated.
Use c() or as.vector() instead.
As you can see, the result is correct, so you are pretty safe to ignore that warning.
A bit late, but found your question when I tumbled into the same warning message. In your case, I think you need to treat a as a vector with c(a) or as.vector(a) inside your function objfun. (So you almost got it with your tries as.vector(a * y), a[1] * y and a * as.vector(y)).
Thus:
install.packages("NlcOptim")
library(NlcOptim)
x <- c(1:4)
y <- c(2,4,6,8)
objfun <- function(a) {
return(sum((x-c(a)*y)^2))
}
x0 <- 1
solnl(x0,objfun = objfun)
Running this code we obtain
$par
[,1]
[1,] 0.5
$fn
[1] 5.913821e-15
$counts
nfval ngval
[1,] 13 3
$lambda
$lambda$lower
[,1]
[1,] 0
$lambda$upper
[,1]
[1,] 0
$grad
[,1]
[1,] 1.033147e-07
$hessian
[,1]
[1,] 240
Warning message:
In cbind(LAMBDA, 0.5 * (LAMBDA + LAMBDA_old)) :
number of rows of result is not a multiple of vector length (arg 1)
I don't know what the last warning means, but it is into the package you are using.
This is confusing indeed, but what you really need to do is, in fact, use as.numeric, like this:
return(sum((x-as.numeric(a)*y)^2))
doing R's job of interpreting a 1x1 array as a scalar. Not sure how as.vector() or c() would have ever been helpful.
I am trying to implement Kernel Ridge Regression in R.
The formula is:
alpha <- ((lambda.I + K)^(-1)) * y
Lambda = 0.1. I = identity matrix the same size as K. y is a feature vector that has the same number of rows as K.
So I tried this in R:
I <- diag(nrow(df_matrix)
lambda <- 0.1
alpha <- (lambda * I + df_matrix) ^ (-1) * df_vector
I get the following error
Error in (0.1 * I + df_matrix)^(-1) * df_vector : non-conformable arrays
Here's some information on my dataset
> nrow(df_matrix)
[1] 8222
> ncol(df_matrix)
[1] 8222
> nrow(df_vector)
[1] 8222
> nrow(I)
[1] 8222
> ncol(I)
[1] 8222
> class(df_matrix)
[1] "matrix"
> class(df_vector)
[1] "matrix"
I bet you want to have here matrix inversion, which is solve(m), instead of element-wise (m^(-1)). Also, matrix multiplication (%*%) instead of element-wise (*). So, altogether is
alpha <- solve(lambda * I + df_matrix) %*% df_vector
You need to use matrix multiplication, %*%. In addition you also need to use solve to compute inverses as raising to the power minus one will just do element wise reciprocals. e.g.:
K <- matrix(runif(9),3)
y <- matrix(runif(3),nrow=3)
solve(lambda*diag(nrow(K))+K) %*% y
[,1]
[1,] 0.50035075
[2,] -0.04985508
[3,] 0.74944867
To transpose a matrix, the matrix has to be quadratic and the determinant has to be different to zero. If you matrix df_matrix fullfil these requirements, then
alpha <- solve(lambda * I + df_matrix) %*% df_vector
I have an R loop that has been giving me error. Here are the dimensions of the matrices..
> dim(A)
[1] 2 2
> dim(backward)
[1] 6848 2
I am trying to run this loop and get the following error:
for (i in t:1){
backward[i,]=A%*%t(backward[i,])}
Error in A %*% t(backward[i, ]) : non-conformable arguments
Where t equals 6848. Thanks for your time.
EDIT with bgoldst code:
> A
[,1] [,2]
[1,] 0.8 0.2
[2,] 0.2 0.8
> backward <- matrix(1:(t*2),t,2);
> dim(backward)
[1] 6848 2
> for (i in t:1) backward[i,] <- A%*%t(backward[i,,drop=F]);
Error in A %*% t(backward[i, , drop = F]) : non-conformable arguments
I'm guessing that your expectation of
backward[i,]
is that it will return a 1x2 matrix, which you would be able to use as the operand of a matrix multiplication. This is incorrect. In R, when you specify a single index within a dimension of a matrix, then by default, R will "drop" that dimension. In the case of the above piece of code, the row dimension is dropped, and you end up with a vector, whose contents are taken from all columns along the indexed row. A vector is not a valid operand to a matrix multiplication.
You can solve this problem by providing the drop argument to the [ operation:
A <- matrix(1:(2*2),2,2);
backward <- matrix(1:(6848*2),6848,2);
t <- nrow(backward); for (i in t:1) backward[i,] <- A%*%t(backward[i,,drop=F]); ## no error
Here's a demo of the effect of drop=F:
backward[1,]
## [1] 20548 27398
backward[1,,drop=F]
## [,1] [,2]
## [1,] 20548 27398
See ?`[` for more info.
Here's a solution that doesn't depend on the drop=F argument:
for (i in t:1) backward[i,] <- A%*%t(matrix(backward[i,],1));
I tried norm, but I think it gives the wrong result. (the norm of c(1, 2, 3) is sqrt(1*1+2*2+3*3), but it returns 6..
x1 <- 1:3
norm(x1)
# Error in norm(x1) : 'A' must be a numeric matrix
norm(as.matrix(x1))
# [1] 6
as.matrix(x1)
# [,1]
# [1,] 1
# [2,] 2
# [3,] 3
norm(as.matrix(x1))
# [1] 6
Does anyone know what's the function to calculate the norm of a vector in R?
norm(c(1,1), type="2") # 1.414214
norm(c(1, 1, 1), type="2") # 1.732051
This is a trivial function to write yourself:
norm_vec <- function(x) sqrt(sum(x^2))
I was surprised that nobody had tried profiling the results for the above suggested methods, so I did that. I've used a random uniform function to generate a list and used that for repetition (Just a simple back of the envelop type of benchmark):
> uut <- lapply(1:100000, function(x) {runif(1000, min=-10^10, max=10^10)})
> norm_vec <- function(x) sqrt(sum(x^2))
> norm_vec2 <- function(x){sqrt(crossprod(x))}
>
> system.time(lapply(uut, norm_vec))
user system elapsed
0.58 0.00 0.58
> system.time(lapply(uut, norm_vec2))
user system elapsed
0.35 0.00 0.34
> system.time(lapply(uut, norm, type="2"))
user system elapsed
6.75 0.00 6.78
> system.time(lapply(lapply(uut, as.matrix), norm))
user system elapsed
2.70 0.00 2.73
It seems that taking the power and then sqrt manually is faster than the builtin norm for real values vectors at least. This is probably because norm internally does an SVD:
> norm
function (x, type = c("O", "I", "F", "M", "2"))
{
if (identical("2", type)) {
svd(x, nu = 0L, nv = 0L)$d[1L]
}
else .Internal(La_dlange(x, type))
}
and the SVD function internally converts the vector into a matrix, and does more complicated stuff:
> svd
function (x, nu = min(n, p), nv = min(n, p), LINPACK = FALSE)
{
x <- as.matrix(x)
...
EDIT (20 Oct 2019):
There have been some comments to point out the correctness issue which the above test case doesn't bring out:
> norm_vec(c(10^155))
[1] Inf
> norm(c(10^155), type="2")
[1] 1e+155
This happens because large numbers are considered as infinity in R:
> 10^309
[1] Inf
So, it looks like:
It seems that taking the power and then sqrt manually is faster than the builtin norm for real values vectors for small numbers.
How small? So that the sum of squares doesn't overflow.
norm(x, type = c("O", "I", "F", "M", "2"))
The default is "O".
"O", "o" or "1" specifies the one norm, (maximum absolute column sum);
"F" or "f" specifies the Frobenius norm (the Euclidean norm of x treated as if it were a vector);
norm(as.matrix(x1),"o")
The result is 6, same as norm(as.matrix(x1))
norm(as.matrix(x1),"f")
The result is sqrt(1*1+2*2+3*3)
So, norm(as.matrix(x1),"f") is answer.
We can also find the norm as :
Result<-sum(abs(x)^2)^(1/2)
OR Even You can also try as:
Result<-sqrt(t(x)%*%x)
Both will give the same answer
I'mma throw this out there too as an equivalent R expression
norm_vec(x) <- function(x){sqrt(crossprod(x))}
Don't confuse R's crossprod with a similarly named vector/cross product. That naming is known to cause confusion especially for those with a physics/mechanics background.
Answer for Euclidean length of a vector (k-norm) with scaling to avoid destructive underflow and overflow is
norm <- function(x, k) { max(abs(x))*(sum((abs(x)/max(abs(x)))^k))^(1/k) }
See below for explanation.
1. Euclidean length of a vector with no scaling:
norm() is a vector-valued function which computes the length of the vector. It takes two arguments such as the vector x of class matrix and the type of norm k of class integer.
norm <- function(x, k) {
# x = matrix with column vector and with dimensions mx1 or mxn
# k = type of norm with integer from 1 to +Inf
stopifnot(k >= 1) # check for the integer value of k greater than 0
stopifnot(length(k) == 1) # check for length of k to be 1. The variable k is not vectorized.
if(k == Inf) {
# infinity norm
return(apply(x, 2, function(vec) max(abs(vec)) ))
} else {
# k-norm
return(apply(x, 2, function(vec) (sum((abs(vec))^k))^(1/k) ))
}
}
x <- matrix(c(1,-2,3,-4)) # column matrix
sapply(c(1:4, Inf), function(k) norm(x = x, k = k))
# [1] 10.000000 5.477226 4.641589 4.337613 4.000000
1-norm (10.0) converges to infinity-norm (4.0).
k-norm is also called as "Euclidean norm in Euclidean n-dimensional space".
Note:
In the norm() function definition, for vectors with real components, the absolute values can be dropped in norm-2k or even indexed norms, where k >= 1.
If you are confused with the norm function definition, you can read each one individually as given below.
norm_1 <- function(x) sum(abs(x))
norm_2 <- function(x) (sum((abs(x))^2))^(1/2)
norm_3 <- function(x) (sum((abs(x))^3))^(1/3)
norm_4 <- function(x) (sum((abs(x))^4))^(1/4)
norm_k <- function(x) (sum((abs(x))^k))^(1/k)
norm_inf <- max(abs(x))
2. Euclidean length of a vector with scaling to avoid destructive overflow and underflow issues:
Note-2:
The only problem with this solution norm() is that it does not guard against overflow or underflow problems as alluded here and here.
Fortunately, someone had already solved this problem for 2-norm (euclidean length) in the blas (basic linear algebra subroutines) fortran library. A description of this problem can be found in the textbook of "Numerical Methods and Software by Kahaner, Moler and Nash" - Chapter-1, Section 1.3, page - 7-9.
The name of the fortran subroutine is dnrm2.f, which handles destructive overflow and underflow issues in the norm() by scaling with the maximum of the vector components. The destructive overflow and underflow problem arise due to radical operation in the norm() function.
I will show how to implement dnrm2.f in R below.
#1. find the maximum among components of vector-x
max_x <- max(x)
#2. scale or divide the components of vector by max_x
scaled_x <- x/max_x
#3. take square of the scaled vector-x
sq_scaled_x <- (scaled_x)^2
#4. sum the square of scaled vector-x
sum_sq_scaled_x <- sum(sq_scaled_x)
#5. take square root of sum_sq_scaled_x
rt_sum_sq_scaled_x <- sqrt(sum_sq_scaled_x)
#6. multiply the maximum of vector x with rt_sum_sq_scaled_x
max_x*rt_sum_sq_scaled_x
one-liner of the above 6-steps of dnrm2.f in R is:
# Euclidean length of vector - 2norm
max(x)*sqrt(sum((x/max(x))^2))
Lets try example vectors to compute 2-norm (see other solutions in this thread) for this problem.
x = c(-8e+299, -6e+299, 5e+299, -8e+298, -5e+299)
max(x)*sqrt(sum((x/max(x))^2))
# [1] 1.227355e+300
x <- (c(1,-2,3,-4))
max(x)*sqrt(sum((x/max(x))^2))
# [1] 5.477226
Therefore, the recommended way to implement a generalized solution for k-norm in R is that single line, which guard against the destructive overflow or underflow problems. To improve this one-liner, you can use a combination of norm() without scaling for a vector containing not-too-small or not-too-large components and knorm() with scaling for a vector with too-small or too-large components. Implementing scaling for all vectors results in too many calculations. I did not implement this improvement in knorm() given below.
# one-liner for k-norm - generalized form for all norms including infinity-norm:
max(abs(x))*(sum((abs(x)/max(abs(x)))^k))^(1/k)
# knorm() function using the above one-liner.
knorm <- function(x, k) {
# x = matrix with column vector and with dimensions mx1 or mxn
# k = type of norm with integer from 1 to +Inf
stopifnot(k >= 1) # check for the integer value of k greater than 0
stopifnot(length(k) == 1) # check for length of k to be 1. The variable k is not vectorized.
# covert elements of matrix to its absolute values
x <- abs(x)
if(k == Inf) { # infinity-norm
return(apply(x, 2, function(vec) max(vec)))
} else { # k-norm
return(apply(x, 2, function(vec) {
max_vec <- max(vec)
return(max_vec*(sum((vec/max_vec)^k))^(1/k))
}))
}
}
# 2-norm
x <- matrix(c(-8e+299, -6e+299, 5e+299, -8e+298, -5e+299))
sapply(2, function(k) knorm(x = x, k = k))
# [1] 1.227355e+300
# 1-norm, 2-norm, 3-norm, 4-norm, and infinity-norm
sapply(c(1:4, Inf), function(k) knorm(x = x, k = k))
# [1] 2.480000e+300 1.227355e+300 9.927854e+299 9.027789e+299 8.000000e+299
x <- matrix(c(1,-2,3,-4))
sapply(c(1:4, Inf), function(k) knorm(x = x, k = k))
# [1] 10.000000 5.477226 4.641589 4.337613 4.000000
x <- matrix(c(1,-2,3,-4, 0, -8e+299, -6e+299, 5e+299, -8e+298, -5e+299), nc = 2)
sapply(c(1:4, Inf), function(k) knorm(x = x, k = k))
# [,1] [,2] [,3] [,4] [,5]
# [1,] 1.00e+01 5.477226e+00 4.641589e+00 4.337613e+00 4e+00
# [2,] 2.48e+300 1.227355e+300 9.927854e+299 9.027789e+299 8e+299
If you have a data.frame or a data.table 'DT', and want to compute the Euclidian norm (norm 2) across each row, the apply function can be used.
apply(X = DT, MARGIN = 1, FUN = norm, '2')
Example:
>DT
accx accy accz
1: 9.576807 -0.1629486 -0.2587167
2: 9.576807 -0.1722938 -0.2681506
3: 9.576807 -0.1634264 -0.2681506
4: 9.576807 -0.1545590 -0.2681506
5: 9.576807 -0.1621254 -0.2681506
6: 9.576807 -0.1723825 -0.2682434
7: 9.576807 -0.1723825 -0.2728810
8: 9.576807 -0.1723825 -0.2775187
> apply(X = DT, MARGIN = 1, FUN = norm, '2')
[1] 9.581687 9.582109 9.581954 9.581807 9.581932 9.582114 9.582245 9.582378
Following AbdealiJK's answer,
I experimented further to gain some insight.
Here's one.
x = c(-8e+299, -6e+299, 5e+299, -8e+298, -5e+299)
sqrt(sum(x^2))
norm(x, type='2')
The first result is Inf and the second one is 1.227355e+300 which is quite correct as I show you in the code below.
library(Rmpfr)
y <- mpfr(x, 120)
sqrt(sum(y*y))
The result is 1227354879.... I didn't count the number of trailing numbers but it looks all right. I know there another way around this OVERFLOW problem which is first applying log function to all numbers and summing up, which I do not have time to implement!
Create your matrix as column vise using cbind then the norm function works well with Frobenius norm (the Euclidean norm) as an argument.
x1<-cbind(1:3)
norm(x1,"f")
[1] 3.741657
sqrt(1*1+2*2+3*3)
[1] 3.741657
I'm trying to write a function to determine the euclidean distance between x (one point) and y (a set of n points).
How should I pass y to the function? Until now, I used a matrix like that:
[,1] [,2] [,3]
[1,] 0 2 1
[2,] 1 1 1
Which would pass the points (0,2,1) and (1,1,1) to that function.
However, when I pass x as a normal (column) vector, the two variables don't match in the function.
I either have to transpose x or y, or save a vector of vectors an other way.
My question: What is the standard way to save more than one vector in R? (my matrix y)
Is it just my y transposed or maybe a list or dataframe?
There is no standard way, so you should just pick the most effective one, what on the other hand depends on how this vector of vectors looks just after creation (it is better to avoid any conversion which is not necessary) and on the speed of the function itself.
I believe that a data.frame with columns x, y and z should be pretty good choice; the distance function will be quite simple and fast then:
d<-function(x,y) sqrt((y$x-x[1])^2+(y$y-x[2])^2+(y$z-x[3])^2)
The apply function with the margin argument = 1 seems the most obvious:
> x
[,1] [,2] [,3]
[1,] 0 2 1
[2,] 1 1 1
> apply(x , 1, function(z) crossprod(z, 1:length(z) ) )
[1] 7 6
> 2*2+1*3
[1] 7
> 1*1+2*1+3*1
[1] 6
So if you wanted distances then square-root of the crossproduct of the differences to a chose point seems to work:
> apply(x , 1, function(z) sqrt(sum(crossprod(z -c(0,2,2), z-c(0,2,2) ) ) ) )
[1] 1.000000 1.732051