Under R I developed this script:
Sphere_1 = function (x1,x2) return(x1^2+x2^2) #sphere function / objective function
Initial_search_domain_function <- function(N,p,MIN_t_x,MAX_t_x,MIN_t_y,MAX_t_y , Objfun=custom_fun ) {
x1 <- runif(N, min = MIN_t_x, max = MAX_t_x) # Create x vector (same as in Example 1 & 2)
y1 <- runif(N, min = MIN_t_y, max = MAX_t_y) # Create y vector (same as in Example 2)
m <- outer(x1, y1, Objfun) #matrix of minimum objective values
print("matrix of objective function")
#print(m)
p_minimum_value <- sort(m)[1:p] # search p minimum values of the objective function 'Objfun' within m
X_Y_indices=which(relist(m %in% p_minimum_value, m), arr.ind = TRUE) # retrieve their corresponding row/cloumn at m
#print(X_Y_indices)
print("list of respective positions")
v1=x1[X_Y_indices[,1]] # retrieve their corresponding x-coordinate from x1 list
v2=y1[X_Y_indices[,2]] # retrieve their corresponding y-coordinate from y1 list
respective_positions=rbind(v1,v2) # store those coordinate in a matrix
respective_positions=rbind(respective_positions, `fun(x, y)` = apply(respective_positions, 2, function(x) Objfun(x[1], x[2])))
# compute the objective function for each row
rownames(respective_positions)=c("x:","y:","obj-val")
print(respective_positions)
return(respective_positions)
}
Example of output:
Initial_search_domain_function(40,5,-4.5,4.5,-4.5,4.5, Sphere_1 ) ;
[1] "matrix of objective function"
[1] "list of respective positions"
[,1] [,2] [,3] [,4] [,5]
x: 0.2904639 0.29046393 0.29046393 -0.40499210 0.29046393
y: 0.2894644 0.07744045 0.05273694 0.05273694 0.11452047
obj-val 0.1681589 0.09036632 0.08715048 0.16679979 0.09748423
I'm wanting a way such that the code not only work for a two-variables function f(x,y) , but also for n-dimensions function.
For example if n=3 I could get something like:
[1] "list of respective positions"
[,1] [,2] [,3] [,4] [,5]
x: 0.2904639 0.29046393 0.29046393 -0.40499210 0.29046393
y: 0.2894644 0.07744045 0.05273694 0.05273694 0.11452047
z: 0.2904639 0.27046393 0.50046393 -0.90499210 0.129046393
obj-val 0.1681589 0.09036632 0.08715048 0.16679979 0.09748423
with n-dimensional function like:
Sphere = function (x) return(sum(x^2)) #sphere function / objective function
The main problem is that I don't know how to compute Cartesian products for n-sets with their respective objective function values f (X) where X is n-dimensional.
Here is an p-dimensional function. Note that this is just an exact replica of your code, only made to work for p-dimensions. Notice that I gave it a seed argument in order for one to make comparisons:
Sphere_1_pdim = function (x) return(sum(x^2))
Initial_search_domain_function_pdim <- function(N, p, MIN, MAX, Objfun, seed = NULL) {
stopifnot(length(MIN) == length(MAX),length(N) == 1,length(p) == 1)
dims <- numeric(length(MIN)) + N
set.seed(seed)
X <- Map(runif,N,MIN,MAX)
names(X) <- paste0("X",seq_along(X),":")
m <- array(apply(expand.grid(X),1,Objfun), dims)
p_minimum_value <- sort(m)[1:p]
indices <-which(array(m %in% p_minimum_value, dim(m)), arr.ind = TRUE)
t(cbind(mapply("[",X,data.frame(indices)),"obj-val:" = m[indices]))
}
Initial_search_domain_function_pdim(40, 5, c(-4.5,-4.5,-4.5), c(4.5,4.5,4.5), Sphere_1_pdim, 0)
[,1] [,2] [,3] [,4] [,5]
X1: -0.02070682 -0.02070682 -0.05812824 -0.02070682 -0.05812824
X2: -0.20142340 0.16770837 0.16770837 -0.19309277 -0.19309277
X3: -0.19693769 -0.19693769 -0.19693769 -0.19693769 -0.19693769
obj-val: 0.07978461 0.06733932 0.07028944 0.07649804 0.07944816
Your code output:
Initial_search_domain_function(40,5, -4.5, 4.5, -4.5, 4.5, Sphere_1, 0)
[,1] [,2] [,3] [,4] [,5]
x: -0.02070682 -0.02070682 -0.05812824 -0.02070682 -0.05812824
y: -0.20142340 0.16770837 0.16770837 -0.19309277 -0.19309277
obj-val 0.04100016 0.02855487 0.03150499 0.03771359 0.04066371
Related
Suppose that we have the following code :
x1<- runif(40, min = -4.5, max = 4.5) # Create x vector
y1 <- runif(40, min = -4.5, max = 4.5) # Create y vector
custom_fun <- function(x, y) { # Create custom function in R ; here i used the beale function
z <- (1.5 - x + x*y)^2+(2.25 - x + x*y^2)^2+(2.625 - x + x*y^3)^2
return(z)
}
m <- outer(x1, y1, custom_fun)
print(m)
My question is how the obtain the list of the n minimum values of the matrix m , i need also to retrieve the correspending positions in the vectors x1 and y1 ( which means the couples from x1 and y1 that allow us to obtain those minimum values of the matrix m ).
For example , if the excecution of the previous code gives :
#m
[,1] [,2] [,3] [,4] [,5] [,6]
[1,] 508.64893 11.245614 7887.387 3.4916340 1.517622 3232.9724
[2,] 727.76775 19.629751 17649.225 46.0714031 63.427516 6708.3153
[3,] 2227.94868 7.948157 39784.638 0.3509784 6.073413 15901.8932
[4,] 56.06329 13.568900 432.400 11.4225081 10.455426 205.5854
[5,] 1723.86138 22.552465 39256.366 67.8455312 99.733390 15069.6231
[6,] 1328.44959 21.510099 30755.388 59.7772840 86.142180 11775.4025
and i'm wanting for example the p minimum values of the matrix , let p=3 then i will retrieve the values :
0.3509784 -1.517622- 3.4916340 ( those values are near the optimum of the beale function ).
Next , i need a way to retrieve the couples (x,y) that allow us to obtain those minimum values ( from the current excecution ! ) :
custom_fun(a1,b1)=0.3509784
custom_fun(a2,b2)=1.517622
custom_fun(a3,b3)=3.4916340
so how could i retrieve (a1,b1) from the current x1 and y1 vectors ???
I will need also to retrieve the couples (a2,b2) and (a3,b3).
.
Another example , suppose the current excecution gives the following :
[1] "x4 list :"
[1] 2.0695110 3.7844526 -0.6914949
[1] "y4 list :"
[1] -4.079663 1.771313 -4.206778
[1] "matrix of objective function : denoted m "
[,1] [,2] [,3]
[1,] 20870.387 199.495204 25007.158
[2,] 70719.411 521.366216 84647.031
[3,] 2625.535 1.807467 3115.075
with p=2 ; the values 1.807467 and 521.366216 are good while searching the minimum, but the question is how to get their respective coordinates (x,y) from (x4,y4 ) possibilities.
I wish my question is clear.
Thank you a lot for help !
I guess something like this might help :
p <- 3
#Get p minimum value
p_minimum_value <- sort(m)[1:p]
#Get their row and column position
which(relist(m %in% p_minimum_value, m), arr.ind = TRUE)
Suppose that we have the following beale function :
custom_fun <- function(x, y) { # Create custom function in R
z <- (1.5 - x + x*y)^2+(2.25 - x + x*y^2)^2+(2.625 - x + x*y^3)^2
return(z)
}
and we have this list of positions in a matrix m:
[1] "list of respective positions"
[,1] [,2] [,3]
x: 2.482116 -0.7845145 -3.370810
y: -1.031615 1.2035550 1.203555
How can i compute custom_fun(x,y) in each column with an elegant way ?
I'm wanting to store those values in the last row of the matrix m.
Thank you for help!
You can use apply:
custom_fun <- function(x, y) { # Create custom function in R
z <- (1.5 - x + x*y)^2+(2.25 - x + x*y^2)^2+(2.625 - x + x*y^3)^2
return(z)
}
my_mat <- matrix(rnorm(10), nrow = 2, dimnames = list(c("x", "y"), NULL))
my_mat
#> [,1] [,2] [,3] [,4] [,5]
#> x 0.5631441 0.9349816 -1.0088734 -1.364570 -1.32633896
#> y 0.4978350 -1.3265677 0.4206566 -2.532265 -0.01913554
rbind(my_mat, `fun(x, y)` = apply(my_mat, 2, function(x) custom_fun(x[1], x[2])))
#> [,1] [,2] [,3] [,4] [,5]
#> x 0.5631441 0.9349816 -1.0088734 -1.364570 -1.32633896
#> y 0.4978350 -1.3265677 0.4206566 -2.532265 -0.01913554
#> fun(x, y) 9.3600306 9.4626106 26.4985306 749.993103 36.53218207
Created on 2020-06-21 by the reprex package (v0.3.0)
A function takes two sets of values from two vectors (alpha and beta). I need to place the values of the function output in a matrix with size alpha x beta. The function calculates power values. I appreciate your help. I need a matrix 5x5. I have attempted the following code so far:
alpha = c(0.01,0.05,0.10,0.20)
beta = c(0.50,0.60,0.70,0.80,0.90)
pwrmx <- matrix(data=NA, nrow=alpha, ncol=beta)
for (a in alpha){
for (b in beta){
pwr <- power.prop.test(n=NULL, p1=0.25, p2=0.4, sig.level = a, power = b)
print(pwr$n)
}
}
you were almost there, refer the comments:
alpha = c(0.01,0.05,0.10,0.20)
beta = c(0.50,0.60,0.70,0.80,0.90)
# nrow and ncol depends on the length of alpha and beta
pwrmx <- matrix(data=NA, nrow=length(alpha), ncol=length(beta))
# iterate over the length so that you can use it to assign back at the correct index in matrix
for (i in 1:length(alpha)){
for (j in 1:length(beta)){
# as you are interested in the number n from the power analysis
pwrmx[i,j] <- (power.prop.test(n=NULL, p1=0.25, p2=0.4, sig.level = alpha[i], power = beta[j]))$n
}
}
pwrmx
# . [,1] [,2] [,3] [,4] [,5]
#[1,] 129.38048 155.72219 186.60552 226.29474 287.6656
#[2,] 74.90845 95.24355 119.70057 151.86886 202.8095
#[3,] 52.75810 70.01993 91.18885 119.50901 165.1130
#[4,] 32.02629 45.74482 63.12283 87.00637 126.4575
No need of loops, you can create a function to perform the calculation
func <- function(x, y) power.prop.test(n=NULL, p1=0.25, p2=0.4, sig.level = x, power = y)$n
and then use outer and apply the function (func) on each combination of alpha and beta
outer(alpha, beta, Vectorize(func))
# [,1] [,2] [,3] [,4] [,5]
#[1,] 129.38048 155.72219 186.60552 226.29474 287.6656
#[2,] 74.90845 95.24355 119.70057 151.86886 202.8095
#[3,] 52.75810 70.01993 91.18885 119.50901 165.1130
#[4,] 32.02629 45.74482 63.12283 87.00637 126.4575
If I want to calculate the n-dimensional distance of two vectors, I can use a function such as:
a = c(1:10)
b = seq(20, 23, length.out = length(a))
test_fun =
function(x,y) {
return(
sqrt(
sum(
(x - y) ^ 2
)
)
)
}
n_distance = test_fun(a,b)
Now, I want to expand this to a matrix setting: I want to calculate the n-dimensional distance for each pair of rows of two matrices.
set.seed(123)
a_mtx = matrix(1:30, ncol = 5)
b_mtx = matrix(sample(1:15,15), ncol = 5)
n_distance_mtx =
matrix(
NA,
nrow = nrow(b_mtx),
ncol = nrow(a_mtx)
)
for(i in 1:nrow(b_mtx)) {
for(j in 1:nrow(a_mtx)) {
n_distance_mtx[i,j] =
test_fun(a_mtx[j,], b_mtx[i,])
}
}
Where each column of n_distance_mtx contains the distance metrics between each row of a_mtx and b_mtx (so n_distance_mtx[,1] is the distance between a_mtx[1,] and b_mtx[1:3,].
If I calculate column means on n_distance_mtx I can obtain the mean distance between each row in a_mtx and all rows of b_mtx.
colMeans(n_distance_mtx)
#[1] 23.79094 24.90281 26.15618 27.53303 29.01668 30.59220
So 23.79094 is the mean distance between a_mtx[1,] and b_mtx[1:3,], and 24.90281 is the mean distance between a_mtx[2,] and b_mtx[1:3,], and so on.
Question: How can I arrive at the same solution without using for-loops?
I want to apply this method to matrices with much larger dimension (on the order of hundreds of thousands of rows). Looking at this and this, it seems there must be a way to accomplish this with a Vectorized outer function, but I have been unable to generate such a function.
test_fun_vec =
Vectorize(
function(x,y) {
outer(
x,
y,
test_fun
)
}
)
test_fun_vec(a_mtx,b_mtx)
#[1] 4 0 2 7 4 6 3 5 1 5 7 5 10 0 9 11 15 17 8 11 9 12 10 16
#[25] 10 22 20 25 15 24
We can use Vectorize with outer
f1 <- Vectorize(function(i, j) test_fun(a_mtx[j, ], b_mtx[i, ]))
out <- outer(seq_len(nrow(b_mtx)), seq_len(nrow(a_mtx)), FUN = f1)
out
# [,1] [,2] [,3] [,4] [,5] [,6]
#[1,] 20.88061 21.84033 22.97825 24.26932 25.69047 27.22132
#[2,] 24.87971 25.57342 26.43861 27.45906 28.61818 29.89983
#[3,] 25.61250 27.29469 29.05168 30.87070 32.74141 34.65545
colMeans(out)
#[1] 23.79094 24.90281 26.15618 27.53303 29.01668 30.59220
identical(n_distance_mtx, out)
#[1] TRUE
If I unsderstood your question right, you want the Euclidean distance between each vector (row) in a_mtx to the other vectors in b_mtx.
If so, you could use apply twice like this:
result = apply(a_mtx, 1, function(x){ apply(b_mtx, 1, function(y){ test_fun(x,y) })})
This gives a distance matrix:
[,1] [,2] [,3] [,4] [,5] [,6]
[1,] 20.88061 21.84033 22.97825 24.26932 25.69047 27.22132
[2,] 24.87971 25.57342 26.43861 27.45906 28.61818 29.89983
[3,] 25.61250 27.29469 29.05168 30.87070 32.74141 34.65545
where the row index is the corresponding vector (row) from b_mtx and the column index is the corresponding vector from a_mtx
Finally, obtain the mean distance using:
colMeans(result)
[1] 23.79094 24.90281 26.15618 27.53303 29.01668 30.59220
I have some problems with the transformation of a matrix and the names of the rows and columns.
My problem is as follows:
As input-matrix I have a (symmetric) correlation matrix like this one:
The correlation-vector is given by the values of the lower triangular matrix:
Now, I want to compute the variance-covariance-matrix of the these correlations, which are approximately normally distributed with the variance-covariance-matrix:
The variances can be approximated by
-> N is the sample size (in this example N = 66)
The covariances can be approximated by
For example the covariance between r_02 and r_13 is given by
Now, I want to define a function in R which gets the correlation matrix as input and returns the variance-covariance matrix. However, I have problems to implement the calculation of the covariances. My idea is to give names to the elements of the correlation_vector as shown above (r_01, r_02...). Then I want to create the empty variance-cocariance matrix, which has the length of the correlation_vector. The rows and the columns should have the same names as the correlation_vector, so I can call them for example by [01][03]. Then I want to implement a for-loop which sets the value of i and j as well as k and l as shown in the formula for the covariance to the columns and rows of the correlations that I need as input for the covariance-formula. These must always be six different values (ij; ik; il; jk; jl; lk). This is my idea, but I don't now how to implement this in R.
This is my code (without the calculation of the covariances):
require(corpcor)
correlation_matrix_input <- matrix(data=c(1.00,0.561,0.393,0.561,0.561,1.00,0.286,0.549,0.393,0.286,1.00,0.286,0.561,0.549,0.286,1.00),ncol=4,byrow=T)
N <- 66 # Sample Size
vector_of_correlations <- sm2vec(correlation_matrix_input, diag=F) # lower triangular matrix of correlation_matrix_input
variance_covariance_matrix <- matrix(nrow = length(vector_of_correlations), ncol = length(vector_of_correlations)) # creates the empty variance-covariance matrix
# function to fill the matrix by calculating the variance and the covariances
variances_covariances <- function(vector_of_correlations_input, sample_size) {
for (i in (seq(along = vector_of_correlations_input))) {
for (j in (seq(along = vector_of_correlations_input))) {
# calculate the variances for the diagonale
if (i == j) {
variance_covariance_matrix[i,j] = ((1-vector_of_correlations_input[i]**2)**2)/sample_size
}
# calculate the covariances
if (i != j) {
variance_covariance_matrix[i,j] = ???
}
}
}
return(variance_covariance_matrix);
}
Does anyone have an idea, how to implement the calculation of the covariances using the formula shown above?
I would be grateful for any kind of help regarding this problem!!!
It's easier if you keep r as a matrix and use this helper function to make things clearer:
covr <- function(r, i, j, k, l, n){
if(i==k && j==l)
return((1-r[i,j]^2)^2/n)
( 0.5 * r[i,j]*r[k,l]*(r[i,k]^2 + r[i,l]^2 + r[j,k]^2 + r[j,l]^2) +
r[i,k]*r[j,l] + r[i,l]*r[j,k] - (r[i,j]*r[i,k]*r[i,l] +
r[j,i]*r[j,k]*r[j,l] + r[k,i]*r[k,j]*r[k,l] + r[l,i]*r[l,j]*r[l,k]) )/n
}
Now define this second function:
vcovr <- function(r, n){
p <- combn(nrow(r), 2)
q <- seq(ncol(p))
outer(q, q, Vectorize(function(x,y) covr(r, p[1,x], p[2,x], p[1,y], p[2,y], n)))
}
And voila:
> vcovr(correlation_matrix_input, 66)
[,1] [,2] [,3] [,4] [,5] [,6]
[1,] 0.007115262 0.001550264 0.002917481 0.003047666 0.003101602 0.001705781
[2,] 0.001550264 0.010832674 0.001550264 0.006109565 0.001127916 0.006109565
[3,] 0.002917481 0.001550264 0.007115262 0.001705781 0.003101602 0.003047666
[4,] 0.003047666 0.006109565 0.001705781 0.012774221 0.002036422 0.006625868
[5,] 0.003101602 0.001127916 0.003101602 0.002036422 0.007394554 0.002036422
[6,] 0.001705781 0.006109565 0.003047666 0.006625868 0.002036422 0.012774221
EDIT:
For the transformed Z values, as in your comment, you can use this:
covrZ <- function(r, i, j, k, l, n){
if(i==k && j==l)
return(1/(n-3))
covr(r, i, j, k, l, n) / ((1-r[i,j]^2)*(1-r[k,l]^2))
}
And simply replace it in vcovr:
vcovrZ <- function(r, n){
p <- combn(nrow(r), 2)
q <- seq(ncol(p))
outer(q, q, Vectorize(function(x,y) covrZ(r, p[1,x], p[2,x], p[1,y], p[2,y], n)))
}
New result:
> vcovrZ(correlation_matrix_input,66)
[,1] [,2] [,3] [,4] [,5] [,6]
[1,] 0.015873016 0.002675460 0.006212598 0.004843517 0.006478743 0.002710920
[2,] 0.002675460 0.015873016 0.002675460 0.007869213 0.001909452 0.007869213
[3,] 0.006212598 0.002675460 0.015873016 0.002710920 0.006478743 0.004843517
[4,] 0.004843517 0.007869213 0.002710920 0.015873016 0.003174685 0.007858948
[5,] 0.006478743 0.001909452 0.006478743 0.003174685 0.015873016 0.003174685
[6,] 0.002710920 0.007869213 0.004843517 0.007858948 0.003174685 0.015873016
I wrote an approach using combn and row/column indices to generate the different combinations of p.
variances_covariances <- function(m, n) {
r <- m[lower.tri(m)]
var <- (1-r^2)^2
## generate row/column indices
rowIdx <- rep(1:nrow(m), times=colSums(lower.tri(m)))
colIdx <- rep(1:ncol(m), times=rowSums(lower.tri(m)))
## generate combinations
cov <- combn(length(r), 2, FUN=function(i) {
## current row/column indices
cr <- rowIdx[i] ## i,k
cc <- colIdx[i] ## j,l
## define 6 cases
p.ij <- m[cr[1], cc[1]]
p.ik <- m[cr[1], cr[2]]
p.il <- m[cr[1], cc[2]]
p.jk <- m[cc[1], cr[2]]
p.jl <- m[cc[1], cc[2]]
p.kl <- m[cr[2], cc[2]]
## calculate covariance
co <- 0.5 * p.ij * p.kl * (p.ik^2 + p.il^2 + p.jk^2 + p.jl^2) +
p.ik * p.jl + p.il * p.jk -
(p.ij * p.ik * p.il + p.ij * p.jk * p.jl + p.ik * p.jk * p.kl + p.il * p.jl * p.kl)
return(co)
})
## create output matrix
com <- matrix(NA, ncol=length(r), nrow=length(r))
com[lower.tri(com)] <- cov
com[upper.tri(com)] <- t(com)[upper.tri(com)]
diag(com) <- var
return(com/n)
}
Output:
m <- matrix(data=c(1.000, 0.561, 0.393, 0.561,
0.561, 1.000, 0.286, 0.549,
0.393, 0.286, 1.000, 0.286,
0.561, 0.549, 0.286, 1.00), ncol=4, byrow=T)
variances_covariances(m, 66)
# [,1] [,2] [,3] [,4] [,5] [,6]
#[1,] 0.007115262 0.001550264 0.001550264 0.003101602 0.003101602 0.001705781
#[2,] 0.001550264 0.010832674 0.010832674 0.001127916 0.001127916 0.006109565
#[3,] 0.001550264 0.010832674 0.007115262 0.001127916 0.001127916 0.006109565
#[4,] 0.003101602 0.001127916 0.001127916 0.012774221 0.007394554 0.002036422
#[5,] 0.003101602 0.001127916 0.001127916 0.007394554 0.007394554 0.002036422
#[6,] 0.001705781 0.006109565 0.006109565 0.002036422 0.002036422 0.012774221
I hope, I have done everything right.
salam/hello
variance_covariance_matrix<- diag (variance vector, length (r),length (r))
pcomb <- combn(length(r), 2)
for (k in 1:length(r)){
i<- pcomb[1,k]
j<- pcomb[2,k]
variance_covariance_matrix[i,j]<- variance_covariance_matrix [j,i]<- genCorr[k] * sqrt (sig2g[i]) * sqrt (sig2g[j])
}