I have this code and I want to run it but I got some error, I think It's about type of my data but I can't understand how should I write it for preventing it.
function dacmm(i0::Int64, i1::Int64, j0::Int64, j1::Int64,
k0::Int64, k1::Int64, A::Int64, B::Int64, c::Int64, n::Int64, basecase::Int64)
## A, B, C are matrices
## We compute C = A * B
if n > basecase
n = n/2
dacmm(i0, i1, j0, j1, k0, k1, A, B, c, n, basecase)
dacmm(i0, i1, j0, j1+n, k0, k1+n, A, B, c, n, basecase)
dacmm(i0+n, i1, j0, j1, k0+n, k1, A, B, c, n, basecase)
dacmm(i0+n, i1, j0, j1+n, k0+n, k1+n, A, B, c, n, basecase)
dacmm(i0, i1+n, j0+n, j1, k0, k1, A, B, C, n, basecase)
dacmm(i0, i1+n, j0+n, j1+n, k0, k1+n, A, B, c, n, basecase)
dacmm(i0+n, i1+n, j0+n, j1, k0+n, k1, A, B, c, n, basecase)
dacmm(i0+n, i1+n, j0+n, j1+n, k0+n, k1+n, A, B, c, n, basecase)
else
for i= 1:n, j=1:n, k=1:n
c[i+k0,k1+j] = c[i+k0,k1+j] + A[i+i0,i1+k] * B[k+j0,j1+j]
end
end
end
n=4;
basecase = 2;
A = [rem(rand(Int32),5) for i =1:n, j = 1:n];
B = [rem(rand(Int32),5) for i =1:n, j = 1:n];
C = zeros(Int32,n,n);
error:
ArgumentError: invalid index: 1.0
Stacktrace:
[1] to_indices at ./indices.jl:215 [inlined]
[2] to_indices at ./indices.jl:213 [inlined]
[3] getindex at ./abstractarray.jl:882 [inlined]
[4] dacmm(::Int64, ::Int64, ::Int64, ::Int64, ::Int64, ::Int64, ::Array{Int64,2}, ::Array{Int64,2}, ::Array{Int32,2}, ::Float64, ::Int64) at ./In[24]:16
[5] dacmm(::Int64, ::Int64, ::Int64, ::Int64, ::Int64, ::Int64, ::Array{Int64,2}, ::Array{Int64,2}, ::Array{Int32,2}, ::Int64, ::Int64) at ./In[24]:6
[6] include_string(::String, ::String) at ./loading.jl:515
As the stack trace points out, you tried to call a method of the dacmm function which takes a Float64 in the penultimate argument:
|
V
[4] dacmm(::Int64, ::Int64, ... ::Float64, ::Int64) at ./In[24]:16
[5] dacmm(::Int64, ::Int64, ... ::Int64, ::Int64) at ./In[24]:6
but no such method is available. You ended up there because n = n/2 returned a float not an integer.
The problem doesn't occur in the original code because there, the function's arguments weren't too restricted with type information.
Related
How is it possible to attain the value of #NLexpression when the variables are fixed? In the following code variables have fixed but value of K1 has not been reached.
using JuMP, Distributions,Juniper
#-----Model parameters--------------------------------------------------------
sig, C1, c0 = 2, 300, 10;
E, landa, T0, T1, T2, gam1, gam2, a1, a2, a3, ap = 0.05, 0.01, 0, 2, 2, 1, 1, 0.5, 0.1, 50, 25;
xhat=[2.807064523673271;23.0;1.3349699464500042];
f(x) = cdf(Normal(0, 1), x);
#---------------------------------------------------------------------------
ALT= Model(optimizer_with_attributes(Juniper.Optimizer, "nl_solver"=>optimizer_with_attributes(Ipopt.Optimizer, "print_level" => 0),
"mip_solver"=>optimizer_with_attributes(Gurobi.Optimizer, "logLevel" => 0),"registered_functions" =>[Juniper.register( :f, 1, f; autodiff = true)])
);
# variables-----------------------------------------------------------------
JuMP.register(ALT, :f, 1, f; autodiff = true);
#variable(ALT, hp == xhat[3]);
#variable(ALT, Lp ==xhat[1]);
#variable(ALT, np==xhat[2], Int);
#---------------------------------------------------------------------------
C1=rand(100:100:300);
sig=rand(0.5:0.5:2);
#---------------------------------------------------------------------------
k1=#NLexpression(ALT,hp/(1-f(Lp-sig*sqrt(np))+f(-Lp - sig*sqrt(np))));
JuMP.value(k1);
the error is this:
julia> JuMP.value(k1)
ERROR: type Nothing has no field status
Stacktrace:
[1] getproperty(::Nothing, ::Symbol) at .\Base.jl:33
[2] get at C:\Users\admin\.julia\packages\Juniper\dNHnx\src\MOI_wrapper\results.jl:4 [inlined]
[3] get(::MathOptInterface.Bridges.LazyBridgeOptimizer{Juniper.Optimizer}, ::MathOptInterface.TerminationStatus) at C:\Users\admin\.julia\packages\MathOptInterface\bygN7\src\Bridges\bridge_optimizer.jl:587
[4] get(::MathOptInterface.Utilities.CachingOptimizer{MathOptInterface.AbstractOptimizer,MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}, ::MathOptInterface.TerminationStatus) at C:\Users\admin\.julia\packages\MathOptInterface\bygN7\src\Utilities\cachingoptimizer.jl:553
[5] _moi_get_result(::MathOptInterface.Utilities.CachingOptimizer{MathOptInterface.AbstractOptimizer,MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}, ::MathOptInterface.VariablePrimal, ::Vararg{Any,N} where N) at C:\Users\admin\.julia\packages\JuMP\YXK4e\src\JuMP.jl:844
[6] get(::Model, ::MathOptInterface.VariablePrimal, ::VariableRef) at C:\Users\admin\.julia\packages\JuMP\YXK4e\src\JuMP.jl:877
[7] value(::VariableRef; result::Int64) at C:\Users\admin\.julia\packages\JuMP\YXK4e\src\variables.jl:767
[8] #103 at C:\Users\admin\.julia\packages\JuMP\YXK4e\src\nlp.jl:1159 [inlined]
[9] value(::NonlinearExpression, ::JuMP.var"#103#104"{Int64}) at C:\Users\admin\.julia\packages\JuMP\YXK4e\src\nlp.jl:1102
[10] #value#102 at C:\Users\admin\.julia\packages\JuMP\YXK4e\src\nlp.jl:1159 [inlined]
[11] value(::NonlinearExpression) at C:\Users\admin\.julia\packages\JuMP\YXK4e\src\nlp.jl:1159
[12] top-level scope at none:1
would you please help me how the error is solved?
Thanks.
Please update to Juniper v0.8.0. I fixed this issue a few days ago.
p.s., in future, please consider posting on the JuMP community forum: https://discourse.julialang.org/c/domain/opt/13. There are more readers of JuMP-specific questions.
I am new to Julia, so please forgive me if this is too basic. I am trying to run the following script.
using Pkg
Pkg.add("DataFrames")
using DataFrames
function LS(x,y,a)
T = size(x,1)
N = size(x,2)
pred = fill(0.0,T)
w= fill(0.0,N)
for t = 1:T
x1 = x[t,:]
pred[t] = transpose(w) * x1
err = a*(y[t]-pred[t])
w = w + (err * x1)
end
return pred
end
input = readtable("input.csv")
output = readtable("label.csv")
en = convert(Array, input)
out = convert(Array, output)
a = 0.5
prediction = LS(en,out,a)
print(prediction)
When I run I get the following error:
ERROR: LoadError: MethodError: no method matching
*(::Array{Float64,1}, ::Array{Union{Missing, Float64},1}) Closest candidates are: *(::Any, ::Any, !Matched::Any, !Matched::Any...) at
operators.jl:502 *(!Matched::LinearAlgebra.Adjoint{#s571,#s570}
where #s570<:Union{DenseArray{T<:Union{Complex{Float32},
Complex{Float64}, Float32, Float64},2},
ReinterpretArray{T<:Union{Complex{Float32}, Complex{Float64}, Float32,
Float64},2,S,A} where S where A<:Union{SubArray{T,N,A,I,true} where
I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where A<:DenseArray
where N where T, DenseArray}, ReshapedArray{T<:Union{Complex{Float32},
Complex{Float64}, Float32, Float64},2,A,MI} where
MI<:Tuple{Vararg{SignedMultiplicativeInverse{Int64},N} where N} where
A<:Union{ReinterpretArray{T,N,S,A} where S where
A<:Union{SubArray{T,N,A,I,true} where
I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where A<:DenseArray
where N where T, DenseArray} where N where T, SubArray{T,N,A,I,true}
where I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where
A<:DenseArray where N where T, DenseArray},
SubArray{T<:Union{Complex{Float32}, Complex{Float64}, Float32,
Float64},2,A,I,L} where L where I<:Tuple{Vararg{Union{Int64,
AbstractRange{Int64}, AbstractCartesianIndex},N} where N} where
A<:Union{ReinterpretArray{T,N,S,A} where S where
A<:Union{SubArray{T,N,A,I,true} where
I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where A<:DenseArray
where N where T, DenseArray} where N where T, ReshapedArray{T,N,A,MI}
where MI<:Tuple{Vararg{SignedMultiplicativeInverse{Int64},N} where N}
where A<:Union{ReinterpretArray{T,N,S,A} where S where
A<:Union{SubArray{T,N,A,I,true} where
I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where A<:DenseArray
where N where T, DenseArray} where N where T, SubArray{T,N,A,I,true}
where I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where
A<:DenseArray where N where T, DenseArray} where N where T,
DenseArray}} where #s571, ::Union{DenseArray{S,1},
ReinterpretArray{S,1,S,A} where S where
A<:Union{SubArray{T,N,A,I,true} where
I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where A<:DenseArray
where N where T, DenseArray}, ReshapedArray{S,1,A,MI} where
MI<:Tuple{Vararg{SignedMultiplicativeInverse{Int64},N} where N} where
A<:Union{ReinterpretArray{T,N,S,A} where S where
A<:Union{SubArray{T,N,A,I,true} where
I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where A<:DenseArray
where N where T, DenseArray} where N where T, SubArray{T,N,A,I,true}
where I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where
A<:DenseArray where N where T, DenseArray}, SubArray{S,1,A,I,L} where
L where I<:Tuple{Vararg{Union{Int64, AbstractRange{Int64},
AbstractCartesianIndex},N} where N} where
A<:Union{ReinterpretArray{T,N,S,A} where S where
A<:Union{SubArray{T,N,A,I,true} where
I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where A<:DenseArray
where N where T, DenseArray} where N where T, ReshapedArray{T,N,A,MI}
where MI<:Tuple{Vararg{SignedMultiplicativeInverse{Int64},N} where N}
where A<:Union{ReinterpretArray{T,N,S,A} where S where
A<:Union{SubArray{T,N,A,I,true} where
I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where A<:DenseArray
where N where T, DenseArray} where N where T, SubArray{T,N,A,I,true}
where I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where
A<:DenseArray where N where T, DenseArray} where N where T,
DenseArray}}) where {T<:Union{Complex{Float32}, Complex{Float64},
Float32, Float64}, S} at
/Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/LinearAlgebra/src/matmul.jl:98
*(!Matched::LinearAlgebra.Adjoint{#s571,#s570} where #s570<:LinearAlgebra.AbstractTriangular where #s571, ::AbstractArray{T,1} where T) at
/Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/LinearAlgebra/src/triangular.jl:1805
...
Stacktrace: 2 LS(::Array{Union{Missing, Float64},2},
::Array{Union{Missing, Float64},2}, ::Float64) at
/Users/wj/Desktop/Julia/NLSR.jl:16
2 top-level scope at none:0
[3] include at ./boot.jl:317 [inlined]
[4] include_relative(::Module, ::String) at ./loading.jl:1044
[5] include(::Module, ::String) at ./sysimg.jl:29
[6] exec_options(::Base.JLOptions) at ./client.jl:231
[7] _start() at ./client.jl:425
How can I successfully run this script?
Remark:
The algorithm pseudo code is as follows:
Script updated after the comment by Warren
The code works now!
I'm not sure what you're trying to do here, but it doesn't look anything like least squares to me.
Having said that, the problem in your script is in this line here:
w = w + (err * x1)
x1 and err will both be vectors, and there is no multiplication operator method for two vectors. For example, rand(2) * rand(2) will also error. This is because usage of * is ambiguous in this situation.
If you want the dot product, use dot(x1, err).
If you want the element-wise product, use x1 .* err, which broadcasts * across the elements of the inputs.
There are several other issues with your code, but I'm hesitant to try and correct them, since, as stated, I'm not really sure what you're trying to do.
I am trying to use maximum likelihood to estimate the normal linear model in Julia. I have used the following code to simulate the process with just an intercept and an anonymous function per the Optim documentation regarding values that do not change:
using Optim
nobs = 500
nvar = 1
β = ones(nvar)*3.0
x = [ones(nobs) randn(nobs,nvar-1)]
ε = randn(nobs)*0.5
y = x*β + ε
function LL_anon(X, Y, β, σ)
-(-length(X)*log(2π)/2 - length(X)*log(σ) - (sum((Y - X*β).^2) / (2σ^2)))
end
LL_anon(X,Y, pars) = LL_anon(X,Y, pars...)
res2 = optimize(vars -> LL_anon(x,y, vars...), [1.0,1.0]) # Start values: β=1.0, σ=1.0
This actually recovered the parameters and I received the following output:
* Algorithm: Nelder-Mead
* Starting Point: [1.0,1.0]
* Minimizer: [2.980587812647935,0.5108406803949835]
* Minimum: 3.736217e+02
* Iterations: 47
* Convergence: true
* √(Σ(yᵢ-ȳ)²)/n < 1.0e-08: true
* Reached Maximum Number of Iterations: false
* Objective Calls: 92
However, when I try and set nvar = 2, i.e. an intercept plus an additional covariate, I get the following error message:
MethodError: no method matching LL_anon(::Array{Float64,2}, ::Array{Float64,1}, ::Float64, ::Float64, ::Float64)
Closest candidates are:
LL_anon(::Any, ::Any, ::Any, ::Any) at In[297]:2
LL_anon(::Array{Float64,1}, ::Array{Float64,1}, ::Any, ::Any) at In[113]:2
LL_anon(::Any, ::Any, ::Any) at In[297]:4
...
Stacktrace:
[1] (::##245#246)(::Array{Float64,1}) at .\In[299]:1
[2] value!!(::NLSolversBase.NonDifferentiable{Float64,Array{Float64,1},Val{false}}, ::Array{Float64,1}) at C:\Users\dolacomb\.julia\v0.6\NLSolversBase\src\interface.jl:9
[3] initial_state(::Optim.NelderMead{Optim.AffineSimplexer,Optim.AdaptiveParameters}, ::Optim.Options{Float64,Void}, ::NLSolversBase.NonDifferentiable{Float64,Array{Float64,1},Val{false}}, ::Array{Float64,1}) at C:\Users\dolacomb\.julia\v0.6\Optim\src\multivariate/solvers/zeroth_order\nelder_mead.jl:136
[4] optimize(::NLSolversBase.NonDifferentiable{Float64,Array{Float64,1},Val{false}}, ::Array{Float64,1}, ::Optim.NelderMead{Optim.AffineSimplexer,Optim.AdaptiveParameters}, ::Optim.Options{Float64,Void}) at C:\Users\dolacomb\.julia\v0.6\Optim\src\multivariate/optimize\optimize.jl:25
[5] #optimize#151(::Array{Any,1}, ::Function, ::Tuple{##245#246}, ::Array{Float64,1}) at C:\Users\dolacomb\.julia\v0.6\Optim\src\multivariate/optimize\interface.jl:62
[6] #optimize#148(::Array{Any,1}, ::Function, ::Function, ::Array{Float64,1}) at C:\Users\dolacomb\.julia\v0.6\Optim\src\multivariate/optimize\interface.jl:52
[7] optimize(::Function, ::Array{Float64,1}) at C:\Users\dolacomb\.julia\v0.6\Optim\src\multivariate/optimize\interface.jl:52
I'm not sure why adding an additional variable is causing this issue but it seems like a type instability problem.
The second issue is that when I use my original working example and set the starting values to [2.0,2.0], I get the following error message:
log will only return a complex result if called with a complex argument. Try log(complex(x)).
Stacktrace:
[1] nan_dom_err at .\math.jl:300 [inlined]
[2] log at .\math.jl:419 [inlined]
[3] LL_anon(::Array{Float64,2}, ::Array{Float64,1}, ::Float64, ::Float64) at .\In[302]:2
[4] (::##251#252)(::Array{Float64,1}) at .\In[304]:1
[5] value(::NLSolversBase.NonDifferentiable{Float64,Array{Float64,1},Val{false}}, ::Array{Float64,1}) at C:\Users\dolacomb\.julia\v0.6\NLSolversBase\src\interface.jl:19
[6] update_state!(::NLSolversBase.NonDifferentiable{Float64,Array{Float64,1},Val{false}}, ::Optim.NelderMeadState{Array{Float64,1},Float64,Array{Float64,1}}, ::Optim.NelderMead{Optim.AffineSimplexer,Optim.AdaptiveParameters}) at C:\Users\dolacomb\.julia\v0.6\Optim\src\multivariate/solvers/zeroth_order\nelder_mead.jl:193
[7] optimize(::NLSolversBase.NonDifferentiable{Float64,Array{Float64,1},Val{false}}, ::Array{Float64,1}, ::Optim.NelderMead{Optim.AffineSimplexer,Optim.AdaptiveParameters}, ::Optim.Options{Float64,Void}, ::Optim.NelderMeadState{Array{Float64,1},Float64,Array{Float64,1}}) at C:\Users\dolacomb\.julia\v0.6\Optim\src\multivariate/optimize\optimize.jl:51
[8] optimize(::NLSolversBase.NonDifferentiable{Float64,Array{Float64,1},Val{false}}, ::Array{Float64,1}, ::Optim.NelderMead{Optim.AffineSimplexer,Optim.AdaptiveParameters}, ::Optim.Options{Float64,Void}) at C:\Users\dolacomb\.julia\v0.6\Optim\src\multivariate/optimize\optimize.jl:25
[9] #optimize#151(::Array{Any,1}, ::Function, ::Tuple{##251#252}, ::Array{Float64,1}) at C:\Users\dolacomb\.julia\v0.6\Optim\src\multivariate/optimize\interface.jl:62
Again, I’m not sure why this is happening and since start values are very important I’d like to know how to overcome this issue and they are not too far off from the true values.
Any help would be greatly appreciated!
Splatting causes the problem. E.g. it transforms [1, 2, 3] into three parameters while your function accepts only two.
Use the following call:
res2 = optimize(vars -> LL_anon(x,y, vars[1:end-1], vars[end]), [1.0,1.0,1.0])
and you can remove the following line from your code
LL_anon(X,Y, pars) = LL_anon(X,Y, pars...)
or replace it with:
LL_anon(X,Y, pars) = LL_anon(X,Y, pars[1:end-1], pars[end])
but it would not be called by optimization routine unless you change a call to:
res2 = optimize(vars -> LL_anon(x,y, vars), [1.0,1.0,1.0])
Finally - to get good performance of this code I would recommend to wrap it all in a function.
EDIT: now I see a second question. The reason is that σ can become negative in the optimization process and then log(σ) fails. The simplest thing to do in this case is to take log(abs(σ))) like this:
function LL_anon(X, Y, β, σ)
-(-length(X)*log(2π)/2 - length(X)*log(abs(σ)) - (sum((Y - X*β).^2) / (2σ^2)))
end
Of course then you have to take absolute value of σ as a solution as you might get a negative value from optimization routine.
A cleaner way would be to optimize over e.g. log(σ) not σ like this:
function LL_anon(X, Y, β, logσ)
-(-length(X)*log(2π)/2 - length(X)*logσ - (sum((Y - X*β).^2) / (2(exp(logσ))^2)))
end
but then you have to use exp(logσ) to recover σ after optimization finishes.
I have asked around regarding this and have another option. The main reason for looking at this problem is twofold. One, to learn how to use the optimization routines in Julia in a canonical situation and two, to expand this to spatial econometric models. With that in mind, I'm posting the other suggested code from the Julia message board so that others may see another solution.
using Optim
nobs = 500
nvar = 2
β = ones(nvar) * 3.0
x = [ones(nobs) randn(nobs, nvar - 1)]
ε = randn(nobs) * 0.5
y = x * β + ε
function LL_anon(X, Y, β, log_σ)
σ = exp(log_σ)
-(-length(X) * log(2π)/2 - length(X) * log(σ) - (sum((Y - X * β).^2) / (2σ^2)))
end
opt = optimize(vars -> LL_anon(x,y, vars[1:nvar], vars[nvar + 1]),
ones(nvar+1))
# Use forward autodiff to get first derivative, then optimize
fun1 = OnceDifferentiable(vars -> LL_anon(x, y, vars[1:nvar], vars[nvar + 1]),
ones(nvar+1))
opt1 = optimize(fun1, ones(nvar+1))
Results of Optimization Algorithm
Algorithm: L-BFGS
Starting Point: [1.0,1.0,1.0]
Minimizer: [2.994204150985705,2.9900626550063305, …]
Minimum: 3.538340e+02
Iterations: 12
Convergence: true
|x - x’| ≤ 1.0e-32: false
|x - x’| = 8.92e-12
|f(x) - f(x’)| ≤ 1.0e-32 |f(x)|: false
|f(x) - f(x’)| = 9.64e-16 |f(x)|
|g(x)| ≤ 1.0e-08: true
|g(x)| = 6.27e-09
Stopped by an increasing objective: true
Reached Maximum Number of Iterations: false
Objective Calls: 50
Gradient Calls: 50
opt1.minimizer
3-element Array{Float64,1}:
2.9942
2.99006
-1.0651 #Note: needs to be exponentiated
# Get Hessian, use Newton!
fun2 = TwiceDifferentiable(vars -> LL_anon(x, y, vars[1:nvar], vars[nvar + 1]),
ones(nvar+1))
opt2 = optimize(fun2, ones(nvar+1))
Results of Optimization Algorithm
Algorithm: Newton’s Method
Starting Point: [1.0,1.0,1.0]
Minimizer: [2.99420415098702,2.9900626550079026, …]
Minimum: 3.538340e+02
Iterations: 9
Convergence: true
|x - x’| ≤ 1.0e-32: false
|x - x’| = 1.36e-11
|f(x) - f(x’)| ≤ 1.0e-32 |f(x)|: false
|f(x) - f(x’)| = 1.61e-16 |f(x)|
|g(x)| ≤ 1.0e-08: true
|g(x)| = 6.27e-09
Stopped by an increasing objective: true
Reached Maximum Number of Iterations: false
Objective Calls: 45
Gradient Calls: 45
Hessian Calls: 9
fieldnames(fun2)
13-element Array{Symbol,1}:
:f
:df
:fdf
:h
:F
:DF
:H
:x_f
:x_df
:x_h
:f_calls
:df_calls
:h_calls
opt2.minimizer
3-element Array{Float64,1}:
2.98627
3.00654
-1.11313
numerical_hessian = (fun2.H) #.H is the numerical Hessian
3×3 Array{Float64,2}:
64.8715 -9.45045 0.000121521
-0.14568 66.4507 0.0
1.87326e-6 4.10675e-9 44.7214
From here, one can use the numerical Hessian to obtain the standard errors for the estimates and form t-statistics, etc. for their own functions.
Again, thank you for providing an answer and I hope people find this information useful.
I am interested in calculating quantity:
where x_i is a 1xD vector (one out of my N data of dimension D), μ is a DxK matrix and W is a list of K DxD matrices.
This should result in a 1XK vector. I try it for all N and K in the following way that works:
res = zeros(N,K);
for i in 1:N
for k in 1:K
res[i,k] = (x_matrix[i,:]-mus_matrix[:,k])'*
w_matrix[k]*(x_matrix[i,:]-mus_matrix[:,k])
If I try to vectorize it, using the following:
res = zeros(N,K);
for i in 1:N
res[i,:] = (x_matrix[i,:].-mus_matrix)'.*w_matrix.*(x_matrix[i,:].-mus_matrix)
I get the following error:
ERROR: DimensionMismatch("arrays could not be broadcast to a common size")
Stacktrace:
[1] _bcs1(::Base.OneTo{Int64}, ::Base.OneTo{Int64}) at ./broadcast.jl:70
[2] _bcs at ./broadcast.jl:63 [inlined]
[3] broadcast_shape(::Tuple{Base.OneTo{Int64},Base.OneTo{Int64}}, ::Tuple{Base.OneTo{Int64}}, ::Tuple{Base.OneTo{Int64},Base.OneTo{Int64}}, ::Vararg{Tuple{Base.OneTo{Int64},Base.OneTo{Int64}},N} where N) at ./broadcast.jl:57 (repeats 3 times)
[4] broadcast_indices(::Array{Float64,2}, ::Array{Any,1}, ::Array{Float64,1}, ::Vararg{Any,N} where N) at ./broadcast.jl:53
[5] broadcast_c(::Function, ::Type{Array}, ::Array{Float64,2}, ::Array{Any,1}, ::Vararg{Any,N} where N) at ./broadcast.jl:311
[6] broadcast(::Function, ::Array{Float64,2}, ::Array{Any,1}, ::Array{Float64,1}, ::Vararg{Any,N} where N) at ./broadcast.jl:434
Here is an example:
julia> N = 5
5
julia> D=2
2
julia> K = 4
4
julia> W=[]
0-element Array{Any,1}
julia> x = rand(N,D)
5×2 Array{Float64,2}:
0.576477 0.9575
0.184454 0.660436
0.470267 0.729649
0.648879 0.782561
0.626453 0.111332
julia> mu = rand(K,D)
4×2 Array{Float64,2}:
0.989281 0.00126782
0.659106 0.66136
0.50843 0.289442
0.327962 0.523229
julia> for i in 1:K
push!(W,rand(D,D))
end
And then run
julia> (x_matrix[i,:]-mus_matrix[:,k])'*
w_matrix[k]*(x_matrix[i,:]-mus_matrix[:,k])
34649.850360744866
But with the second code
julia> (x_matrix[i,:].-mus_matrix)'.*w_matrix.*(x_matrix[i,:].-mus_matrix)
ERROR: DimensionMismatch("arrays could not be broadcast to a common size")
Stacktrace:
[1] _bcs1(::Base.OneTo{Int64}, ::Base.OneTo{Int64}) at ./broadcast.jl:70
[2] _bcs at ./broadcast.jl:63 [inlined]
[3] broadcast_shape(::Tuple{Base.OneTo{Int64},Base.OneTo{Int64}}, ::Tuple{Base.OneTo{Int64}}, ::Tuple{Base.OneTo{Int64},Base.OneTo{Int64}}, ::Vararg{Tuple{Base.OneTo{Int64},Base.OneTo{Int64}},N} where N) at ./broadcast.jl:57 (repeats 3 times)
[4] broadcast_indices(::Array{Float64,2}, ::Array{Any,1}, ::Array{Float64,1}, ::Vararg{Any,N} where N) at ./broadcast.jl:53
[5] broadcast_c(::Function, ::Type{Array}, ::Array{Float64,2}, ::Array{Any,1}, ::Vararg{Any,N} where N) at ./broadcast.jl:311
[6] broadcast(::Function, ::Array{Float64,2}, ::Array{Any,1}, ::Array{Float64,1}, ::Vararg{Any,N} where N) at ./broadcast.jl:434
TL/DR: optimized variant below, but Einsum looks nicer, IMHO.
Looks like a case for using Einstein summation notation. In Julia, Einsum.jl can do this:
julia> N = 5
5
julia> D = 3
3
julia> K = 10
10
julia> x = rand(N, D)
5×3 Array{Float64,2}:
0.587436 0.210529 0.261725
0.527269 0.457477 0.482939
0.52726 0.411209 0.138872
0.89107 0.464789 0.758392
0.885267 0.931014 0.672959
julia> μ = rand(D, K)
3×10 Array{Float64,2}:
0.280792 0.265066 0.81437 0.503377 0.0717916 … 0.275872 0.609961 0.0820088 0.0042564
0.0177643 0.0959438 0.563948 0.332433 0.088527 0.691971 0.0296638 0.604488 0.956057
0.668128 0.444816 0.74203 0.518232 0.48689 0.465067 0.117469 0.729514 0.109973
julia> W = rand(K, D, D)
10×3×3 Array{Float64,3}:
[:, :, 1] =
0.320861 0.662103 0.219234
0.780944 0.769377 0.566203
0.466207 0.428527 0.330901
0.15534 0.035435 0.346737
0.810676 0.328116 0.469505
0.676575 0.668204 0.285334
0.455551 0.211295 0.85295
0.229995 0.741487 0.783361
0.0937583 0.401419 0.47032
0.956335 0.434213 0.967791
[:, :, 2] =
0.275903 0.130298 0.184485
0.941648 0.940107 0.439454
0.425292 0.252654 0.797115
0.0203406 0.594075 0.484809
0.164309 0.941597 0.455314
0.73628 0.109502 0.920664
0.906305 0.177235 0.540193
0.360038 0.0486971 0.20626
0.914357 0.699901 0.295872
0.284143 0.659117 0.291479
[:, :, 3] =
0.138311 0.921371 0.353719
0.345247 0.70865 0.246736
0.361364 0.636543 0.343837
0.752149 0.581561 0.346399
0.705888 0.24765 0.703952
0.992327 0.369668 0.109407
0.341624 0.223715 0.970667
0.762169 0.94248 0.917569
0.0367128 0.589345 0.121106
0.826602 0.692111 0.229499
julia> using Einsum
julia> #einsum r[n,k] := (x[n,i] - μ[i,k]) * W[k,i,j] * (x[n,j] - μ[j,k])
julia> r
5×10 Array{Float64,2}:
0.0176889 0.087092 0.522184 0.0417967 … -0.0430999 0.041266 -0.0596579 0.432076
0.0521066 0.364059 0.181515 0.00434307 -0.0248712 0.226976 -0.0686294 0.437169
-0.0472136 0.127803 0.458812 0.0119074 0.0391649 -0.0190299 -0.0585371 0.264379
0.468634 1.16498 -0.00263205 0.192809 0.273537 1.13787 -0.0653081 1.41321
0.749655 2.20266 0.0205068 0.420249 0.573358 1.42499 0.441232 1.67574
Which #macroexpands to essentially the following loops (plus preparation and bounds checking):
begin
local k
for k = 1:size(μ, 2)
begin
local n
for n = 1:size(x, 1)
begin
local s = zero(T)
begin
local j
for j = 1:size(W, 3)
begin
local i
for i = 1:size(x, 2)
s += (x[n, i] - μ[i, k]) * W[k, i, j] * (x[n, j] - μ[j, k])
end
end
end
end
r[n, k] = s
end
end
end
end
end
Now, to find something more performant, I compared a couple of variants using BenchmarkTools.jl. You can see the full code and results on my laptop here. It shows that the Einsum variant already is in fact better than the original:
# Original:
# memory estimate: 1017.73 MiB
# allocs estimate: 3429967
# median time: 361.982 ms (15.94% GC)
# Einsum:
# memory estimate: 2.64 MiB
# allocs estimate: 76
# median time: 127.536 ms (0.00% GC)
By far the most efficient and least allocating variant is the following, which requires x = x' and W = permutedims(W, [2, 3, 1]) (assuming you can change your representation easily):
function test_optimized!(res, x, μ, W)
z = zero(eltype(x))
for k = 1:size(μ, 1)
for n = 1:size(x, 1)
res[n, k] = z
for i = 1:size(W, 1)
for j = 1:size(W, 2)
#inbounds res[n, k] += (x[i, n] - μ[i, k]) * W[i, j, k] * (x[j, n] - μ[j, k])
end
end
end
end
end
function test_optimized(x, μ, W)
res = zeros(N, K)
test_optimized!(res, x, μ, W)
res
end
This brings us down to
# memory estimate: 2.63 MiB
# allocs estimate: 2
# median time: 521.215 μs (0.00% GC)
It uses a couple of "tricks" that can be found in the docs: filling a preallocated matrix in a separate method, accessing strides in column-major order, and using #inbounds (although that only improves things at the order of a microsecond).
There is also TensorOperations.jl, which I think does more intelligent things under the hood, but it fail on this:
julia> #tensor r[n,k] := (x[n,i] - μ[i,k]) * W[k,i,j] * (x[n,j] - μ[j,k])
ERROR: TensorOperations.IndexError{String}("invalid index specification: (:n, :i) to (:i, :k)")
Stacktrace:
[1] add_indices(::Tuple{Symbol,Symbol}, ::Tuple{Symbol,Symbol}) at /home/philipp/.julia/v0.6/TensorOperations/src/implementation/indices.jl:22
[2] + at /home/philipp/.julia/v0.6/TensorOperations/src/indexnotation/sum.jl:40 [inlined]
[3] -(::TensorOperations.IndexedObject{(:n, :i),:N,Array{Float64,2},Int64}, ::TensorOperations.IndexedObject{(:i, :k),:N,Array{Float64,2},Int64}) at /home/philipp/.julia/v0.6/TensorOperations/src/indexnotation/sum.jl:44
I guess that's deliberate and has to do with efficiency, see this issue.
I wrote a function that computes a row of Pascals Triangle.
function calc_row(s, prev::Int64, num::Int64, den::Int64)
c = 1
next_num = num
next_den = den
if prev > 0
c = convert(Int64, round(prev * (num / den)))
next_num = num - 1
next_den = den + 1
end
s = push!(s, c)
if next_num > 0
calc_row(s, c, next_num, next_den)
else
s
end
end
function row(r)
s = []
calc_row(s, 0, r, 1)
end
With bigger inputs, like row(392) it dies with:
ERROR: LoadError: InexactError()
in calc_row(::Array{Any,1}, ::Int64, ::Int64, ::Int64) at /var/tmp/148-3.julia:17
in calc_row(::Array{Any,1}, ::Int64, ::Int64, ::Int64) at /var/tmp/148-3.julia:26 (repeats 10 times)
in row(::Int64) at /var/tmp/148-3.julia:34
in include_from_node1(::String) at ./loading.jl:488
in include_from_node1(::String) at /Applications/Julia-0.5.app/Contents/Resources/julia/lib/julia/sys.dylib:?
in process_options(::Base.JLOptions) at ./client.jl:262
in _start() at ./client.jl:318
in _start() at /Applications/Julia-0.5.app/Contents/Resources/julia/lib/julia/sys.dylib:?
while loading /var/tmp/148-3.julia, in expression starting on line 37
The offending line is:
c = convert(Int64, round(prev * (num / den)))
How can I avoid this error while still working with integers when computing an entry in the triangle?
Essentially, the code in the question is correct. To do large row number BigInts are better because Int64 is not enough to hold such numbers. Additionally, floating-point numbers for integer calculations are a source of trouble and rounding errors. So, a little touched up version:
function calc_row(s, prev, num, den)
c = one(den)
next_num = num
next_den = den
if prev > 0
c = (prev * num) ÷ den
next_num = num - 1
next_den = den + 1
end
s = push!(s, c)
return next_num > 0 ? calc_row(s, c, next_num, next_den) : s
end
function row{T}(r::T)
s = Vector{T}(0)
calc_row(s, zero(T), r, one(T))
end
And row(BigInt(392)) gives:
393-element Array{BigInt,1}:
1
392
76636
9962680
968870630
75184360888
4849391277276
267409290432648
12869072102071185
549080409688370560
21029779691064592448
730306894726061301376
23187243907552446318688
677780975759225353930880
18348499272339029224271680
462382181662943536451646336
10894880155433107077641916792
⋮
10894880155433107077641916792
462382181662943536451646336
18348499272339029224271680
677780975759225353930880
23187243907552446318688
730306894726061301376
21029779691064592448
549080409688370560
12869072102071185
267409290432648
4849391277276
75184360888
968870630
9962680
76636
392
1