I am new to Julia, so please forgive me if this is too basic. I am trying to run the following script.
using Pkg
Pkg.add("DataFrames")
using DataFrames
function LS(x,y,a)
T = size(x,1)
N = size(x,2)
pred = fill(0.0,T)
w= fill(0.0,N)
for t = 1:T
x1 = x[t,:]
pred[t] = transpose(w) * x1
err = a*(y[t]-pred[t])
w = w + (err * x1)
end
return pred
end
input = readtable("input.csv")
output = readtable("label.csv")
en = convert(Array, input)
out = convert(Array, output)
a = 0.5
prediction = LS(en,out,a)
print(prediction)
When I run I get the following error:
ERROR: LoadError: MethodError: no method matching
*(::Array{Float64,1}, ::Array{Union{Missing, Float64},1}) Closest candidates are: *(::Any, ::Any, !Matched::Any, !Matched::Any...) at
operators.jl:502 *(!Matched::LinearAlgebra.Adjoint{#s571,#s570}
where #s570<:Union{DenseArray{T<:Union{Complex{Float32},
Complex{Float64}, Float32, Float64},2},
ReinterpretArray{T<:Union{Complex{Float32}, Complex{Float64}, Float32,
Float64},2,S,A} where S where A<:Union{SubArray{T,N,A,I,true} where
I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where A<:DenseArray
where N where T, DenseArray}, ReshapedArray{T<:Union{Complex{Float32},
Complex{Float64}, Float32, Float64},2,A,MI} where
MI<:Tuple{Vararg{SignedMultiplicativeInverse{Int64},N} where N} where
A<:Union{ReinterpretArray{T,N,S,A} where S where
A<:Union{SubArray{T,N,A,I,true} where
I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where A<:DenseArray
where N where T, DenseArray} where N where T, SubArray{T,N,A,I,true}
where I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where
A<:DenseArray where N where T, DenseArray},
SubArray{T<:Union{Complex{Float32}, Complex{Float64}, Float32,
Float64},2,A,I,L} where L where I<:Tuple{Vararg{Union{Int64,
AbstractRange{Int64}, AbstractCartesianIndex},N} where N} where
A<:Union{ReinterpretArray{T,N,S,A} where S where
A<:Union{SubArray{T,N,A,I,true} where
I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where A<:DenseArray
where N where T, DenseArray} where N where T, ReshapedArray{T,N,A,MI}
where MI<:Tuple{Vararg{SignedMultiplicativeInverse{Int64},N} where N}
where A<:Union{ReinterpretArray{T,N,S,A} where S where
A<:Union{SubArray{T,N,A,I,true} where
I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where A<:DenseArray
where N where T, DenseArray} where N where T, SubArray{T,N,A,I,true}
where I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where
A<:DenseArray where N where T, DenseArray} where N where T,
DenseArray}} where #s571, ::Union{DenseArray{S,1},
ReinterpretArray{S,1,S,A} where S where
A<:Union{SubArray{T,N,A,I,true} where
I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where A<:DenseArray
where N where T, DenseArray}, ReshapedArray{S,1,A,MI} where
MI<:Tuple{Vararg{SignedMultiplicativeInverse{Int64},N} where N} where
A<:Union{ReinterpretArray{T,N,S,A} where S where
A<:Union{SubArray{T,N,A,I,true} where
I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where A<:DenseArray
where N where T, DenseArray} where N where T, SubArray{T,N,A,I,true}
where I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where
A<:DenseArray where N where T, DenseArray}, SubArray{S,1,A,I,L} where
L where I<:Tuple{Vararg{Union{Int64, AbstractRange{Int64},
AbstractCartesianIndex},N} where N} where
A<:Union{ReinterpretArray{T,N,S,A} where S where
A<:Union{SubArray{T,N,A,I,true} where
I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where A<:DenseArray
where N where T, DenseArray} where N where T, ReshapedArray{T,N,A,MI}
where MI<:Tuple{Vararg{SignedMultiplicativeInverse{Int64},N} where N}
where A<:Union{ReinterpretArray{T,N,S,A} where S where
A<:Union{SubArray{T,N,A,I,true} where
I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where A<:DenseArray
where N where T, DenseArray} where N where T, SubArray{T,N,A,I,true}
where I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where
A<:DenseArray where N where T, DenseArray} where N where T,
DenseArray}}) where {T<:Union{Complex{Float32}, Complex{Float64},
Float32, Float64}, S} at
/Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/LinearAlgebra/src/matmul.jl:98
*(!Matched::LinearAlgebra.Adjoint{#s571,#s570} where #s570<:LinearAlgebra.AbstractTriangular where #s571, ::AbstractArray{T,1} where T) at
/Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/LinearAlgebra/src/triangular.jl:1805
...
Stacktrace: 2 LS(::Array{Union{Missing, Float64},2},
::Array{Union{Missing, Float64},2}, ::Float64) at
/Users/wj/Desktop/Julia/NLSR.jl:16
2 top-level scope at none:0
[3] include at ./boot.jl:317 [inlined]
[4] include_relative(::Module, ::String) at ./loading.jl:1044
[5] include(::Module, ::String) at ./sysimg.jl:29
[6] exec_options(::Base.JLOptions) at ./client.jl:231
[7] _start() at ./client.jl:425
How can I successfully run this script?
Remark:
The algorithm pseudo code is as follows:
Script updated after the comment by Warren
The code works now!
I'm not sure what you're trying to do here, but it doesn't look anything like least squares to me.
Having said that, the problem in your script is in this line here:
w = w + (err * x1)
x1 and err will both be vectors, and there is no multiplication operator method for two vectors. For example, rand(2) * rand(2) will also error. This is because usage of * is ambiguous in this situation.
If you want the dot product, use dot(x1, err).
If you want the element-wise product, use x1 .* err, which broadcasts * across the elements of the inputs.
There are several other issues with your code, but I'm hesitant to try and correct them, since, as stated, I'm not really sure what you're trying to do.
Related
I'm trying to solve a life cycle problem in economics using Julia but I'm having trouble with NLsolve. The model boils down to trying to solve two a two equation system to find optimal leisure hours and capital stock each working period. The economic agent after retirement sets leisure = 1 and I only need to solve a single non linear equation for capital. This part works fine. It's solving the two equation system that seems to break down.
As I'm fairly new to Julia / programming in general so any advice would be very helpful. Also advice / points / recommendations on all aspects of the code will be greatly appreciated. The model is solved backwards from the final time period.
My attempt
using Parameters
using Roots
using Plots
using NLsolve
using ForwardDiff
Model = #with_kw (α = 0.66,
δ = 0.02,
τ = 0.015,
β = 1/1.01,
T = 70,
Ret = 40,
);
function du_c(c, l, η=2, γ=2)
if c>0 && l>0
return (c+1e-6)^(-η) * l^((1-η)*γ)
else
return Inf
end
end
function du_l(c, l, η=2, γ=2)
if l>0 && c>0
return γ * (c+1e-6)^(1-η) * l^(γ*(1-η)-1)
else
return Inf
end
end
function create_euler_work(x, y, m, k, l, r, w, t)
# x = todays capital, y = leisure
#unpack α, β, τ, δ, T, Ret = m
c_1 = x*(1+r) + (1-τ)*w*(1-y) - k[t+1]
c_2 = k[t+1]*(1+r) + (1-τ)*w*(1-l[t+1]) - k[t+2]
return du_c(c_1,y) - β*(1+r)*du_c(c_2,l[t+1])
end
function create_euler_retire(x, m, k, r, b, t)
# Holds at time periods Ret onwards
#unpack α, β, τ, δ, T, Ret = m
c_1 = x*(1+r) + b - k[t+1]
c_2 = k[t+1]*(1+r) + b - k[t+2]
return du_c(c_1,1) - β*(1+r)*du_c(c_2,1)
end
function create_euler_lyw(x, y, m, k, r, w, b, t)
# x = todays capital, y = leisure
#unpack α, β, τ, δ, T, Ret = m
c_1 = x*(1+r) + (1-τ)*w*(1-y) - k[t+1]
c_2 = k[t+1]*(1+r) + b - k[t+2]
return du_c(c_1,y) - β*(1+r)*du_c(c_2,1)
end
function create_foc(x, y, m, k, r, w, t)
# x = todays capital, l= leisure
#unpack α, β, τ, δ, T = m
c = x*(1+r) + (1-τ)*w*(1-y) - k[t+1]
return du_l(c,y) - (1-τ)*w*du_c(c,y)
end
function life_cycle(m, guess, r, w, b, initial)
#unpack α, β, τ, δ, T, Ret = m
k = zeros(T+1);
l = zeros(T);
k[T] = guess
println("Period t = $(T+1) Retirment, k = $(k[T+1]), l.0 = NA")
println("Period t = $T Retirment, k = $(k[T]), l = 1.0")
########################## Retirment ################################
for t in T-1:-1:Ret+1
euler(x) = create_euler_retire(x, m, k, r, b, t)
k[t] = find_zero(euler, (0,100))
l[t] = 1
println("Period t = $t Retirment, k = $(k[t]), l = $(l[t])")
end
###################### Retirement Year #############################
for t in Ret:Ret
euler(x,y) = create_euler_lyw(x, y, m, k, r, w, b, t)
foc(x,y) = create_foc(x, y, m, k, r, w, t)
function f!(F, x)
F[1] = euler(x[1], x[2])
F[2] = foc(x[1], x[2])
end
res = nlsolve(f!, [5; 0.7], autodiff = :forward)
k[t] = res.zero[1]
l[t] = res.zero[2]
println("Period t = $t Working, k = $(k[t]), l = $(l[t])")
end
############################ Working ###############################
for t in Ret-1:-1:1
euler(x,y) = create_euler_work(x, y, m, k, l, r, w, t)
foc(x,y) = create_foc(x, y, m, k, r, w, t)
function f!(F, x)
F[1] = euler(x[1], x[2])
F[2] = foc(x[1], x[2])
end
res = nlsolve(f!, [5; 0.7], autodiff = :forward)
k[t] = res.zero[1]
l[t] = res.zero[2]
println("Period t = $t Working, k = $(k[t]), l = $(l[t])")
end
#####################################################################
return k[1] - initial, k, l
end
m = Model();
residual, k, l = life_cycle(m, 0.3, 0.03, 1.0, 0.0, 0.0)
The code seems to break on period 35 with the error "During the resolution of the nonlinear system, the evaluation of following equations resulted in a non-finite number: [1,2]" However the solutions seem to go weird at period 37.
I am trying to use maximum likelihood to estimate the normal linear model in Julia. I have used the following code to simulate the process with just an intercept and an anonymous function per the Optim documentation regarding values that do not change:
using Optim
nobs = 500
nvar = 1
β = ones(nvar)*3.0
x = [ones(nobs) randn(nobs,nvar-1)]
ε = randn(nobs)*0.5
y = x*β + ε
function LL_anon(X, Y, β, σ)
-(-length(X)*log(2π)/2 - length(X)*log(σ) - (sum((Y - X*β).^2) / (2σ^2)))
end
LL_anon(X,Y, pars) = LL_anon(X,Y, pars...)
res2 = optimize(vars -> LL_anon(x,y, vars...), [1.0,1.0]) # Start values: β=1.0, σ=1.0
This actually recovered the parameters and I received the following output:
* Algorithm: Nelder-Mead
* Starting Point: [1.0,1.0]
* Minimizer: [2.980587812647935,0.5108406803949835]
* Minimum: 3.736217e+02
* Iterations: 47
* Convergence: true
* √(Σ(yᵢ-ȳ)²)/n < 1.0e-08: true
* Reached Maximum Number of Iterations: false
* Objective Calls: 92
However, when I try and set nvar = 2, i.e. an intercept plus an additional covariate, I get the following error message:
MethodError: no method matching LL_anon(::Array{Float64,2}, ::Array{Float64,1}, ::Float64, ::Float64, ::Float64)
Closest candidates are:
LL_anon(::Any, ::Any, ::Any, ::Any) at In[297]:2
LL_anon(::Array{Float64,1}, ::Array{Float64,1}, ::Any, ::Any) at In[113]:2
LL_anon(::Any, ::Any, ::Any) at In[297]:4
...
Stacktrace:
[1] (::##245#246)(::Array{Float64,1}) at .\In[299]:1
[2] value!!(::NLSolversBase.NonDifferentiable{Float64,Array{Float64,1},Val{false}}, ::Array{Float64,1}) at C:\Users\dolacomb\.julia\v0.6\NLSolversBase\src\interface.jl:9
[3] initial_state(::Optim.NelderMead{Optim.AffineSimplexer,Optim.AdaptiveParameters}, ::Optim.Options{Float64,Void}, ::NLSolversBase.NonDifferentiable{Float64,Array{Float64,1},Val{false}}, ::Array{Float64,1}) at C:\Users\dolacomb\.julia\v0.6\Optim\src\multivariate/solvers/zeroth_order\nelder_mead.jl:136
[4] optimize(::NLSolversBase.NonDifferentiable{Float64,Array{Float64,1},Val{false}}, ::Array{Float64,1}, ::Optim.NelderMead{Optim.AffineSimplexer,Optim.AdaptiveParameters}, ::Optim.Options{Float64,Void}) at C:\Users\dolacomb\.julia\v0.6\Optim\src\multivariate/optimize\optimize.jl:25
[5] #optimize#151(::Array{Any,1}, ::Function, ::Tuple{##245#246}, ::Array{Float64,1}) at C:\Users\dolacomb\.julia\v0.6\Optim\src\multivariate/optimize\interface.jl:62
[6] #optimize#148(::Array{Any,1}, ::Function, ::Function, ::Array{Float64,1}) at C:\Users\dolacomb\.julia\v0.6\Optim\src\multivariate/optimize\interface.jl:52
[7] optimize(::Function, ::Array{Float64,1}) at C:\Users\dolacomb\.julia\v0.6\Optim\src\multivariate/optimize\interface.jl:52
I'm not sure why adding an additional variable is causing this issue but it seems like a type instability problem.
The second issue is that when I use my original working example and set the starting values to [2.0,2.0], I get the following error message:
log will only return a complex result if called with a complex argument. Try log(complex(x)).
Stacktrace:
[1] nan_dom_err at .\math.jl:300 [inlined]
[2] log at .\math.jl:419 [inlined]
[3] LL_anon(::Array{Float64,2}, ::Array{Float64,1}, ::Float64, ::Float64) at .\In[302]:2
[4] (::##251#252)(::Array{Float64,1}) at .\In[304]:1
[5] value(::NLSolversBase.NonDifferentiable{Float64,Array{Float64,1},Val{false}}, ::Array{Float64,1}) at C:\Users\dolacomb\.julia\v0.6\NLSolversBase\src\interface.jl:19
[6] update_state!(::NLSolversBase.NonDifferentiable{Float64,Array{Float64,1},Val{false}}, ::Optim.NelderMeadState{Array{Float64,1},Float64,Array{Float64,1}}, ::Optim.NelderMead{Optim.AffineSimplexer,Optim.AdaptiveParameters}) at C:\Users\dolacomb\.julia\v0.6\Optim\src\multivariate/solvers/zeroth_order\nelder_mead.jl:193
[7] optimize(::NLSolversBase.NonDifferentiable{Float64,Array{Float64,1},Val{false}}, ::Array{Float64,1}, ::Optim.NelderMead{Optim.AffineSimplexer,Optim.AdaptiveParameters}, ::Optim.Options{Float64,Void}, ::Optim.NelderMeadState{Array{Float64,1},Float64,Array{Float64,1}}) at C:\Users\dolacomb\.julia\v0.6\Optim\src\multivariate/optimize\optimize.jl:51
[8] optimize(::NLSolversBase.NonDifferentiable{Float64,Array{Float64,1},Val{false}}, ::Array{Float64,1}, ::Optim.NelderMead{Optim.AffineSimplexer,Optim.AdaptiveParameters}, ::Optim.Options{Float64,Void}) at C:\Users\dolacomb\.julia\v0.6\Optim\src\multivariate/optimize\optimize.jl:25
[9] #optimize#151(::Array{Any,1}, ::Function, ::Tuple{##251#252}, ::Array{Float64,1}) at C:\Users\dolacomb\.julia\v0.6\Optim\src\multivariate/optimize\interface.jl:62
Again, I’m not sure why this is happening and since start values are very important I’d like to know how to overcome this issue and they are not too far off from the true values.
Any help would be greatly appreciated!
Splatting causes the problem. E.g. it transforms [1, 2, 3] into three parameters while your function accepts only two.
Use the following call:
res2 = optimize(vars -> LL_anon(x,y, vars[1:end-1], vars[end]), [1.0,1.0,1.0])
and you can remove the following line from your code
LL_anon(X,Y, pars) = LL_anon(X,Y, pars...)
or replace it with:
LL_anon(X,Y, pars) = LL_anon(X,Y, pars[1:end-1], pars[end])
but it would not be called by optimization routine unless you change a call to:
res2 = optimize(vars -> LL_anon(x,y, vars), [1.0,1.0,1.0])
Finally - to get good performance of this code I would recommend to wrap it all in a function.
EDIT: now I see a second question. The reason is that σ can become negative in the optimization process and then log(σ) fails. The simplest thing to do in this case is to take log(abs(σ))) like this:
function LL_anon(X, Y, β, σ)
-(-length(X)*log(2π)/2 - length(X)*log(abs(σ)) - (sum((Y - X*β).^2) / (2σ^2)))
end
Of course then you have to take absolute value of σ as a solution as you might get a negative value from optimization routine.
A cleaner way would be to optimize over e.g. log(σ) not σ like this:
function LL_anon(X, Y, β, logσ)
-(-length(X)*log(2π)/2 - length(X)*logσ - (sum((Y - X*β).^2) / (2(exp(logσ))^2)))
end
but then you have to use exp(logσ) to recover σ after optimization finishes.
I have asked around regarding this and have another option. The main reason for looking at this problem is twofold. One, to learn how to use the optimization routines in Julia in a canonical situation and two, to expand this to spatial econometric models. With that in mind, I'm posting the other suggested code from the Julia message board so that others may see another solution.
using Optim
nobs = 500
nvar = 2
β = ones(nvar) * 3.0
x = [ones(nobs) randn(nobs, nvar - 1)]
ε = randn(nobs) * 0.5
y = x * β + ε
function LL_anon(X, Y, β, log_σ)
σ = exp(log_σ)
-(-length(X) * log(2π)/2 - length(X) * log(σ) - (sum((Y - X * β).^2) / (2σ^2)))
end
opt = optimize(vars -> LL_anon(x,y, vars[1:nvar], vars[nvar + 1]),
ones(nvar+1))
# Use forward autodiff to get first derivative, then optimize
fun1 = OnceDifferentiable(vars -> LL_anon(x, y, vars[1:nvar], vars[nvar + 1]),
ones(nvar+1))
opt1 = optimize(fun1, ones(nvar+1))
Results of Optimization Algorithm
Algorithm: L-BFGS
Starting Point: [1.0,1.0,1.0]
Minimizer: [2.994204150985705,2.9900626550063305, …]
Minimum: 3.538340e+02
Iterations: 12
Convergence: true
|x - x’| ≤ 1.0e-32: false
|x - x’| = 8.92e-12
|f(x) - f(x’)| ≤ 1.0e-32 |f(x)|: false
|f(x) - f(x’)| = 9.64e-16 |f(x)|
|g(x)| ≤ 1.0e-08: true
|g(x)| = 6.27e-09
Stopped by an increasing objective: true
Reached Maximum Number of Iterations: false
Objective Calls: 50
Gradient Calls: 50
opt1.minimizer
3-element Array{Float64,1}:
2.9942
2.99006
-1.0651 #Note: needs to be exponentiated
# Get Hessian, use Newton!
fun2 = TwiceDifferentiable(vars -> LL_anon(x, y, vars[1:nvar], vars[nvar + 1]),
ones(nvar+1))
opt2 = optimize(fun2, ones(nvar+1))
Results of Optimization Algorithm
Algorithm: Newton’s Method
Starting Point: [1.0,1.0,1.0]
Minimizer: [2.99420415098702,2.9900626550079026, …]
Minimum: 3.538340e+02
Iterations: 9
Convergence: true
|x - x’| ≤ 1.0e-32: false
|x - x’| = 1.36e-11
|f(x) - f(x’)| ≤ 1.0e-32 |f(x)|: false
|f(x) - f(x’)| = 1.61e-16 |f(x)|
|g(x)| ≤ 1.0e-08: true
|g(x)| = 6.27e-09
Stopped by an increasing objective: true
Reached Maximum Number of Iterations: false
Objective Calls: 45
Gradient Calls: 45
Hessian Calls: 9
fieldnames(fun2)
13-element Array{Symbol,1}:
:f
:df
:fdf
:h
:F
:DF
:H
:x_f
:x_df
:x_h
:f_calls
:df_calls
:h_calls
opt2.minimizer
3-element Array{Float64,1}:
2.98627
3.00654
-1.11313
numerical_hessian = (fun2.H) #.H is the numerical Hessian
3×3 Array{Float64,2}:
64.8715 -9.45045 0.000121521
-0.14568 66.4507 0.0
1.87326e-6 4.10675e-9 44.7214
From here, one can use the numerical Hessian to obtain the standard errors for the estimates and form t-statistics, etc. for their own functions.
Again, thank you for providing an answer and I hope people find this information useful.
I am interested in calculating quantity:
where x_i is a 1xD vector (one out of my N data of dimension D), μ is a DxK matrix and W is a list of K DxD matrices.
This should result in a 1XK vector. I try it for all N and K in the following way that works:
res = zeros(N,K);
for i in 1:N
for k in 1:K
res[i,k] = (x_matrix[i,:]-mus_matrix[:,k])'*
w_matrix[k]*(x_matrix[i,:]-mus_matrix[:,k])
If I try to vectorize it, using the following:
res = zeros(N,K);
for i in 1:N
res[i,:] = (x_matrix[i,:].-mus_matrix)'.*w_matrix.*(x_matrix[i,:].-mus_matrix)
I get the following error:
ERROR: DimensionMismatch("arrays could not be broadcast to a common size")
Stacktrace:
[1] _bcs1(::Base.OneTo{Int64}, ::Base.OneTo{Int64}) at ./broadcast.jl:70
[2] _bcs at ./broadcast.jl:63 [inlined]
[3] broadcast_shape(::Tuple{Base.OneTo{Int64},Base.OneTo{Int64}}, ::Tuple{Base.OneTo{Int64}}, ::Tuple{Base.OneTo{Int64},Base.OneTo{Int64}}, ::Vararg{Tuple{Base.OneTo{Int64},Base.OneTo{Int64}},N} where N) at ./broadcast.jl:57 (repeats 3 times)
[4] broadcast_indices(::Array{Float64,2}, ::Array{Any,1}, ::Array{Float64,1}, ::Vararg{Any,N} where N) at ./broadcast.jl:53
[5] broadcast_c(::Function, ::Type{Array}, ::Array{Float64,2}, ::Array{Any,1}, ::Vararg{Any,N} where N) at ./broadcast.jl:311
[6] broadcast(::Function, ::Array{Float64,2}, ::Array{Any,1}, ::Array{Float64,1}, ::Vararg{Any,N} where N) at ./broadcast.jl:434
Here is an example:
julia> N = 5
5
julia> D=2
2
julia> K = 4
4
julia> W=[]
0-element Array{Any,1}
julia> x = rand(N,D)
5×2 Array{Float64,2}:
0.576477 0.9575
0.184454 0.660436
0.470267 0.729649
0.648879 0.782561
0.626453 0.111332
julia> mu = rand(K,D)
4×2 Array{Float64,2}:
0.989281 0.00126782
0.659106 0.66136
0.50843 0.289442
0.327962 0.523229
julia> for i in 1:K
push!(W,rand(D,D))
end
And then run
julia> (x_matrix[i,:]-mus_matrix[:,k])'*
w_matrix[k]*(x_matrix[i,:]-mus_matrix[:,k])
34649.850360744866
But with the second code
julia> (x_matrix[i,:].-mus_matrix)'.*w_matrix.*(x_matrix[i,:].-mus_matrix)
ERROR: DimensionMismatch("arrays could not be broadcast to a common size")
Stacktrace:
[1] _bcs1(::Base.OneTo{Int64}, ::Base.OneTo{Int64}) at ./broadcast.jl:70
[2] _bcs at ./broadcast.jl:63 [inlined]
[3] broadcast_shape(::Tuple{Base.OneTo{Int64},Base.OneTo{Int64}}, ::Tuple{Base.OneTo{Int64}}, ::Tuple{Base.OneTo{Int64},Base.OneTo{Int64}}, ::Vararg{Tuple{Base.OneTo{Int64},Base.OneTo{Int64}},N} where N) at ./broadcast.jl:57 (repeats 3 times)
[4] broadcast_indices(::Array{Float64,2}, ::Array{Any,1}, ::Array{Float64,1}, ::Vararg{Any,N} where N) at ./broadcast.jl:53
[5] broadcast_c(::Function, ::Type{Array}, ::Array{Float64,2}, ::Array{Any,1}, ::Vararg{Any,N} where N) at ./broadcast.jl:311
[6] broadcast(::Function, ::Array{Float64,2}, ::Array{Any,1}, ::Array{Float64,1}, ::Vararg{Any,N} where N) at ./broadcast.jl:434
TL/DR: optimized variant below, but Einsum looks nicer, IMHO.
Looks like a case for using Einstein summation notation. In Julia, Einsum.jl can do this:
julia> N = 5
5
julia> D = 3
3
julia> K = 10
10
julia> x = rand(N, D)
5×3 Array{Float64,2}:
0.587436 0.210529 0.261725
0.527269 0.457477 0.482939
0.52726 0.411209 0.138872
0.89107 0.464789 0.758392
0.885267 0.931014 0.672959
julia> μ = rand(D, K)
3×10 Array{Float64,2}:
0.280792 0.265066 0.81437 0.503377 0.0717916 … 0.275872 0.609961 0.0820088 0.0042564
0.0177643 0.0959438 0.563948 0.332433 0.088527 0.691971 0.0296638 0.604488 0.956057
0.668128 0.444816 0.74203 0.518232 0.48689 0.465067 0.117469 0.729514 0.109973
julia> W = rand(K, D, D)
10×3×3 Array{Float64,3}:
[:, :, 1] =
0.320861 0.662103 0.219234
0.780944 0.769377 0.566203
0.466207 0.428527 0.330901
0.15534 0.035435 0.346737
0.810676 0.328116 0.469505
0.676575 0.668204 0.285334
0.455551 0.211295 0.85295
0.229995 0.741487 0.783361
0.0937583 0.401419 0.47032
0.956335 0.434213 0.967791
[:, :, 2] =
0.275903 0.130298 0.184485
0.941648 0.940107 0.439454
0.425292 0.252654 0.797115
0.0203406 0.594075 0.484809
0.164309 0.941597 0.455314
0.73628 0.109502 0.920664
0.906305 0.177235 0.540193
0.360038 0.0486971 0.20626
0.914357 0.699901 0.295872
0.284143 0.659117 0.291479
[:, :, 3] =
0.138311 0.921371 0.353719
0.345247 0.70865 0.246736
0.361364 0.636543 0.343837
0.752149 0.581561 0.346399
0.705888 0.24765 0.703952
0.992327 0.369668 0.109407
0.341624 0.223715 0.970667
0.762169 0.94248 0.917569
0.0367128 0.589345 0.121106
0.826602 0.692111 0.229499
julia> using Einsum
julia> #einsum r[n,k] := (x[n,i] - μ[i,k]) * W[k,i,j] * (x[n,j] - μ[j,k])
julia> r
5×10 Array{Float64,2}:
0.0176889 0.087092 0.522184 0.0417967 … -0.0430999 0.041266 -0.0596579 0.432076
0.0521066 0.364059 0.181515 0.00434307 -0.0248712 0.226976 -0.0686294 0.437169
-0.0472136 0.127803 0.458812 0.0119074 0.0391649 -0.0190299 -0.0585371 0.264379
0.468634 1.16498 -0.00263205 0.192809 0.273537 1.13787 -0.0653081 1.41321
0.749655 2.20266 0.0205068 0.420249 0.573358 1.42499 0.441232 1.67574
Which #macroexpands to essentially the following loops (plus preparation and bounds checking):
begin
local k
for k = 1:size(μ, 2)
begin
local n
for n = 1:size(x, 1)
begin
local s = zero(T)
begin
local j
for j = 1:size(W, 3)
begin
local i
for i = 1:size(x, 2)
s += (x[n, i] - μ[i, k]) * W[k, i, j] * (x[n, j] - μ[j, k])
end
end
end
end
r[n, k] = s
end
end
end
end
end
Now, to find something more performant, I compared a couple of variants using BenchmarkTools.jl. You can see the full code and results on my laptop here. It shows that the Einsum variant already is in fact better than the original:
# Original:
# memory estimate: 1017.73 MiB
# allocs estimate: 3429967
# median time: 361.982 ms (15.94% GC)
# Einsum:
# memory estimate: 2.64 MiB
# allocs estimate: 76
# median time: 127.536 ms (0.00% GC)
By far the most efficient and least allocating variant is the following, which requires x = x' and W = permutedims(W, [2, 3, 1]) (assuming you can change your representation easily):
function test_optimized!(res, x, μ, W)
z = zero(eltype(x))
for k = 1:size(μ, 1)
for n = 1:size(x, 1)
res[n, k] = z
for i = 1:size(W, 1)
for j = 1:size(W, 2)
#inbounds res[n, k] += (x[i, n] - μ[i, k]) * W[i, j, k] * (x[j, n] - μ[j, k])
end
end
end
end
end
function test_optimized(x, μ, W)
res = zeros(N, K)
test_optimized!(res, x, μ, W)
res
end
This brings us down to
# memory estimate: 2.63 MiB
# allocs estimate: 2
# median time: 521.215 μs (0.00% GC)
It uses a couple of "tricks" that can be found in the docs: filling a preallocated matrix in a separate method, accessing strides in column-major order, and using #inbounds (although that only improves things at the order of a microsecond).
There is also TensorOperations.jl, which I think does more intelligent things under the hood, but it fail on this:
julia> #tensor r[n,k] := (x[n,i] - μ[i,k]) * W[k,i,j] * (x[n,j] - μ[j,k])
ERROR: TensorOperations.IndexError{String}("invalid index specification: (:n, :i) to (:i, :k)")
Stacktrace:
[1] add_indices(::Tuple{Symbol,Symbol}, ::Tuple{Symbol,Symbol}) at /home/philipp/.julia/v0.6/TensorOperations/src/implementation/indices.jl:22
[2] + at /home/philipp/.julia/v0.6/TensorOperations/src/indexnotation/sum.jl:40 [inlined]
[3] -(::TensorOperations.IndexedObject{(:n, :i),:N,Array{Float64,2},Int64}, ::TensorOperations.IndexedObject{(:i, :k),:N,Array{Float64,2},Int64}) at /home/philipp/.julia/v0.6/TensorOperations/src/indexnotation/sum.jl:44
I guess that's deliberate and has to do with efficiency, see this issue.
I have this code and I want to run it but I got some error, I think It's about type of my data but I can't understand how should I write it for preventing it.
function dacmm(i0::Int64, i1::Int64, j0::Int64, j1::Int64,
k0::Int64, k1::Int64, A::Int64, B::Int64, c::Int64, n::Int64, basecase::Int64)
## A, B, C are matrices
## We compute C = A * B
if n > basecase
n = n/2
dacmm(i0, i1, j0, j1, k0, k1, A, B, c, n, basecase)
dacmm(i0, i1, j0, j1+n, k0, k1+n, A, B, c, n, basecase)
dacmm(i0+n, i1, j0, j1, k0+n, k1, A, B, c, n, basecase)
dacmm(i0+n, i1, j0, j1+n, k0+n, k1+n, A, B, c, n, basecase)
dacmm(i0, i1+n, j0+n, j1, k0, k1, A, B, C, n, basecase)
dacmm(i0, i1+n, j0+n, j1+n, k0, k1+n, A, B, c, n, basecase)
dacmm(i0+n, i1+n, j0+n, j1, k0+n, k1, A, B, c, n, basecase)
dacmm(i0+n, i1+n, j0+n, j1+n, k0+n, k1+n, A, B, c, n, basecase)
else
for i= 1:n, j=1:n, k=1:n
c[i+k0,k1+j] = c[i+k0,k1+j] + A[i+i0,i1+k] * B[k+j0,j1+j]
end
end
end
n=4;
basecase = 2;
A = [rem(rand(Int32),5) for i =1:n, j = 1:n];
B = [rem(rand(Int32),5) for i =1:n, j = 1:n];
C = zeros(Int32,n,n);
error:
ArgumentError: invalid index: 1.0
Stacktrace:
[1] to_indices at ./indices.jl:215 [inlined]
[2] to_indices at ./indices.jl:213 [inlined]
[3] getindex at ./abstractarray.jl:882 [inlined]
[4] dacmm(::Int64, ::Int64, ::Int64, ::Int64, ::Int64, ::Int64, ::Array{Int64,2}, ::Array{Int64,2}, ::Array{Int32,2}, ::Float64, ::Int64) at ./In[24]:16
[5] dacmm(::Int64, ::Int64, ::Int64, ::Int64, ::Int64, ::Int64, ::Array{Int64,2}, ::Array{Int64,2}, ::Array{Int32,2}, ::Int64, ::Int64) at ./In[24]:6
[6] include_string(::String, ::String) at ./loading.jl:515
As the stack trace points out, you tried to call a method of the dacmm function which takes a Float64 in the penultimate argument:
|
V
[4] dacmm(::Int64, ::Int64, ... ::Float64, ::Int64) at ./In[24]:16
[5] dacmm(::Int64, ::Int64, ... ::Int64, ::Int64) at ./In[24]:6
but no such method is available. You ended up there because n = n/2 returned a float not an integer.
The problem doesn't occur in the original code because there, the function's arguments weren't too restricted with type information.
I've been stuck on this problem for a while now, and can't think of how to solve it.
Consider the following function f : N → N .
f(0) = 2, f(1) = 0, f(2) = 3,
f(n) = 3f(n-3) + 2f(n-2) - f(n-1) for n≥3.
Define an iterative version of f.
I know my solution should look something like this
fun myFun 0 = 2
| myFun 1 = 0
| myFun 2 = 3
| myFun n =
let
(* code *)
in
funHelp(3,2,0,n)
end ;
I know iterative functions are only suppose to use one recursive call, while letting the arguments do all the work. I can't figure out how to do it with this problem, though! Any help would be much appreciated!
I assume this is homework, so I just give you a partial solution:
fun f n =
let
fun iter(0, n0, n1, n2) = n0
| iter(1, n0, n1, n2) = n1
| iter(2, n0, n1, n2) = n2
| iter(n, n0, n1, n2) = iter(n - 1, n1, n2, ???)
in
iter(n, 2, 0, 3)
end
Filling in the ??? shouldn't be too hard.
fun funHelp 0 = (2,0,3)
| funHelp n = let val (x,y,z) = funHelp n - 1
in
(y,z,(3 * x) + (2 * y) - z)
end
fun myFun n = let val (x,_,_) = funHelp n
in
x
end
This should do what you seem to want, if a bit messily.