In Julia, creating a Weights vector in statsbase - julia

I am playing a bit with Julia.
Consider this function:
function drawValues(fromDistribution, byCount)
#=
inputs:
fromDistribution :
A 2D array
Each element is an array with two elements
The first one is a value, and the second one is the probability of that value
We will draw a value out of this distribution from a random number generator
byCount :
An integer
We draw that many values from the source distribution
=#
values = []
wts = []
for i = 1:length(fromDistribution)
push!(values, fromDistribution[i][1])
push!(wts , fromDistribution[i][2])
end
w = Weights(wts)
res = []
for i = 1:byCount
r = sample(values, w)
push!(res, r)
end
plot(values, wts)
print(res)
end
This throws the error :
ERROR: MethodError: no method matching Weights(::Array{Any,1},
::Float64) Closest candidates are: Weights(::var"#18#V",
::var"#16#S") where {var"#16#S"<:Real, var"#17#T"<:Real,
var"#18#V"<:AbstractArray{var"#17#T",1}} at
/home/hedgehog/.julia/packages/StatsBase/EA8Mh/src/weights.jl:13
Weights(::Any) at
/home/hedgehog/.julia/packages/StatsBase/EA8Mh/src/weights.jl:16
Stacktrace: [1] Weights(::Array{Any,1}) at
/home/hedgehog/.julia/packages/StatsBase/EA8Mh/src/weights.jl:16 [2]
drawValues(::Array{Array{Float64,1},1}, ::Int64) at
/home/hedgehog/LASER.jl:51 [3] top-level scope at REPL[13]:1 [4]
run_repl(::REPL.AbstractREPL, ::Any) at
/build/julia/src/julia-1.5.3/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:288
It seems, that the second definition ( Weights(::Array{Any,1})) whould fit. But somehow Julia sees two input arguments?
Please help.
Version details :
Julia Version 1.5.3
Commit 788b2c77c1* (2020-11-09 13:37 UTC)
Platform Info:
OS: Linux (x86_64-pc-linux-gnu)
CPU: AMD Ryzen 7 3700X 8-Core Processor
WORD_SIZE: 64
LIBM: libopenlibm
LLVM:
libLLVM-10.0.1 (ORCJIT, znver2)

Your Vectors have elements of type any.
It should be:
wts = Float64[]
When you write wts=[] it is an equivalent of wts=Any[].
Have a look at the weight methods:
julia> methods(weights)
# 3 methods for generic function "weights":
[1] weights(vs::AbstractArray{T,1} where T<:Real) in StatsBase at c:\JuliaPkg\Julia1.5.3\packages\StatsBase\EA8Mh\src\weights.jl:76
[2] weights(vs::AbstractArray{T,N} where N where T<:Real) in StatsBase at c:\JuliaPkg\Julia1.5.3\packages\StatsBase\EA8Mh\src\weights.jl:77
[3] weights(model::StatisticalModel) in StatsBase at c:\JuliaPkg\Julia1.5.3\packages\StatsBase\EA8Mh\src\statmodels.jl:143
A container having elements of subtype of Real is required.
Similarly for the other containers providing the types is recommended as well:
value = Float64[]
res = Float64[] # or maybe Int[] depending on what your code does

Related

Modify sparse identity matrix in julia sparse

I'm creating an identity sparse matrix because I would like to modify some entries of this matrix:
Is = copy(sparse(I, N^2, N^2))
However, I got the error (e.g. for Is[1,3] = -1)
ERROR: LoadError: InexactError: Bool(-1.0)
Is there a way I can modify Is? I don't want to run two loops of size N because this parameter is very big.
Example:
julia> using SparseArrays, LinearAlgebra
julia> Is = copy(sparse(I, N^2, N^2));
julia> Is[3, 5] = -1.0;
ERROR: InexactError: Bool(-1.0)
Stacktrace:
[1] Bool
# ./float.jl:158 [inlined]
[2] convert
# ./number.jl:7 [inlined]
[3] _setindex_scalar!(A::SparseMatrixCSC{Bool, Int64}, _v::Float64, _i::Int64, _j::Int64)
# SparseArrays ~/Desktop/Julia/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/SparseArrays/src/sparsematrix.jl:2670
[4] setindex!(A::SparseMatrixCSC{Bool, Int64}, _v::Float64, _i::Int64, _j::Int64)
# SparseArrays ~/Desktop/Julia/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/SparseArrays/src/sparsematrix.jl:2667
[5] top-level scope
# REPL[11]:1
Looks like Is is expected to contain booleans, so Julia attempts to convert -1.0 to a Boolean and fails. Indeed, Is contains booleans:
julia> typeof(Is)
SparseMatrixCSC{Bool, Int64}
Why is that? Apparently, I is UniformScaling{Bool}, so I is an identity matrix of booleans. You can multiply it by 1.0 to get floats:
julia> 1.0*I
UniformScaling{Float64}
1.0*I
Then create the new matrix:
julia> Is = copy(sparse(1.0*I, N^2, N^2));
julia> eltype(Is)
Float64
julia> Is[3,5] = -1.0;

How to pass a list of parameters to workers in Julia Distributed

with Julia 1.5.3, I wanted to pass a list or parameters to the distributed workers.
I first tried in a non distributed way :
using Distributed
#everywhere begin
using SharedArrays
solve(a,b,c) = return (1,2,3)
d_rates = LinRange(0.01, 0.33, 5)
m_rates = LinRange(0.01, 0.25, 5)
population_size = 10^3
max_iterations_perloop = 10^3
nb_repeats = 2
nb_params = length(d_rates)*length(m_rates)*nb_repeats
para = enumerate(Base.product(d_rates, m_rates, population_size, max_iterations_perloop, 1:nb_repeats))
results = SharedArray{Tuple{Int, Int, Int}}(nb_params)
end
for (y , x) in para
results[y] = solve(x[1], x[2], x[3])
end
which worked fine. And then changed the final loop to:
#sync #distributed for (y , x) in para
results[y] = solve(x[1], x[2], x[3])
end
I then got an error (truncated):
ERROR: LoadError: TaskFailedException:
MethodError: no method matching firstindex(::Base.Iterators.Enumerate{Base.Iterators.ProductIterator{Tuple{LinRange{Float64},LinRange{Float64},Int64,Int64,UnitRange{Int64}}}})
Closest candidates are:
firstindex(::Cmd) at process.jl:638
firstindex(::Core.SimpleVector) at essentials.jl:599
firstindex(::Base64.Buffer) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Base64/src/buffer.jl:18
...
Stacktrace:
[1] (::Distributed.var"#159#161"{var"#271#272",Base.Iterators.Enumerate{Base.Iterators.ProductIterator{Tuple{LinRange{Float64},LinRange{Float64},Int64,Int64,UnitRange{Int64}}}}})() at ./task.jl:332
Stacktrace:
[1] sync_end(::Channel{Any}) at ./task.jl:314
[2] top-level scope at task.jl:333
[3] include_string(::Function, ::Module, ::String, ::String) at ./loading.jl:1088
[4] include_string(::Module, ::String, ::String) at ./loading.jl:1096
[5] invokelatest(::Any, ::Any, ::Vararg{Any,N} where N; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at ./essentials.jl:710
[6] invokelatest(::Any, ::Any, ::Vararg{Any,N} where N) at ./essentials.jl:709
Is it possible to pass such a list, if so how?
I assume that all your workers are on a single server and that you have actually added some workers using the addprocs command. The first problem with your code is that you create the SharedArray on all workers. Rather than that the syntax of a SharedArray is the following:
help?> SharedArray
SharedArray{T}(dims::NTuple; init=false, pids=Int[])
SharedArray{T,N}(...)
Construct a SharedArray of a bits type T and size dims across the processes specified by pids - all of which have to be on the same host. (...)
This means that you create SharedArray only once from the master worker and you can specify the workers that are aware of it using the pids argument (if you do not specify pids all worker processes have the access).
Hence your code will look like this:
using Distributed, SharedArrays
addprocs(4)
#everywhere using SharedArrays
#everywhere solve(a,b,c) = return (1,2,3)
#(...) # your setup code without #everywhere
results = SharedArray{Tuple{Int, Int, Int}}(nb_params)
#sync #distributed for (y , x) in collect(para)
results[y] = solve(x[1], x[2], x[3])
end
Note that you will need collect because #distributed macro needs to know the size of the Vector and it does not work good with iterators.

InexactError: Int64(::Float64)

I am still learning the language Julia and i have this error. I am writing an mosquito population model and i am trying to run my main function a 100 times. This main function uses many other functions to calculate the subpopulation levels.
# Importing KNMI data
xf = XLSX.readxlsx("C:/Scriptie_mosquitoes/knmi_csv.xlsx")
sh = xf["knmi_csv"]
temperature = sh["B3:B368"]
precip = sh["F3:F368"]
subpopulation_amount = 100
imat_list1 = zeros(100,length(temperature))
imat_list = Array{Float64}(imat_list1)
adul_list1 = zeros(100,length(temperature))
adul_list = Array{Float64}(adul_list1)
egg_list1 = zeros(100,length(temperature))
egg_list = Array{Float64}(egg_list1)
diaegg_list1 = zeros(100,length(temperature))
diaegg_list = Array{Float64}(diaegg_list1)
imat_list[1] = 100.0
adul_list[1] = 1000.0
egg_list[1] = 100.0
diaegg_list[1] = 100.0
for counter = 1:1:subpopulation_amount
u = Distributions.Normal()
temp_change = rand(u)
tempa = temperature .+ temp_change
println(tempa)
e = Distributions.Normal()
precip_change = rand(e)
println("hallo", precip_change)
println(counter,tempa,precip,precip_change)
main(counter,tempa::Array{Float64,2},precip::Array{Any,2},precip_change::Float64,imat_list::Array{Float64,2},adul_list::Array{Float64,2},egg_list::Array{Float64,2},diaegg_list::Array{Float64,2})
end
However i get this error which i tried to fix with all the Float64 stuf. I doesn't work unfortunatly. I hope some of you guys see the problem or can help me with understanding the error message.
ERROR: InexactError: Int64(87.39533010546728)
Stacktrace:
[1] Int64 at .\float.jl:710 [inlined]
[2] convert at .\number.jl:7 [inlined]
[3] setindex! at .\array.jl:825 [inlined]
[4] main(::Int64, ::Array{Float64,2}, ::Array{Any,2}, ::Float64, ::Array{Float64,2}, ::Array{Float64,2}, ::Array{Float64,2}, ::Array{Float64,2}) at .\REPL[905]:19
[5] top-level scope at .\REPL[938]:10
You can check the documentation for InexactError by typing ?InexactError:
help?> InexactError
search: InexactError
InexactError(name::Symbol, T, val)
Cannot exactly convert val to type T in a method of function name.
I think that explains it nicely. There is no Int64 that represents the value 87.39533010546728.
You have a variety of options available. Check their help to learn more about them:
julia> trunc(Int, 87.39533010546728)
87
julia> Int(round(87.39533010546728))
87
julia> Int(floor(87.39533010546728))
87
We do not see the code of main. However it seems that you are using values of one of the Arrays that you have as its argument to use for indexing some vector in your code. And since vector indices need to be integers it fails. Most likely some variable is in wrong place in your main - look around [] operators.
When debugging you could also try to change your Arrays to Int elements and see which change causes the problem to stop. E.g. round.(Int, tempa) etc.
The problem is just what it says: you cannot exactly represent a decimal number (87.39) as an integer.
You need to decide what you want to do here - one option is to just round() your decimal number before converting it to an integer.
It's hard to say from the code you posted where exactly the error occurs, but one potentially less obvious way for this to happen is if you try to index into an array (e.g. my_array[i]), and your calculations lead to i having a non-integer value.

error generator in Julia JuMP optimization error

I am converting a code in julia 0.6 to 1.2.
Here is the old version:
#variable(model, use[i=eachindex(n), j=1:m], Bin)
Used = [indmax(getvalue(use[i,j])
for j=1:m) for i=eachindex(n)]
I converted to the following,
#variable(model, use[i=eachindex(n), j=1:m], Bin)
JuMP.optimize!(model)
Used = [argmax(JuMP.value(use[i,j])
for j=1:m) for i=eachindex(n)]
but with error:
MethodError: no method matching keys(::Base.Generator{UnitRange{Int64},getfield(Main, Symbol("##261#266")){Int64,JuMP.Containers.SparseAxisArray{VariableRef,2,Tuple{Any,Any}}}})
Closest candidates are:
keys(!Matched::Core.SimpleVector) at essentials.jl:606
keys(!Matched::Cmd) at process.jl:963
keys(!Matched::BenchmarkTools.BenchmarkGroup) at /Users/shuaiwang/.julia/packages/BenchmarkTools/7aqwe/src/groups.jl:31
...
pairs(::Base.Generator{UnitRange{Int64},getfield(Main, Symbol("##261#266")){Int64,JuMP.Containers.SparseAxisArray{VariableRef,2,Tuple{Any,Any}}}}) at abstractdict.jl:132
_findmax(::Base.Generator{UnitRange{Int64},getfield(Main, Symbol("##261#266")){Int64,JuMP.Containers.SparseAxisArray{VariableRef,2,Tuple{Any,Any}}}}, ::Colon) at array.jl:2068
findmax(::Base.Generator{UnitRange{Int64},getfield(Main, Symbol("##261#266")){Int64,JuMP.Containers.SparseAxisArray{VariableRef,2,Tuple{Any,Any}}}}) at array.jl:2065
argmax(::Base.Generator{UnitRange{Int64},getfield(Main, Symbol("##261#266")){Int64,JuMP.Containers.SparseAxisArray{VariableRef,2,Tuple{Any,Any}}}}) at array.jl:2153
(::getfield(Main, Symbol("##260#265")){ScenarioGraph,JuMP.Containers.SparseAxisArray{VariableRef,2,Tuple{Any,Any}}})(::Int64) at none:0
iterate at generator.jl:47 [inlined]
collect at array.jl:606 [inlined]
The problem seems to be unrelated to JuMP. The fix to your code is:
Used = [argmax([JuMP.value(use[i,j]) for j=1:m]) for i=eachindex(n)]
(I have not tested the whole code as it was not complete)
And the core of the issue is that you are not allowed to use argmax on generators, you have to pass a collection that supports pairs to it, e.g.:
julia> argmax(i for i in 1:3)
ERROR: MethodError: no method matching keys(::Base.Generator{UnitRange{Int64},getfield(Main, Symbol("##15#16"))})
fails, but
julia> argmax([i for i in 1:3])
3
julia> argmax((1,2,3))
3
julia> argmax((a=1,b=2,c=3))
:c
julia> argmax(Dict(:a=>1,:b=>2,:c=>3))
:c
work

How do i convert a SymPy expression into polynomials?

I want to change a SymPy expression, for example x+y ( x=symbols("x") and the same thing for y), into a polynomial, and then get the generators of this polynomial and the length of this polynomial.
I've tried
op=x+y
op = op[:as_poly](domain="C")
op_a = op.x[:gens]
nab = op[:length]()
but it seems that it doesn't work .
The error that i'm getting is this:
ERROR: MethodError: no method matching getindex(::Sym, ::Symbol)
Closest candidates are:
getindex(::Sym, ::Sym...) at /Users/midow/.julia/packages/SymPy/1Cwgd/src/utils.jl:18
getindex(::Number) at number.jl:75
getindex(::Number, ::Integer) at number.jl:77
...
Stacktrace:
[1] top-level scope at REPL[11]:1
As #phipsgabler mentioned, the interface changed from getindex to getproperty so
using SymPy
x=symbols("x")
y=symbols("y")
op = op.as_poly(domain="C")
op_a = op.x.gens
nab = op.length()
should give you what you want.

Resources