I would like to run the function f() on all my 4 processors (intel i7) and fetch the rands sum, as follows:
using Distributed;
#everywhere function f()
return sum(rand(10000))
end
#sync for w in workers()
#async begin
res = #spawnat w f()
values[w-1] = fetch(res)
end
end
But, getting the following error:
ERROR: TaskFailedException
nested task error: MethodError: no method matching setindex!(::typeof(values), ::Float64, ::Int64)
Stacktrace:
[1] macro expansion
# ./REPL[58]:4 [inlined]
[2] (::var"#68#70"{Channel{Any}, Int64})()
# Main ./task.jl:411
Stacktrace:
[1] sync_end(c::Channel{Any})
# Base ./task.jl:369
[2] top-level scope
# task.jl:388
Please guide me in resolving the issue!
For your code the easiest way would be (assuming Julia has been run with -p 4 command line parameter or you have run addprocs(4):
julia> values = #distributed (append!) for i in 1:4
[f()]
end
4-element Vector{Float64}:
5001.232864826896
4999.244031827526
4966.883114472259
5014.022690758762
If you want to do #spawns yourself this code works:
julia> values = Vector{Float64}(undef, 4);
julia> #sync for w in workers()
#async values[w-1] = fetch(#spawnat w f())
end
julia> values
4-element Vector{Float64}:
5029.967318172736
4993.1064528029
5016.491407076979
5062.0706219606345
However your code mostly didn't work because the type of your values was not Vector{Float64}. Here is how to replicate your error:
julia> vv()=0
vv (generic function with 1 method)
julia> vv[1]=11
ERROR: MethodError: no method matching setindex!(::typeof(vv), ::Int64, ::Int64)
Related
I am running through basic examples for Flux, and I am getting MethodErrors when running code verbatim through the REPL. I am using a fresh REPL and am on VERSION 1.8.2 for linux.
julia> using Flux
julia> actual(x) = 4x + 2
actual (generic function with 1 method)
julia> x_train, x_test = hcat(0:5...), hcat(6:10...)
([0 1 … 4 5], [6 7 … 9 10])
julia> y_train, y_test = actual.(x_train), actual.(x_test)
([2 6 … 18 22], [26 30 … 38 42])
julia> model = Dense(1 => 1)
ERROR: MethodError: no method matching Dense(::Pair{Int64, Int64})
Closest candidates are:
Dense(::Any, ::Any) at ~/.julia/packages/Flux/qXNjB/src/layers/basic.jl:88
Dense(::S, ::T, ::F) where {F, S, T} at ~/.julia/packages/Flux/qXNjB/src/layers/basic.jl:83
Dense(::Integer, ::Integer) at ~/.julia/packages/Flux/qXNjB/src/layers/basic.jl:90
...
Stacktrace:
[1] top-level scope
# REPL[5]:1
I got a similar error when using gradient from another example
I was expecting the REPL to return
Let's say, I want to pass a function to another function:
function foo()
return 0;
end
function bar(func)
return func();
end
print(bar(foo));
But you can make functions typesafe:
function func(t::Int)
print(t);
end
func(0); #produces no error
func("Hello world"); #produces an error
I didn't found out, how I combine both, that means, how can I explicitly define a parameter of bar, like func, to be a function, possibly with certain input / output argument types.
Thanks in advance for any help.
If I understand you correctly you want to make sure the passed function returns a specific type? The simplest thing is to just typeassert the return value at runtime:
julia> function f(func)
val = func()::Int # Error if the return value is not of type Int
return val
end
f (generic function with 1 method)
julia> f(() -> 1)
1
julia> f(() -> 1.0)
ERROR: TypeError: in typeassert, expected Int64, got Float64
Stacktrace:
[1] f(::var"#7#8") at ./REPL[5]:2
[2] top-level scope at REPL[8]:1
Alternatively you can use the FunctionWrappers.jl package (which will convert to the specified return type, or error if the conversion is not possible):
julia> using FunctionWrappers: FunctionWrapper
julia> function f(func::FunctionWrapper{Int,<:Tuple})
val = func()
return val
end;
julia> function f(func)
fw = FunctionWrapper{Int,Tuple{}}(func)
return f(fw)
end;
julia> f(() -> 1)
1
julia> f(() -> 1.0) # Can convert to Int
1
julia> f(() -> 1.2) # Can not convert to Int
ERROR: InexactError: Int64(1.2)
A function is of type Function. You can easily check this:
julia> foo() = 1;
julia> T = typeof(foo)
typeof(foo)
julia> supertype(T)
Function
julia> foo isa Function
true
This will not necessarily cover all callable types, as you can make any type callable:
julia> struct Callable end
julia> (::Callable)(x::Number) = x + one(x)
julia> callable = Callable()
Callable()
julia> callable(5)
6
julia> callable isa Function
false
If I have a function signature like f(args...; kwargs...), how can I get a specific keyword out of kwargs? Naïvely typing kwargs.x does not work:
julia> f(args...; kwargs...) = kwargs.x
f (generic function with 1 method)
julia> f(x=1)
ERROR: type Pairs has no field x
Stacktrace:
[1] getproperty(::Base.Iterators.Pairs{Symbol,Int64,Tuple{Symbol},NamedTuple{(:x,),Tuple{Int64}}}, ::Symbol) at ./Base.jl:20
[2] #f#7(::Base.Iterators.Pairs{Symbol,Int64,Tuple{Symbol},NamedTuple{(:x,),Tuple{Int64}}}, ::typeof(f)) at ./REPL[2]:1
[3] (::var"#kw##f")(::NamedTuple{(:x,),Tuple{Int64}}, ::typeof(f)) at ./none:0
[4] top-level scope at REPL[3]:1
This question appeared on the JuliaLang Slack channel in the #helpdesk. For an automatic invite to the very helpful julia slack, simply fill out https://slackinvite.julialang.org
The reason this happens is that splatted keyword arguments are not stored in a named tuple by default. We can see how they're stored like so:
julia> g(;kwargs...) = kwargs
g (generic function with 1 method)
julia> g(a=1)
pairs(::NamedTuple) with 1 entry:
:a => 1
julia> g(a=1) |> typeof
Base.Iterators.Pairs{Symbol,Int64,Tuple{Symbol},NamedTuple{(:a,),Tuple{Int64}}}
So the splatted kwargs are instead stored as some sort of iterator object. However, we can easily convert that kwargs iterator to a NamedTuple like so: (;kwargs...) and then access it in the way we'd expect, so your example would translate into
julia> f(args...; kwargs...) = (;kwargs...).x
f (generic function with 1 method)
julia> f(x=1, y=2)
1
Of course, the more idiomatic way to do this would be to instead write the function as
julia> f(args...; x, kwargs...) = x
f (generic function with 1 method)
julia> f(x=1, y=2)
1
but this assumes you know the name you want to access (x) at the time when you write the function.
A brief sidenote: If we return to our example of g(;kwargs...) = kwargs, we can ask for the fieldnames of the iterator object the was returned like so:
julia> g(x=1, y=2) |> typeof |> fieldnames
(:data, :itr)
Hm, what is this data field?
julia> g(x=1, y=2).data
(x = 1, y = 2)
Aha! so we can actually get the kwargs as a named tuple using that, i.e. f(;kwargs...) = kwargs.data.x would work, but I wouldn't recommend this approach since it seems to rely on undocumented behaviour, so it may be a mere implementation detail that is not guaranteed to be stable across julia versions.
I am converting a code in julia 0.6 to 1.2.
Here is the old version:
#variable(model, use[i=eachindex(n), j=1:m], Bin)
Used = [indmax(getvalue(use[i,j])
for j=1:m) for i=eachindex(n)]
I converted to the following,
#variable(model, use[i=eachindex(n), j=1:m], Bin)
JuMP.optimize!(model)
Used = [argmax(JuMP.value(use[i,j])
for j=1:m) for i=eachindex(n)]
but with error:
MethodError: no method matching keys(::Base.Generator{UnitRange{Int64},getfield(Main, Symbol("##261#266")){Int64,JuMP.Containers.SparseAxisArray{VariableRef,2,Tuple{Any,Any}}}})
Closest candidates are:
keys(!Matched::Core.SimpleVector) at essentials.jl:606
keys(!Matched::Cmd) at process.jl:963
keys(!Matched::BenchmarkTools.BenchmarkGroup) at /Users/shuaiwang/.julia/packages/BenchmarkTools/7aqwe/src/groups.jl:31
...
pairs(::Base.Generator{UnitRange{Int64},getfield(Main, Symbol("##261#266")){Int64,JuMP.Containers.SparseAxisArray{VariableRef,2,Tuple{Any,Any}}}}) at abstractdict.jl:132
_findmax(::Base.Generator{UnitRange{Int64},getfield(Main, Symbol("##261#266")){Int64,JuMP.Containers.SparseAxisArray{VariableRef,2,Tuple{Any,Any}}}}, ::Colon) at array.jl:2068
findmax(::Base.Generator{UnitRange{Int64},getfield(Main, Symbol("##261#266")){Int64,JuMP.Containers.SparseAxisArray{VariableRef,2,Tuple{Any,Any}}}}) at array.jl:2065
argmax(::Base.Generator{UnitRange{Int64},getfield(Main, Symbol("##261#266")){Int64,JuMP.Containers.SparseAxisArray{VariableRef,2,Tuple{Any,Any}}}}) at array.jl:2153
(::getfield(Main, Symbol("##260#265")){ScenarioGraph,JuMP.Containers.SparseAxisArray{VariableRef,2,Tuple{Any,Any}}})(::Int64) at none:0
iterate at generator.jl:47 [inlined]
collect at array.jl:606 [inlined]
The problem seems to be unrelated to JuMP. The fix to your code is:
Used = [argmax([JuMP.value(use[i,j]) for j=1:m]) for i=eachindex(n)]
(I have not tested the whole code as it was not complete)
And the core of the issue is that you are not allowed to use argmax on generators, you have to pass a collection that supports pairs to it, e.g.:
julia> argmax(i for i in 1:3)
ERROR: MethodError: no method matching keys(::Base.Generator{UnitRange{Int64},getfield(Main, Symbol("##15#16"))})
fails, but
julia> argmax([i for i in 1:3])
3
julia> argmax((1,2,3))
3
julia> argmax((a=1,b=2,c=3))
:c
julia> argmax(Dict(:a=>1,:b=>2,:c=>3))
:c
work
I would like to form expressions like
julia> ex = :(a[$i,$j] + b[$i,$j])
because I am interested in writting a program that it is able to write a system of equations in which the user only gives the structure of them and then that julia writes explicitly the chunk of code involved. More generally, I would like to embed functions in a expression, something like
julia> ex = :(a[$myfunc(i,j),$j] + b[$i,$j])
Upon evaluating the first line of code, I get
ERROR: UndefVarError: i not defined
in eval(::Module, ::Any) at ./boot.jl:226
On the other hand, if I write
julia> ex = :(a[i,j] + b[i,j])
:(a[i,j] + b[i,j])
julia> for i in 1:2
for j in 1:2
println(eval(ex))
end
end
ERROR: UndefVarError: i not defined
in eval(::Module, ::Any) at ./boot.jl:226
[inlined code] from ./boot.jl:225
in anonymous at ./<no file>:4294967295
in eval(::Module, ::Any) at ./boot.jl:226
How is interpolation done in this case?
eval works at global scope, so it cannot reference any local variables. What you can do, however, is interpolate that expression into a larger chunk of code that's intended to be executed at the global scope:
julia> a = [1 2; 3 4]
b = [5 6; 7 8]
ex = :(a[i,j] + b[i,j]);
julia> #eval for i in 1:2
for j in 1:2
println($ex)
end
end
6
8
10
12
The #eval macro is a simple shorthand for quoting the passed syntax and passing it along to the eval() function. It'd probably even be better to define a custom function:
julia> #eval function f()
for i in 1:2
for j in 1:2
println($ex)
end
end
end
f (generic function with 1 method)
julia> f()
6
8
10
12
The usual caveats about eval apply: any passed code can be maliciously crafted to do nefarious or surprising things. Or just crash the program.