I saw this example in the Julia language documentation. It uses something called Base. What is this Base?
immutable Squares
count::Int
end
Base.start(::Squares) = 1
Base.next(S::Squares, state) = (state*state, state+1)
Base.done(S::Squares, s) = s > S.count;
Base.eltype(::Type{Squares}) = Int # Note that this is defined for the type
Base.length(S::Squares) = S.count;
Base is a module which defines many of the functions, types and macros used in the Julia language. You can view the files for everything it contains here or call whos(Base) to print a list.
In fact, these functions and types (which include things like sum and Int) are so fundamental to the language that they are included in Julia's top-level scope by default.
This means that we can just use sum instead of Base.sum every time we want to use that particular function. Both names refer to the same thing:
Julia> sum === Base.sum
true
Julia> #which sum # show where the name is defined
Base
So why, you might ask, is it necessary is write things like Base.start instead of simply start?
The point is that start is just a name. We are free to rebind names in the top-level scope to anything we like. For instance start = 0 will rebind the name 'start' to the integer 0 (so that it no longer refers to Base.start).
Concentrating now on the specific example in docs, if we simply wrote start(::Squares) = 1, then we find that we have created a new function with 1 method:
Julia> start
start (generic function with 1 method)
But Julia's iterator interface (invoked using the for loop) requires us to add the new method to Base.start! We haven't done this and so we get an error if we try to iterate:
julia> for i in Squares(7)
println(i)
end
ERROR: MethodError: no method matching start(::Squares)
By updating the Base.start function instead by writing Base.start(::Squares) = 1, the iterator interface can use the method for the Squares type and iteration will work as we expect (as long as Base.done and Base.next are also extended for this type).
I'll grant that for something so fundamental, the explanation is buried a bit far down in the documentation, but http://docs.julialang.org/en/release-0.4/manual/modules/#standard-modules describes this:
There are three important standard modules: Main, Core, and Base.
Base is the standard library (the contents of base/). All modules
implicitly contain using Base, since this is needed in the vast
majority of cases.
Related
Usually the multiple dispatch in julia is straightforward if one of the parameters in a function changes data type, for example Float64 vs Complex{Float64}. How can I implement multiple dispatch if the parameter is an integer, and I want two functions, one for even and other for odd values?
You may be able to solve this with a #generated function: https://docs.julialang.org/en/v1/manual/metaprogramming/#Generated-functions-1
But the simplest solution is to use an ordinary branch in your code:
function foo(x::MyType{N}) where {N}
if isodd(N)
return _oddfoo(x)
else
return _evenfoo(x)
end
end
This may seem as a defeat for the type system, but if N is known at compile-time, the compiler will actually select only the correct branch, and you will get static dispatch to the correct function, without loss of performance.
This is idiomatic, and as far as I know the recommended solution in most cases.
I expect that with type dispatch you ultimately still are calling after a check on odd versus even, so the most economical of code, without a run-time penatly, is going to be having the caller check the argument and call the proper function.
If you nevertheless have to be type based, for some reason unrelated to run-time efficiency, here is an example of such:
abstract type HasParity end
struct Odd <: HasParity
i::Int64
Odd(i::Integer) = new(isodd(i) ? i : error("not odd"))
end
struct Even <: HasParity
i::Int64
Even(i::Integer) = new(iseven(i) ? i : error("not even"))
end
parity(i) = return iseven(i) ? Even(i) : Odd(i)
foo(i::Odd) = println("$i is odd.")
foo(i::Even) = println("$i is even.")
for n in 1:4
k::HasParity = parity(n)
foo(k)
end
So here's other option which I think is cleaner and more multiple dispatch oriented (given by a coworker). Let's think N is the natural number to be checked and I want two functions that do different stuff depending if N is even or odd. Thus
boolN = rem(N,2) == 0
(...)
function f1(::Val{true}, ...)
(...)
end
function f1(::Val{false}, ...)
(...)
end
and to call the function just do
f1(Val(boolN))
As #logankilpatrick pointed out the dispatch system is type based. What you are dispatching on, though, is well established pattern known as a trait.
Essentially your code looks like
myfunc(num) = iseven(num) ? _even_func(num) : _odd_func(num)
In Julia, how do I create custom types MyOrderedDictA and MyOrderedDictB such that:
Each has all the functionality of an OrderdDict, and can be passed to any function that accepts AbstractDicts
They are distinct from each other, so that I can take advantage of multiple dispatch.
I suspect\hope this is straightforward, but haven’t been able to figure it out.
Basically, what you have to do is to define your type MyOrderedDictA, wrapping a regular OrderedDict, and forward all functions that one can apply to an OrderedDict to this wrapped dict.
Unfortunately, the AbstractDict interface is (to my knowledge) currently not documented (cf. AbstractArray). You could look at their definition and check which functions are defined for them. Alternatively, there is the more practical approach to just use your MyOrderedDictA and whenever you get an error message, because a function is not defined, you forward this function "on-the-fly".
In any case, using the macro #forward from Lazy.jl you can do something along the lines of the following.
using Lazy
struct MyOrderedDictA{T,S} <: AbstractDict{T,S}
dict::OrderedDict{T,S}
end
MyOrderedDictA{T,S}(args...; kwargs...) where {T,S} = new{T,S}(OrderedDict{T,S}(args...; kwargs...))
function MyOrderedDictA(args...; kwargs...)
d = OrderedDict(args...; kwargs...)
MyOrderedDictA{keytype(d),valtype(d)}(d)
end
#forward MyOrderedDictA.dict (Base.length, Base.iterate, Base.getindex, Base.setindex!)
d = MyOrderedDictA(2=>1, 1=>2)
Others will be better placed to answer this, but a quick take:
For this you will need to look at the OrderedDict implementation, and specifically which methods are defined for OrderedDicts. If you want to be able to pass it to methods accepting AbstractDicts you need to subtype it like struct MyDictA{T, S} <: AbstractDict{T, S}
If you define two structs they will automatically be discting from each other!? (I might be misunderstanding the question here)
In julia how do we know if a type is manipulated by value or by reference?
In java for example (at least for the sdk):
the basic types (those that have names starting with lower case letters, like "int") are manipulated by value
Objects (those that have names starting with capital letters, like "HashMap") and arrays are manipulated by reference
It is therefore easy to know what happens to a type modified inside a function.
I am pretty sure my question is a duplicate but I can't find the dup...
EDIT
This code :
function modifyArray(a::Array{ASCIIString,1})
push!(a, "chocolate")
end
function modifyInt(i::Int)
i += 7
end
myarray = ["alice", "bob"]
modifyArray(myarray)
#show myarray
myint = 1
modifyInt(myint)
#show myint
returns :
myarray = ASCIIString["alice","bob", "chocolate"]
myint = 1
which was a bit confusing to me, and the reason why I submitted this question. The comment of #StefanKarpinski clarified the issue.
My confusion came from the fact i considred += as an operator , a method like push! which is modifying the object itself . but it is not.
i += 7 should be seen as i = i + 7 ( a binding to a different object ). Indeed this behavior will be the same for modifyArray if I use for example a = ["chocolate"].
The corresponding terms in Julia are mutable and immutable types:
immutable objects (either bitstypes, such as Int or composite types declared with immutable, such as Complex) cannot be modified once created, and so are passed by copying.
mutable objects (arrays, or composite types declared with type) are passed by reference, so can be modified by calling functions. By convention such functions end with an exclamation mark (e.g., sort!), but this is not enforced by the language.
Note however that an immutable object can contain a mutable object, which can still be modified by a function.
This is explained in more detail in the FAQ.
I think the most rigourous answer is the one in
Julia function argument by reference
Strictly speaking, Julia is not "call-by-reference" but "call-by-value where the value is a
reference" , or "call-by-sharing", as used by most languages such as
python, java, ruby...
Is there a way to enforce a dictionary being constant?
I have a function which reads out a file for parameters (and ignores comments) and stores it in a dict:
function getparameters(filename::AbstractString)
f = open(filename,"r")
dict = Dict{AbstractString, AbstractString}()
for ln in eachline(f)
m = match(r"^\s*(?P<key>\w+)\s+(?P<value>[\w+-.]+)", ln)
if m != nothing
dict[m[:key]] = m[:value]
end
end
close(f)
return dict
end
This works just fine. Since i have a lot of parameters, which i will end up using on different places, my idea was to let this dict be global. And as we all know, global variables are not that great, so i wanted to ensure that the dict and its members are immutable.
Is this a good approach? How do i do it? Do i have to do it?
Bonus answerable stuff :)
Is my code even ok? (it is the first thing i did with julia, and coming from c/c++ and python i have the tendencies to do things differently.) Do i need to check whether the file is actually open? Is my reading of the file "julia"-like? I could also readall and then use eachmatch. I don't see the "right way to do it" (like in python).
Why not use an ImmutableDict? It's defined in base but not exported. You use one as follows:
julia> id = Base.ImmutableDict("key1"=>1)
Base.ImmutableDict{String,Int64} with 1 entry:
"key1" => 1
julia> id["key1"]
1
julia> id["key1"] = 2
ERROR: MethodError: no method matching setindex!(::Base.ImmutableDict{String,Int64}, ::Int64, ::String)
in eval(::Module, ::Any) at .\boot.jl:234
in macro expansion at .\REPL.jl:92 [inlined]
in (::Base.REPL.##1#2{Base.REPL.REPLBackend})() at .\event.jl:46
julia> id2 = Base.ImmutableDict(id,"key2"=>2)
Base.ImmutableDict{String,Int64} with 2 entries:
"key2" => 2
"key1" => 1
julia> id.value
1
You may want to define a constructor which takes in an array of pairs (or keys and values) and uses that algorithm to define the whole dict (that's the only way to do so, see the note at the bottom).
Just an added note, the actual internal representation is that each dictionary only contains one key-value pair, and a dictionary. The get method just walks through the dictionaries checking if it has the right value. The reason for this is because arrays are mutable: if you did a naive construction of an immutable type with a mutable field, the field is still mutable and thus while id["key1"]=2 wouldn't work, id.keys[1]=2 would. They go around this by not using a mutable type for holding the values (thus holding only single values) and then also holding an immutable dict. If you wanted to make this work directly on arrays, you could use something like ImmutableArrays.jl but I don't think that you'd get a performance advantage because you'd still have to loop through the array when checking for a key...
First off, I am new to Julia (I have been using/learning it since only two weeks). So do not put any confidence in what I am going to say unless it is validated by others.
The dictionary data structure Dict is defined here
julia/base/dict.jl
There is also a data structure called ImmutableDict in that file. However as const variables aren't actually const why would immutable dictionaries be immutable?
The comment states:
ImmutableDict is a Dictionary implemented as an immutable linked list,
which is optimal for small dictionaries that are constructed over many individual insertions
Note that it is not possible to remove a value, although it can be partially overridden and hidden
by inserting a new value with the same key
So let us call what you want to define as a dictionary UnmodifiableDict to avoid confusion. Such object would probably have
a similar data structure as Dict.
a constructor that takes a Dict as input to fill its data structure.
specialization (a new dispatch?) of the the method setindex! that is called by the operator [] =
in order to forbid modification of the data structure. This should be the case of all other functions that end with ! and hence modify the data.
As far as I understood, It is only possible to have subtypes of abstract types. Therefore you can't make UnmodifiableDict as a subtype of Dict and only redefine functions such as setindex!
Unfortunately this is a needed restriction for having run-time types and not compile-time types. You can't have such a good performance without a few restrictions.
Bottom line:
The only solution I see is to copy paste the code of the type Dict and its functions, replace Dict by UnmodifiableDict everywhere and modify the functions that end with ! to raise an exception if called.
you may also want to have a look at those threads.
https://groups.google.com/forum/#!topic/julia-users/n-lqjybIO_w
https://github.com/JuliaLang/julia/issues/1974
REVISION
Thanks to Chris Rackauckas for pointing out the error in my earlier response. I'll leave it below as an illustration of what doesn't work. But, Chris is right, the const declaration doesn't actually seem to improve performance when you feed the dictionary into the function. Thus, see Chris' answer for the best resolution to this issue:
D1 = [i => sind(i) for i = 0.0:5:3600];
const D2 = [i => sind(i) for i = 0.0:5:3600];
function test(D)
for jdx = 1:1000
# D[2] = 2
for idx = 0.0:5:3600
a = D[idx]
end
end
end
## Times given after an initial run to allow for compiling
#time test(D1); # 0.017789 seconds (4 allocations: 160 bytes)
#time test(D2); # 0.015075 seconds (4 allocations: 160 bytes)
Old Response
If you want your dictionary to be a constant, you can use:
const MyDict = getparameters( .. )
Update Keep in mind though that in base Julia, unlike some other languages, it's not that you cannot redefine constants, instead, it's just that you get a warning when doing so.
julia> const a = 2
2
julia> a = 3
WARNING: redefining constant a
3
julia> a
3
It is odd that you don't get the constant redefinition warning when adding a new key-val pair to the dictionary. But, you still see the performance boost from declaring it as a constant:
D1 = [i => sind(i) for i = 0.0:5:3600];
const D2 = [i => sind(i) for i = 0.0:5:3600];
function test1()
for jdx = 1:1000
for idx = 0.0:5:3600
a = D1[idx]
end
end
end
function test2()
for jdx = 1:1000
for idx = 0.0:5:3600
a = D2[idx]
end
end
end
## Times given after an initial run to allow for compiling
#time test1(); # 0.049204 seconds (1.44 M allocations: 22.003 MB, 5.64% gc time)
#time test2(); # 0.013657 seconds (4 allocations: 160 bytes)
To add to the existing answers, if you like immutability and would like to get performant (but still persistent) operations which change and extend the dictionary, check out FunctionalCollections.jl's PersistentHashMap type.
If you want to maximize performance and take maximal advantage of immutability, and you don't plan on doing any operations on the dictionary whatsoever, consider implementing a perfect hash function-based dictionary. In fact, if your dictionary is a compile-time constant, these can even be computed ahead of time (using metaprogramming) and precompiled.
In Julia, is there any way to get the name of a passed to a function?
x = 10
function myfunc(a)
# do something here
end
assert(myfunc(x) == "x")
Do I need to use macros or is there a native method that provides introspection?
You can grab the variable name with a macro:
julia> macro mymacro(arg)
string(arg)
end
julia> #mymacro(x)
"x"
julia> #assert(#mymacro(x) == "x")
but as others have said, I'm not sure why you'd need that.
Macros operate on the AST (code tree) during compile time, and the x is passed into the macro as the Symbol :x. You can turn a Symbol into a string and vice versa. Macros replace code with code, so the #mymacro(x) is simply pulled out and replaced with string(:x).
Ok, contradicting myself: technically this is possible in a very hacky way, under one (fairly limiting) condition: the function name must have only one method signature. The idea is very similar the answers to such questions for Python. Before the demo, I must emphasize that these are internal compiler details and are subject to change. Briefly:
julia> function foo(x)
bt = backtrace()
fobj = eval(current_module(), symbol(Profile.lookup(bt[3]).func))
Base.arg_decl_parts(fobj.env.defs)[2][1][1]
end
foo (generic function with 1 method)
julia> foo(1)
"x"
Let me re-emphasize that this is a bad idea, and should not be used for anything! (well, except for backtrace display). This is basically "stupid compiler tricks", but I'm showing it because it can be kind of educational to play with these objects, and the explanation does lead to a more useful answer to the clarifying comment by #ejang.
Explanation:
bt = backtrace() generates a ... backtrace ... from the current position. bt is an array of pointers, where each pointer is the address of a frame in the current call stack.
Profile.lookup(bt[3]) returns a LineInfo object with the function name (and several other details about each frame). Note that bt[1] and bt[2] are in the backtrace-generation function itself, so we need to go further up the stack to get the caller.
Profile.lookup(...).func returns the function name (the symbol :foo)
eval(current_module(), Profile.lookup(...)) returns the function object associated with the name :foo in the current_module(). If we modify the definition of function foo to return fobj, then note the equivalence to the foo object in the REPL:
julia> function foo(x)
bt = backtrace()
fobj = eval(current_module(), symbol(Profile.lookup(bt[3]).func))
end
foo (generic function with 1 method)
julia> foo(1) == foo
true
fobj.env.defs returns the first Method entry from the MethodTable for foo/fobj
Base.decl_arg_parts is a helper function (defined in methodshow.jl) that extracts argument information from a given Method.
the rest of the indexing drills down to the name of the argument.
Regarding the restriction that the function have only one method signature, the reason is that multiple signatures will all be listed (see defs.next) in the MethodTable. As far as I know there is no currently exposed interface to get the specific method associated with a given frame address. (as an exercise for the advanced reader: one way to do this would be to modify the address lookup functionality in jl_getFunctionInfo to also return the mangled function name, which could then be re-associated with the specific method invocation; however, I don't think we currently store a reverse mapping from mangled name -> Method).
Note also that (1) backtraces are slow (2) there is no notion of "function-local" eval in Julia, so even if one has the variable name, I believe it would be impossible to actually access the variable (and the compiler may completely elide local variables, unused or otherwise, put them in a register, etc.)
As for the IDE-style introspection use mentioned in the comments: foo.env.defs as shown above is one place to start for "object introspection". From the debugging side, Gallium.jl can inspect DWARF local variable info in a given frame. Finally, JuliaParser.jl is a pure-Julia implementation of the Julia parser that is actively used in several IDEs to introspect code blocks at a high level.
Another method is to use the function's vinfo. Here is an example:
function test(argx::Int64)
vinfo = code_lowered(test,(Int64,))
string(vinfo[1].args[1][1])
end
test (generic function with 1 method)
julia> test(10)
"argx"
The above depends on knowing the signature of the function, but this is a non-issue if it is coded within the function itself (otherwise some macro magic could be needed).