Writing a simple tool for debuging julia 0.6 - julia

There seems to be no easy-to-use debugger for julia 0.6 and so I tried to construct a simple println debugger as suggested here. The following code is a module that provides #out macro for writing expressions and their values into a file:
module debug
using DataStructures: OrderedDict
import JSON
function fn_print_dict(file, d)
print(file, JSON.json(d,2))
end
function println_alltypes(file_to_write, eval_relevant_symbol, typeof_eval_relevant_symbol)
if typeof_eval_relevant_symbol <: Dict
fn_print_dict(file_to_write, eval_relevant_symbol)
elseif typeof_eval_relevant_symbol <: AbstractArray || typeof_eval_relevant_symbol <: Set
itr_collection_counter = 1
dict_collection = OrderedDict()
for item in eval_relevant_symbol
dict_collection[itr_collection_counter] = item
itr_collection_counter += 1
end
println_alltypes(file_to_write, dict_collection, Dict)
else
println(file_to_write, eval_relevant_symbol)
end
println(file_to_write)
end
export #out
macro out(relevant_symbols...)
return esc(quote
file_to_write_str = joinpath(homedir(), "output.txt")
if !isdefined(current_module(),:debug_counter_int)
debug_counter_int = 0
end
debug_counter_int += 1
if debug_counter_int == 1
open(file_to_write_str, "w") do file_to_write
println(file_to_write)
end
end
open(file_to_write_str, "a") do file_to_write
println(file_to_write, "===================================")
end
for relevant_symbol in $relevant_symbols
relevant_symbol_str = string(relevant_symbol)
eval_relevant_symbol = eval(relevant_symbol)
typeof_eval_relevant_symbol = typeof(eval_relevant_symbol)
open(file_to_write_str, "a") do file_to_write
println(file_to_write, "[$relevant_symbol_str]", " ($typeof_eval_relevant_symbol):")
$println_alltypes(file_to_write, eval_relevant_symbol, typeof_eval_relevant_symbol)
end
end
end)
end
end
For example,
julia> using debug
julia> a = [4, 5, Dict("a"=>44, "b"=>55)]
3-element Array{Any,1}:
4
5
Dict("b"=>55,"a"=>44)
julia> #out 2+3
julia> #out a
julia> #out [a, a]
julia>
Will write the following content into output.txt:
===================================
[2 + 3] (Int64):
5
===================================
[a] (Array{Any,1}):
{
"1": 4,
"2": 5,
"3": {
"b": 55,
"a": 44
}
}
===================================
[[a, a]] (Array{Array{Any,1},1}):
{
"1": [
4,
5,
{
"b": 55,
"a": 44
}
],
"2": [
4,
5,
{
"b": 55,
"a": 44
}
]
}
The issue is that #out will raise an error when used in a function such as
function fn(y)
#out y
end
fn (generic function with 1 method)
I appreciate any help in fixing this issue or proposing an alternative approach. I know that eval should not be used in a macro, but I'm not sure what would be the alternative here.

I don't have quite enough metaprogramming experience to debug your macro easily, but as a stopgap I might suggest the built-in #show macro, which should do most of what you want, except it'll print the results to stdout rather than to file (but you should be able to just redirect to a file when running at the command line if that's something you need).
As the comments have suggested, Julia 0.6 is indeed ancient by now. Given your task of upgrading a large Julia 0.6 codebase, I can heavily second #jling's suggestion of running on Julia 0.7, which is equivalent to 0.6 except with deprecations to warn you about all the things you will need to change to run on 1.x.

Related

unexpected behavior of dict.keys on in

for faster colision control I create dict which is used as sparse array from array of objects T2.
But this cod throw exeption
KeyError: key [142, 69, 77] not found
Here us my code:
Ans=Dict{T1,Vector{T2}}()
for i in L
pos=pos_func(i)
if (pos in Ans.keys)
push!(Ans[pos],i)
else
Ans[pos]=Vector{T2}([i])
end
end
I catched event and printed pos, Ans.keys and (pos in Ans.keys). I found that pos is one of Ans.keys and (pos in Ans.keys)==True. But anyway I cannot get Ans[pos].
Julia Version 1.4.0 Commit b8e9a9ecc6 (2020-03-21 16:36 UTC)
What is the reason for such behaviour? Why same code can works half times?
You should use the keys() function in order to get the keys of your dictionary instead of accessing the keys field. (Note that, in most cases, it is not a good idea to access internal fields of Julia objects, especially when accessor methods exist).
And in the particular case of testing whether a given key appears in a Dict, using haskey() would be even more idomatic.
The following should work:
# Some definitions so that your example is runnable
julia> T1 = Int;
julia> T2 = Int;
julia> L = 1:10;
julia> pos_func(i) = i%3;
julia> Ans=Dict{T1,Vector{T2}}()
Dict{Int64,Array{Int64,1}} with 0 entries
julia> for i in L
pos=pos_func(i)
if haskey(Ans, pos) # <- keys(Ans) instead of Ans.keys
push!(Ans[pos],i)
else
Ans[pos] = T2[i] # or maybe simply [i], unless your collection L is heterogeneous
end
end
julia> Ans
Dict{Int64,Array{Int64,1}} with 3 entries:
0 => [3, 6, 9]
2 => [2, 5, 8]
1 => [1, 4, 7, 10]

How do I turn a string into a specific enum type in a macro?

I have a module with an enum defined in it.
module myModule
#enum type A B B C D
end
type1 = myModule.A
Now I want to declare an instance of this enum type but I only have a string specifying which type it is. I tried the following:
str = "B"
type2 = eval(:(myModule.Symbol($str)))
But I get a warning message which I do not quite understand:
WARNING: replacing module myModule.
and the type of type2 is also just a Symbol.
Probably the simplest way is to use getproperty:
julia> module myModule
#enum type A B C D
end
Main.myModule
julia> str = "B";
julia> getproperty(myModule, Symbol(str))
B::type = 1
Alternatively, you could create your expression as a string and then parse and evaluate it:
julia> eval(Meta.parse(string("myModule.", str)))
B::type = 1
Or, the same thing, but with string interpolation instead of using the string function:
julia> eval(Meta.parse("myModule.$str"))
B::type = 1
Note that the syntax myModule.Symbol(str) is not equivalent to myModule.B. It looks like that syntax really just calls Symbol(str) in the global scope. For example, try the following:
julia> myModule.length([1, 2, 3])
3
julia> #code_lowered myModule.length([1, 2, 3])
CodeInfo(
1 ─ %1 = (Base.arraylen)(a)
└── return %1
)

Specializing method calls in order in meta-programming

I have issue after calling my macro:
#introspectable square(x) = x * x
Then when calling
square(3)
i should be able to get 9, cause the function call has been specialized to execute an attribute of the structure which is Julia code, however when I enter the macro, the code seems to be directly evaluated.
What i have tried:
struct IntrospectableFunction
name
parameters
native_function
end
(f::IntrospectableFunction)(x) = f.native_function(x)
macro introspectable(expr)
name = expr.args[1].args[1]
parameters = tuple(expr.args[1].args[2:end]...)
body = expr.args[2].args[2]
:( global $name = IntrospectableFunction( :( name ), $parameters, :( body ) ))
end
#introspectable square(x) = x * x
square(3)
The answer should be 9 , however i get "Object of type symbol are not callable ". However if i replace :( body ) with x -> x * x i get the desired result, my objective is generalizing the macro-call.
I usually find it easier to work with expressions in macros (it is not the shortest way to write things, but, from my experience, it is much easier to control what gets generated).
Therefore I would rewrite your code as:
macro introspectable(expr)
name = expr.args[1].args[1]
parameters = expr.args[1].args[2:end]
anon = Expr(Symbol("->"), Expr(:tuple, parameters...), expr.args[2].args[2])
constr = Expr(:call, :IntrospectableFunction, QuoteNode(name), Tuple(parameters), anon)
esc(Expr(:global, Expr(Symbol("="), name, constr)))
end
Now, as you said you wanted generality I would define your functor like this:
(f::IntrospectableFunction)(x...) = f.native_function(x...)
(in this way you allow multiple positional arguments to be passed).
Now let us test our definitions:
julia> #introspectable square(x) = x * x
IntrospectableFunction(:square, (:x,), getfield(Main, Symbol("##3#4"))())
julia> square(3)
9
julia> #macroexpand #introspectable square(x) = x * x
:(global square = IntrospectableFunction(:square, (:x,), ((x,)->x * x)))
julia> #introspectable toarray(x,y) = [x,y]
IntrospectableFunction(:toarray, (:x, :y), getfield(Main, Symbol("##5#6"))())
julia> toarray("a", 10)
2-element Array{Any,1}:
"a"
10
julia> #macroexpand #introspectable toarray(x,y) = [x,y]
:(global toarray = IntrospectableFunction(:toarray, (:x, :y), ((x, y)->[x, y])))
julia> function localscopetest()
#introspectable globalfun(x...) = x
end
localscopetest (generic function with 1 method)
julia> localscopetest()
IntrospectableFunction(:globalfun, (:(x...),), getfield(Main, Symbol("##9#10"))())
julia> globalfun(1,2,3,4,5)
(1, 2, 3, 4, 5)
julia> function f()
v = 100
#introspectable localbinding(x) = (v, x)
end
f (generic function with 1 method)
julia> f()
IntrospectableFunction(:localbinding, (:x,), getfield(Main, Symbol("##11#12")){Int64}(100))
julia> localbinding("x")
(100, "x")
(note that it is useful to use #macroexpand to make sure our macro works as expected)
EDIT - how to handle a minimal multiple dispatch
I am writing a non-macro example because it is related to the data structure:
Use e.g. such a definition:
struct IntrospectableFunction
name::Symbol
method_array::Vector{Pair{Type{<:Tuple}, Function}}
end
function (f::IntrospectableFunction)(x...)
for m in f.method_array
if typeof(x) <: first(m)
return last(m)(x...)
end
end
error("signature not found")
end
and now you can write:
julia> square = IntrospectableFunction(:square, [Tuple{Any}=>x->x*x,Tuple{Any,Any}=>(x,y)->x*y])
IntrospectableFunction(:square, Pair{DataType,Function}[Tuple{Any}=>##9#11(), Tuple{Any,Any}=>##10#12()])
julia> square(3)
9
julia> square(2,3)
6
Keep in mind that the approach I present is not perfect and universal - it just serves to give a very simple example how you could do it.

Julia: Best practice to unpack parameters inside a function

I can unpack a tuple. I'm trying to write a function (or macro) that would unpack a subset of these from an instance of the type-constructor Parameters(). That is, I know how to do:
a,b,c = unpack(p::Parameters)
But I would like to do something like this:
b,c = unpack(p::Parameters, b,c)
or maybe even lazier:
unpack(p::Parameters, b, c)
This is to avoid writing things like:
function unpack_all_oldstyle(p::Parameters)
a=p.a; b=p.b; c=p.c; ... z=p.z;
return a,b,c,...,z
end
There's something wrong with my approach, but hopefully there is a fix.
In case it wasn't clear from the wording of my question, I'm a total ignoramus. I read about unpacking the ellipsis here: how-to-pass-tuple-as-function-arguments
"module UP tests Unpacking Parameters"
module UP
struct Parameters
a::Int64
b::Int64
c::Int64
end
"this method sets default parameters and returns a tuple of default values"
function Parameters(;
a::Int64 = 3,
b::Int64 = 11,
c::Int64 = 101
)
Parameters(a, b, c)
end
"this function unpacks all parameters"
function unpack_all(p::Parameters)
return p.a, p.b, p.c
end
"this function tests the unpacking function: in the body of the function one can now refer to a rather than p.a : worth the effort if you have dozens of parameters and complicated expressions to compute, e.g. type (-b+sqrt(b^2-4*a*c))/2/a instead of (-p.b+sqrt(p.b^2-4*p.a *p.c))/2/p.a"
function unpack_all_test(p::Parameters)
a, b, c = unpack_all(p)
return a, b, c
end
"""
This function is intended to unpack selected parameters. The first, unnamed argument is the constructor for all parameters. The second argument is a tuple of selected parameters.
"""
function unpack_selected(p::Parameters; x...)
return p.x
end
function unpack_selected_test(p::Parameters; x...)
x = unpack_selected(p, x)
return x
end
export Parameters, unpack_all, unpack_all_test, unpack_selected, unpack_selected_test
end
p = UP.Parameters() # make an instance
UP.unpack_all_test(p)
## (3,11,101) ## Test successful
UP.unpack_selected_test(p, 12)
## 12 ## intended outcome
UP.unpack_selected_test(p, b)
## 11 ## intended outcome
UP.unpack_selected_test(p, c, b, a)
## (101,11,3) ## intended outcome
There already exists one: Parameters.jl.
julia> using Parameters
julia> struct Params
a::Int64
b::Int64
c::Int64
end
julia> #unpack a, c = Params(1,2,3)
Params(1,2,3)
julia> a,c
(1,3)
julia> #with_kw struct Params
a::Int64 = 3
b::Int64 = 11
c::Int64 = 101
end
julia> #unpack c,b,a = Params()
Params
a: Int64 3
b: Int64 11
c: Int64 101
julia> c,b,a
(101,11,3)
BTW, you can fix your unpack_selected by:
unpack_selected(p::Parameters, fields...) = map(x->getfield(p, x), fields).
# note that, the selected field names should be Symbol here
julia> unpack_selected(p, :b)
(11,)
julia> unpack_selected(p, :c, :b, :a)
(101,11,3)

How to use haskey() and in() functions with complex dictionnary keys in Julia?

haskey() and in() functions are very useful to test the content of dictionaries in Julia :
julia> dict = Dict("a" => 1, "b" => 2, "c" => 3, "d" => 4, "e" => 5)
Dict{String,Int64} with 5 entries:
"c" => 3
"e" => 5
"b" => 2
"a" => 1
"d" => 4
julia> haskey(dict, "a")
true
julia> in(("a" => 1), dict)
true
but I was surprised by their behavior with complex keys :
julia> immutable MyT
A::String
B::Int64
end
julia> a = Dict(MyT("Tom",191)=>1,MyT("Bob",20)=>1,MyT("Jo",315)=>1,MyT("Luc",493)=>1)
Dict{MyT,Int64} with 4 entries:
MyT("Tom",191) => 1
MyT("Jo",315) => 1
MyT("Bob",20) => 1
MyT("Luc",493) => 1
julia> keys(a)
Base.KeyIterator for a Dict{MyT,Int64} with 4 entries. Keys:
MyT("Tom",191)
MyT("Jo",315)
MyT("Bob",20)
MyT("Luc",493)
julia> haskey(a, MyT("Tom",191))
false
julia> in((MyT("Tom",191) => 1), a)
false
What I did wrong ?
Thank you very much for your comments !
Thanks to #Michael K. Borregaard, I can propose this solution :
a = Dict{MyT, Int64}()
keyArray = Array{MyT,1}()
keyArray = [MyT("Tom",191),MyT("Bob",20),MyT("Jo",315),MyT("Luc",493)]
for i in keyArray
a[i] = 1
end
println(a)
# Dict(MyT("Tom",191)=>1,MyT("Tom",191)=>1,MyT("Luc",493)=>1,MyT("Jo",315)=>1,MyT("Luc",493)=>1,MyT("Bob",20)=>1,MyT("Jo",315)=>1,MyT("Bob",20)=>1)
keyArray[1] # MyT("Tom",191)
haskey(a, keyArray[1]) # true
But I have to store keys in a separate array. This means that can't warranty the unicity of the keys which is the strength of the dictionaries and why I choose to use it :(
So I have to use another step :
unique(keyArray)
Another better solution :
function CompareKeys(k1::MyT, k2::MyT)
if k1.A == k2.A && k1.B == k2.B
return true
else
return false
end
end
function ExistKey(k::MyT, d::Dict{MyT, Int64})
for i in keys(d)
if CompareKeys(k, i)
return true
end
end
return false
end
a = Dict(MyT("Tom",191)=>1,MyT("Bob",20)=>1,MyT("Jo",315)=>1,MyT("Luc",493)=>1)
ExistKey(MyT("Tom",192),a) # false
ExistKey(MyT("Tom",191),a) # true
Compared to Julia, Go is more straightforward for this problem :
package main
import (
"fmt"
)
type MyT struct {
A string
B int
}
func main() {
dic := map[MyT]int{MyT{"Bob", 10}: 1, MyT{"Jo", 21}: 1}
if _, ok := dic[MyT{"Bob", 10}]; ok {
fmt.Println("key exists")
}
}
// answer is "key exists"
You just need to teach your MyT type that you want it to consider equality in terms of its composite fields:
julia> immutable MyT
A::String
B::Int64
end
import Base: ==, hash
==(x::MyT, y::MyT) = x.A == y.A && x.B == y.B
hash(x::MyT, h::UInt) = hash(x.A, hash(x.B, hash(0x7d6979235cb005d0, h)))
julia> a = Dict(MyT("Tom",191)=>1,MyT("Bob",20)=>1,MyT("Jo",315)=>1,MyT("Luc",493)=>1)
Dict{MyT,Int64} with 4 entries:
MyT("Jo", 315) => 1
MyT("Luc", 493) => 1
MyT("Tom", 191) => 1
MyT("Bob", 20) => 1
julia> haskey(a, MyT("Tom",191))
true
julia> in((MyT("Tom",191) => 1), a)
true
There are lots of good answers here, I'd just like to add a subtlety: this is partly because == calls === rather than recursively calling == when checking for structural equality, and partly because equal (==) strings are not generally identical (===) currently. Specifically, the fact that MyT("foo", 1) != MyT("foo", 1) is because "foo" !== "foo".
Strings are only "immutable by convention" – they are technically mutable, but Julia doesn't expose APIs for mutating them and encourages you not to mutate them. You can, however, access their underlying bytes and mutate that, which allows you to write a program that distinguishes two strings by getting by mutating one and not the other. That means that they cannot be === in the sense of Henry Baker's "EGAL" predicate (also here). If you have an immutable type with only "primitive" type fields, then this does not happen:
julia> immutable MyT2 # `struct MyT2` in 0.6
A::Float64
B::Int
end
julia> x = MyT2(1, 1)
MyT2(1.0, 1)
julia> y = MyT2(1, 1)
MyT2(1.0, 1)
julia> x == y
true
julia> x === y
true
I have already proposed that we change this and have == recursively call ==. This should be fixed, someone just needs to do the work. Moreover, in Julia 1.0 we could make Strings truly immutable rather than merely immutable by convention, and therefore have "foo" === "foo" be true. I've created an issue to discuss and track this change.
You're creating a new object in the haskey call. But two objects created by MyT("Tom", 191) are just two different MyT objects with the same field values.
Instead, do
key1 = MyT("Tom", 191)
a = Dict(key1 => 1)
haskey(a, key1)
see also
key2 = MyT("Tom", 191)
key1 == key2 # false
A julia-ideomatic way to deal with this would be to define an == method for MyT objects, so two objects are equal if they have the same field values. That would allow you to use them like you do.
It depends whether you need the type to be complex. Another easy and performant way to do what you want is to use a Tuple as the key:
a = Dict(("Tom", 191) => 1)
haskey(a, ("Tom", 191)) # true
a[("Tom", 191)] # 1
My approach would be similar to Matt's, but a bit simpler(?). Tuples are perfectly valid dictionary keys, so I would simply overload the relevant functions to convert your type back and forth to a tuple:
julia> immutable M; A::String; B::Int64; end
julia> import Base: =>, haskey, in
julia> =>(a::M, b) = (a.A, a.B)=>b
julia> haskey(a::Dict, b::M) = haskey(a, (b.A, b.B))
julia> in(a::Pair{M, Int64}, b::Int64) = in((a.first.A,a.first.B)=>a.second,b)
julia> a = Dict(M("Dick", 10)=>1, M("Harry", 20)=>2)
Dict{Tuple{String,Int64},Int64} with 2 entries:
("Dick", 10) => 1
("Harry", 20) => 2
julia> haskey(a, M("Dick", 10))
true
julia> in(M("Dick", 10)=>1, a)
true
"Compared to Julia, Go is more straightforward for this problem"
True. It also happens to be more error-prone (depending on your perspective). If you wanted to differentiate between two objects (used as keys) that do not correspond to the same object in memory, then Go's approach of simply testing 'value equality' would have landed you in trouble here (though one could argue 'value equality' generally makes more sense when comparing 'keys').

Resources