What does #with_kw do in Julia? - julia

I am reading some code which looks like:
#with_kw struct HyperParams
batch_size::Int = 128
latent_dim::Int = 100
epochs::Int = 25
verbose_freq::Int = 1000
output_dim::Int = 5
disc_lr::Float64 = 0.0002
gen_lr::Float64 = 0.0002
device::Function = gpu
end
but it is unclear to me what the #with_kw is doing in this context. Is this still a normal struct? It does not look like the macro is part of base Julia so I am unfamiliar with its usage here.

So it looks like the #with_kw is part of the Parameters package where it provides the ability to define default values for a struct fields and keyword arguments. Per the Julia docs here: https://docs.julialang.org/en/v1/manual/types/#Composite-Types it does not look like you can define default values so this is actually quite useful in that case. Here is an example of what is possible with the keyword arguments and default values:
julia> #with_kw struct HyperParams
batch_size::Int = 128
latent_dim::Int = 100
epochs::Int = 25
verbose_freq::Int = 1000
output_dim::Int = 5
disc_lr::Float64 = 0.0002
gen_lr::Float64 = 0.0002
device::Function
end
HyperParams
julia> hyper = HyperParams(device=cpu)
HyperParams
batch_size: Int64 128
latent_dim: Int64 100
epochs: Int64 25
verbose_freq: Int64 1000
output_dim: Int64 5
disc_lr: Float64 0.0002
gen_lr: Float64 0.0002
device: cpu (function of type typeof(cpu))
julia> HyperParams()
ERROR: Field 'device' has no default, supply it with keyword.
Stacktrace:
[1] error(s::String)
# Base ./error.jl:33
[2] HyperParams()
# Main ~/.julia/packages/Parameters/MK0O4/src/Parameters.jl:493
[3] top-level scope
# REPL[61]:1
julia> hyper = HyperParams(epochs=20, device=cpu)
HyperParams
batch_size: Int64 128
latent_dim: Int64 100
epochs: Int64 20
verbose_freq: Int64 1000
output_dim: Int64 5
disc_lr: Float64 0.0002
gen_lr: Float64 0.0002
device: cpu (function of type typeof(cpu))

Related

Shorthand for Int32 literals in Julia

I use lots of Int32s in my code because I have some large arrays of those. But for some x::Int32 we have typeof(x+1) == Int64 since numeric literals are Int64 by default (I have to use 64bit Julia to handle my arrays). The problem is, if I have some function f(x::Int32) then f(x+1) will method error. I don't want to implement a f(x::Int64) = f(convert(Int32, x)) for almost every function and want to use concrete types for type stability. Currently, I simply have expressions like x + Int32(1) all over my code which looks really cluttered. For other types we have shorthands, i.e., 1.f0 gives me a Float32 and big"1" a BigInt. Is there something similar for Int32?
Since you explicitly mention the big_str macro (big"") you can easily define a similar macro for Int32 (the same way the uint128_str and int128_str is defined):
macro i32_str(s)
parse(Int32, s)
end
julia> typeof(i32"1")
Int32
this might still clutter your code too much so alternatively you could exploit that a number followed by a name is multiplication:
struct i32 end
(*)(n, ::Type{i32}) = Int32(n)
julia> typeof(1i32)
Int32
You can make a macro to replace every literal integer with an Int32, a bit like what ChangePrecision.jl does for floats. A very quick first attempt is:
julia> macro literal32(ex)
esc(literal32(ex))
end;
julia> literal32(ex::Expr) = Expr(ex.head, literal32.(ex.args)...);
julia> literal32(i::Int) = Int32(i);
julia> literal32(z) = z; # ignore Symbol, literal floats, etc.
julia> #literal32 [1,2] .+ 3
2-element Vector{Int32}:
4
5
julia> #literal32 function fun(x::AbstractVector)
x[1] + 2 # both 1 and 2 are changed
end
fun (generic function with 1 method)
julia> fun(Int32[3,4]) |> typeof
Int32
One place this may have unexpected consequences is literal type parameters:
julia> #literal32([1,2,3]) isa Array{Int32,1}
true
julia> #literal32 [1,2,3] isa Array{Int32,1}
false
Another is that x^2 will not use Base.literal_pow, e.g. #literal32 Meta.#lower pi^2.
What if you say:
# Or, a::Int32 = 1
julia> a = Int32(1)
1
julia> b::Int32 = a+2
3
julia> typeof(b)
Int32
julia> f(b)
...

How to lock the variable type in Julia?

I want to lock the type of a variable in Julia, how to do? For example, I define an array called weight,
weight = Array{Float64,1}([1,2,3])
Now I want to lock the type of weight as Array{Float64,1}, is it possible?
I mean if do not lock the type of weight, then if I mistakenly or casually do
weight = 1
Then weight will become an Int64 variable, so it is not longer a 1D array. This is obviously not what I want.
I just want to make sure that once I defined weight as 1D Float64 array, then if I change the type of weight, I want Julia gives me an error saying that the type of weight has been changed which is not allowed. Is it possible? Thanks!
This is useful because by doing this, it may preventing me from forgetting weight is an 1D array, and therefore preventing bugs.
For global variables use const:
julia> const weight = Array{Float64,1}([1,2,3])
3-element Vector{Float64}:
1.0
2.0
3.0
julia> weight[1]=11
11
julia> weight=99
ERROR: invalid redefinition of constant weight
Note that redefining the reference will throw a warning:
julia> const u = 5
5
julia> u=11
WARNING: redefinition of constant u. This may fail, cause incorrect answers, or produce other errors
You can circumvent it by using the Ref type:
julia> const z = Ref{Int}(5)
Base.RefValue{Int64}(5)
julia> z[] = 11
11
julia> z[] = "hello"
ERROR: MethodError: Cannot `convert` an object of type String to an object of type Int64
In functions use local with type declaration:
julia> function f()
local a::Int
a="hello"
end;
julia> f()
ERROR: MethodError: Cannot `convert` an object of type String to an object of type Int64
You'd normally write:
weight::Vector{Float64} = Array{Float64,1}([1,2,3])
...but this doesn't seem to be possible in global scope:
julia> weight::Vector{Float64} = Array{Float64,1}([1,2,3])
ERROR: syntax: type declarations on global variables are not yet supported
Stacktrace:
[1] top-level scope
# REPL[8]:1
However, you can do it in local scope or in a struct:
julia> function fun()
weight::Vector{Float64} = Array{Float64,1}([1,2,3])
weight = 1
end
fun (generic function with 1 method)
julia> fun()
ERROR: MethodError: Cannot `convert` an object of type Int64 to an object of type Vector{Float64}
Closest candidates are:
convert(::Type{T}, ::AbstractArray) where T<:Array at array.jl:532
convert(::Type{T}, ::LinearAlgebra.Factorization) where T<:AbstractArray at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/LinearAlgebra/src/factorization.jl:58
convert(::Type{T}, ::T) where T<:AbstractArray at abstractarray.jl:14
...
Stacktrace:
[1] fun()
# Main ./REPL[10]:3
[2] top-level scope
# REPL[11]:1
You could use const, but then redefinition with a value of the same type will cause a warning:
julia> const weight = Array{Float64,1}([1,2,3]);
julia> weight = [2.]
WARNING: redefinition of constant weight. This may fail, cause incorrect answers, or produce other errors.
1-element Vector{Float64}:
2.0

How do I retrieve the sequence indices of an aligned region from a BioAlignments.jl alignment?

I would like to access the indices of the alignment data that is returned from the Julia pairalign function in BioAlignments.jl to know where the alignment occurred in context of the original sequences.
using BioAlignments
using BioSequences
scoremodel = AffineGapScoreModel(EDNAFULL, gap_open=-5, gap_extend=-1);
my_alignment = pairalign(LocalAlignment(),dna"ATATTAGGTATTGATTATTGTACGCGGCCCGGC" , dna"TTGATTATTGT", scoremodel)
alignment(my_alignment)
For example, a script like this will output an alignment object from which I can access the score via score() function. However I wish to know where, in the original sequences that I supplied as inputs, the alignment occurred and know how to call the variable storing this index. Could not seem to find this anywhere in the documentation.
While I do not use those libraries one of the approach to such problems in Julia is to dump such object and in result one can find out the location of needed information.
julia> dump(alignment(my_alignment))
PairwiseAlignment{LongSequence{DNAAlphabet{4}},LongSequence{DNAAlphabet{4}}}
a: AlignedSequence{LongSequence{DNAAlphabet{4}}}
seq: LongSequence{DNAAlphabet{4}}
data: Array{UInt64}((3,)) UInt64[0x8814881844188181, 0x4422244242184881, 0x0000000000000002]
part: UnitRange{Int64}
start: Int64 1
stop: Int64 33
shared: Bool false
aln: Alignment
anchors: Array{AlignmentAnchor}((2,))
1: AlignmentAnchor
seqpos: Int64 10
refpos: Int64 0
op: Operation OP_START
2: AlignmentAnchor
seqpos: Int64 21
refpos: Int64 11
op: Operation OP_SEQ_MATCH
firstref: Int64 1
lastref: Int64 11
b: LongSequence{DNAAlphabet{4}}
data: Array{UInt64}((1,)) UInt64[0x0000084881881488]
part: UnitRange{Int64}
start: Int64 1
stop: Int64 11
shared: Bool false
And now you can see where is the information that you need:
julia> alignment(my_alignment).a.aln.anchors
2-element Array{AlignmentAnchor,1}:
AlignmentAnchor(10, 0, '0')
AlignmentAnchor(21, 11, '=')
The disadvantage of this approach is that the data structures are usually not the part of the library API and could change over the time with new package realeases.

Why does Julia Int64, Float64 and boolean support getindex

In Julia, you can declare an Int64, Bool or a Float64 and index it with 1.
julia> aa = 10
10
julia> typeof(10)
Int64
julia> aa[1]
10
julia> aa[0]
ERROR: BoundsError
Stacktrace:
[1] getindex(::Int64, ::Int64) at .\number.jl:78
[2] top-level scope at none:0
julia> aa[2]
ERROR: BoundsError
Stacktrace:
[1] getindex(::Int64, ::Int64) at .\number.jl:78
[2] top-level scope at none:0
Are there practical or theoretical reasons for this functionality to exist? I have never seen it in any other language I've used (Python, Ruby, Matlab, C++).
The reason is twofold:
Numbers are treated by Julia as 0-dimensional containers.
If you add 1 as a dimension index number in getindex then it is not an error, even if 1 is beyond the dimensionality of the container.
These two rules in combination lead to the behavior you describe. Here are some more examples of the same:
julia> a = 1
1
julia> b = [1,2,3]
3-element Array{Int64,1}:
1
2
3
julia> a[]
1
julia> a[1,1,1,1]
1
julia> b[2,1,1,1,1]
2
and note that standard functions defined for containers are defined for numbers and behave as for 0-dimensional objects, e.g.:
julia> size(a)
()
julia> axes(a)
()
There is an open PR that gives more details how omitted and extra indices work.

Status of default initialisation of struct fields?

Under Julia v0.6, the simple code:
julia> struct A
x::Int = 1
end
generates this error:
ERROR: syntax: "x::Int=1" inside type definition is reserved
This is quite an elusive message: reserved for what?
-> Do I have to understand that this kind of definition is going to be allowed in future Julia revisions?
This is available via Parameters.jl.
julia> using Parameters
julia> #with_kw struct A
a::Int = 6
b::Float64 = -1.1
c::UInt8
end
julia> A(c=4)
A
a: 6
b: -1.1
c: 4

Resources