In Julia, I can use promote to make various types of objects compatible. For example:
>promote(1, 1.0)
(1.0,1.0)
>typeof(promote(1, 1.0))
(Float64, Float64)
However, if I use promote on arrays, it doesn't give me what I want:
>promote([1], [1.0])
([1],[1.0])
>typeof(promote([1], [1.0]))
(Array{Int64,1},Array{Float64,1})
What I want is for the Int64 array to be converted to a Float64 array, so I get something like:
>promote_array([1], [1.0])
([1.0],[1.0])
>typeof(promote_array([1], [1.0]))
(Array{Float64,1},Array{Float64,1})
Here promote_array is a hypothetical function I made up. I'm looking for a real function that does the same. Is there a function in Julia that does what promote_array does above?
I found the function Base.promote_eltype, which I can use to get what I want:
function promote_array(arrays...)
eltype = Base.promote_eltype(arrays...)
tuple([convert(Array{eltype}, array) for array in arrays]...)
end
This promote_array function then gives me the output I'm looking for:
>promote_array([1], [1.0])
([1.0],[1.0])
>typeof(promote_array([1], [1.0]))
(Array{Float64,1},Array{Float64,1})
The above solves my problem, although the existence of Base.promote_eltype suggests there may be an already built solution that I don't know about yet.
Here is what I would do:
function promote_array{S,T}(x::Vector{S},y::Vector{T})
U = promote_type(S,T)
convert(Vector{U},x), convert(Vector{U},y)
end
I'm not sure what your use case is exactly, but the following pattern is something I see as being fairly commonly required for code that has the tightest typing possible while being general:
function foo{S<:Real, T<:Real}(x::Vector{S},y::Vector{T})
length(x) != length(y) && error("Length mismatch")
result = zeros(promote_type(S,T), length(x))
for i in 1:length(x)
# Do some fancy foo-work here
result[i] = x[i] + y[i]
end
return result
end
Related
I have the following struct (simplified), and some calculations done with this struct:
mutable struct XX{VecType}
v::VecType
end
long_calculation(x::XX) = sum(x.v)
as a part of the program i need to update the v value. the struct is callable and mainly used as a cache. here, the use of static arrays helps a lot in speeding up calculations, but the type of v is ultimately defined by an user. my problem lies when assigning new values to XX.v:
function (f::XX)(w)
f.v .= w #here lies the problem
return long_calculation(f)
this works if v <: Array and w is of any value, but it doesn't work when v <: StaticArrays.StaticArray, as setindex! is not defined on that type.
How can i write f.v .= w in a way that, when v allows it, performs an inplace modification, but when not, just creates a new value, and stores it in the XX struct?
There's a package for exactly this use case: BangBang.jl. From there, you can use setindex!!:
f.v = setindex!!(f.v, w)
Here I propose a simple solution that should be enough in most cases. Use multiple dispatch and define the following function:
my_assign!(f::XX, w) = (f.v .= w)
my_assign!(f::XX{<:StaticArray}, w) = (f.v = w)
and then simply call it in your code like this:
function (f::XX)(w)
my_assign!(f, w)
return long_calculation(f)
end
Then if you (or your users) get an error with a default implementation it is easy enough to add another method to my_assign! co cover other special cases when it throws an error.
Would such a solution be enough for you?
I have a vector of type UInt8 and fixed length 10. I think it contains a null-terminated string but when I do String(v) it shows the string + all of the zeros of the rest of the vector.
v = zeros(UInt8, 10)
v[1:5] = Vector{UInt8}("hello")
String(v)
the output is "hello\0\0\0\0\0".
Either I'm packing it wrong or reading it wrong. Any thoughts?
I use this snippet:
"""
nullstring(Vector{UInt8})
Interpret a vector as null terminated string.
"""
nullstring(x::Vector{UInt8}) = String(x[1:findfirst(==(0), x) - 1])
Although I bet there are faster ways to do this.
You can use unsafe_string: unsafe_string(pointer(v)), this does it without a copy, so is very fast. But #laborg's solution is better in almost all cases, because it's safe.
If you want both safety and maximal performance, you have to write a manual function yourself:
function get_string(v::Vector{UInt8})
# Find first zero
zeropos = 0
#inbounds for i in eachindex(v)
iszero(v[i]) && (zeropos = i; break)
end
iszero(zeropos) && error("Not null-terminated")
GC.#preserve v unsafe_string(pointer(v), zeropos - 1)
end
But eh, what are the odds you REALLY need it to be that fast.
You can avoid copying bytes and preserve safety with the following code:
function nullstring!(x::Vector{UInt8})
i = findfirst(iszero, x)
SubString(String(x),1,i-1)
end
Note that after calling it x will be empty and the returned value is Substring rather than String but in many scenarios it does not matter. This code makes half allocations than code by #laborg and is slightly faster (around 10-20%). The code by Jacob is still unbeatable though.
In python I might have a function like this:
def sum_these(x, y=None):
if y is None:
y = 1
return x + y
What is the equivalent use in julia? To be exact I know I could probably do:
function sum_these(x, y=0)
if y == 0
y = 1
end
x + y
end
However I'd rather not use zero, instead some value with the same meaning as None in python
EDIT
Just for clarity's sake, the result of these examples functions isn't important. The events(any) that happen if y is None are important, and for my cases setting y=[some number] is undesirable. After some search I think, unless someone provides a better solution is to do something like:
function sum_these(x, y=nothing)
if y == nothing
do stuff
end
return something
function sum_these(x, y=nothing)
if y == nothing
do stuff
end
return something
end
That's not only perfectly fine, but because nothing is a singleton of type Void, the y==nothing will actually compile away so the if statement is actually no runtime cost here. I talk about this in depth in a blog post, but what it really means is that function auto-specialization allows for checks against nothing to always be free in type-stable/inferrable functions.
However, you may want to consider splitting this into two different functions:
function sum_these(x)
return something
end
function sum_these(x, y)
do stuff
return something
end
Of course, this is just a style difference and the right choice is determined by how much code is shared in the return something.
Maybe use multiple dispatch with an empty fallback?
function f(x, y=nothing)
...
do_something(x, y)
...
return something
end
do_something(x, y) = nothing
function do_something(x, y::Void)
...
end
add other relevant vars to do_something as necessary, and return something or mutate as necessary.
Your question is a bit confusing. It seems like what you want is
function sum_these(x, y=1)
return x + y
end
But that doesn't quite do what you are asking either, since even if you call sum_these(3, 0) in your example, it replaces 0 with 1.
Also in Python, I would use
def sum_these(x, y=1):
return x + y
Perhaps I misunderstand your question.
Best be explained by an example:
I define a type
type myType
α::Float64
β::Float64
end
z = myType( 1., 2. )
Then suppose that I want to pass this type as an argument to a function:
myFunc( x::Vector{Float64}, m::myType ) =
x[1].^m.α+x[2].^m.β
Is there a way to pass myType so that I can actually use it in the body of the function in a "cleaner" fashion as follows:
x[1].^α+x[2].^β
Thanks for any answer.
One way is to use dispatch to a more general function:
myFunc( x::Vector{Float64}, α::Float64, β::Float64) = x[1].^α+x[2].^β
myFunc( x::Vector{Float64}, m::myType = myFunc(x,m.α,m.β)
Or if your functions are longer, you may want to use Parameters.jl's #unpack:
function myFunc( x::Vector{Float64}, m::myType )
#unpack m: α,β #now those are defined
x[1].^α+x[2].^β
end
The overhead of unpacking is small because you're not copying, it's just doing a bunch of α=m.α which is just making an α which points to m.α. For longer equations, this can be a much nicer form if you have many fields and use them in long calculations (for reference, I use this a lot in DifferentialEquations.jl).
Edit
There's another way as noted in the comments. Let me show this. You can define your type (with optional kwargs) using the #with_kw macro from Parameters.jl. For example:
using Parameters
#with_kw type myType
α::Float64 = 1.0 # Give a default value
β::Float64 = 2.0
end
z = myType() # Generate with the default values
Then you can use the #unpack_myType macro which is automatically made by the #with_kw macro:
function myFunc( x::Vector{Float64}, m::myType )
#unpack_myType m
x[1].^α+x[2].^β
end
Again, this only has the overhead of making the references α and β without copying, so it's pretty lightweight.
You could add this to the body of your function:
(α::Float64, β::Float64) = (m.α, m.β)
UPDATE: My original answer was wrong for a subtle reason, but I thought it was a very interesting bit of information so rather than delete it altogether, I'm leaving it with an explanation on why it's wrong. Many thanks to Fengyang for pointing out the global scope of eval! (as well as the use of $ in an Expr context!)
The original answer suggested that:
[eval( parse( string( i,"=",getfield( m,i)))) for i in fieldnames( m)]
would return a list comprehension which had assignment side-effects, since it conceptually would result in something like [α=1., β=2., etc]. The assumption was that this assignment would be within local scope. However, as pointed out, eval is always assessed at global scope, therefore the above one-liner does not do what it's meant to. Example:
julia> type MyType
α::Float64
β::Float64
end
julia> function myFunc!(x::Vector{Float64}, m::MyType)
α=5.; β=6.;
[eval( parse( string( i,"=",getfield( m,i)))) for i in fieldnames( m)]
x[1] = α; x[2] = β; return x
end;
julia> myFunc!([0.,0.],MyType(1., 2.))
2-element Array{Float64,1}:
5.0
6.0
julia> whos()
MyType 124 bytes DataType
myFunc 0 bytes #myFunc
α 8 bytes Float64
β 8 bytes Float64
I.e. as you can see, the intention was for the local variables α and β to be overwritten, but they didn't; eval placed α and β variables at global scope instead. As a matlab programmer I naively assumed that eval() was conceptually equivalent to Matlab, without actually checking. Turns out it's more similar to the evalin('base',...) command.
Thanks again to Fengyand for giving another example of why the phrase "parse and eval" seems to have about the same effect on Julia programmers as the word "it" on the knights who until recently said "NI". :)
Let's say there is a type
immutable Foo
x :: Int64
y :: Float64
end
and there is a variable foo = Foo(1,2.0). I want to construct a new variable bar using foo as a prototype with field y = 3.0 (or, alternatively non-destructively update foo producing a new Foo object). In ML languages (Haskell, OCaml, F#) and a few others (e.g. Clojure) there is an idiom that in pseudo-code would look like
bar = {foo with y = 3.0}
Is there something like this in Julia?
This is tricky. In Clojure this would work with a data structure, a dynamically typed immutable map, so we simply call the appropriate method to add/change a key. But when working with types we'll have to do some reflection to generate an appropriate new constructor for the type. Moreover, unlike Haskell or the various MLs, Julia isn't statically typed, so one does not simply look at an expression like {foo with y = 1} and work out what code should be generated to implement it.
Actually, we can build a Clojure-esque solution to this; since Julia provides enough reflection and dynamism that we can treat the type as a sort of immutable map. We can use fieldnames to get the list of "keys" in order (like [:x, :y]) and we can then use getfield(foo, :x) to get field values dynamically:
immutable Foo
x
y
z
end
x = Foo(1,2,3)
with_slow(x, p) =
typeof(x)(((f == p.first ? p.second : getfield(x, f)) for f in fieldnames(x))...)
with_slow(x, ps...) = reduce(with_slow, x, ps)
with_slow(x, :y => 4, :z => 6) == Foo(1,4,6)
However, there's a reason this is called with_slow. Because of the reflection it's going to be nowhere near as fast as a handwritten function like withy(foo::Foo, y) = Foo(foo.x, y, foo.z). If Foo is parametised (e.g. Foo{T} with y::T) then Julia will be able to infer that withy(foo, 1.) returns a Foo{Float64}, but won't be able to infer with_slow at all. As we know, this kills the crab performance.
The only way to make this as fast as ML and co is to generate code effectively equivalent to the handwritten version. As it happens, we can pull off that version as well!
# Fields
type Field{K} end
Base.convert{K}(::Type{Symbol}, ::Field{K}) = K
Base.convert(::Type{Field}, s::Symbol) = Field{s}()
macro f_str(s)
:(Field{$(Expr(:quote, symbol(s)))}())
end
typealias FieldPair{F<:Field, T} Pair{F, T}
# Immutable `with`
for nargs = 1:5
args = [symbol("p$i") for i = 1:nargs]
#eval with(x, $([:($p::FieldPair) for p = args]...), p::FieldPair) =
with(with(x, $(args...)), p)
end
#generated function with{F, T}(x, p::Pair{Field{F}, T})
:($(x.name.primary)($([name == F ? :(p.second) : :(x.$name)
for name in fieldnames(x)]...)))
end
The first section is a hack to produce a symbol-like object, f"foo", whose value is known within the type system. The generated function is like a macro that takes types as opposed to expressions; because it has access to Foo and the field names it can generate essentially the hand-optimised version of this code. You can also check that Julia is able to properly infer the output type, if you parametrise Foo:
#code_typed with(x, f"y" => 4., f"z" => "hello") # => ...::Foo{Int,Float64,String}
(The for nargs line is essentially a manually-unrolled reduce which enables this.)
Finally, lest I be accused of giving slightly crazy advice, I want to warn that this isn't all that idiomatic in Julia. While I can't give very specific advice without knowing your use case, it's generally best to have fields with a manageable (small) set of fields and a small set of functions which do the basic manipulation of those fields; you can build on those functions to create the final public API. If what you want is really an immutable dict, you're much better off just using a specialised data structure for that.
There is also setindex (without the ! at the end) implemented in the FixedSizeArrays.jl package, which does this in an efficient way.