why CssProviderLeaf lasts about 1600 milliseconds?
julia> using Gtk
julia> name ="FieldName"
"FieldName"
julia> #time Gtk.CssProviderLeaf(data="#$name {background:#C0C0C0;border-width:2px}")
3.397363 seconds (118.32 k allocations: 5.960 MiB, 7.66% compilation time)
GtkCssProviderLeaf()
julia> #time Gtk.CssProviderLeaf(data="#name {background:#C0C0C0;border-width:2px}")
3.968938 seconds (6 allocations: 96 bytes)
GtkCssProviderLeaf()
This occurs in Windows 10 both with version 1.8.1 and with version 1.8.2.
Looking at the function in https://github.com/JuliaGraphics/Gtk.jl/blob/master/src/theme.jl:
function GtkCssProviderLeaf(; data = nothing, filename = nothing)
source_count = (data !== nothing) + (filename !== nothing)
#assert(source_count <= 1,
"GtkCssProvider must have at most one data or filename argument")
provider = GtkCssProviderLeaf(ccall((:gtk_css_provider_new, libgtk), Ptr{GObject}, ()))
if data !== nothing
GError() do error_check
ccall((:gtk_css_provider_load_from_data, libgtk), Bool,
(Ptr{GObject}, Ptr{UInt8}, Clong, Ptr{Ptr{GError}}),
provider, bytestring(data), -1, error_check)
end
elseif filename !== nothing
GError() do error_check
ccall((:gtk_css_provider_load_from_path, libgtk), Bool,
(Ptr{GObject}, Ptr{UInt8}, Ptr{Ptr{GError}}),
provider, bytestring(filename), error_check)
end
end
return provider
end
it seems the delay is in the call to libgtk. You might want to see if you can minimize the times this has to be called in the code you are using.
Thank Bill
a Julia expert shows me that the answer has been very fast:
julia> #time Gtk.CssProviderLeaf(data="#name {background:#C0C0C0;border-width:2px}")
0.000078 seconds (6 allocations: 96 bytes)
My be he is running on Linux system,the problem can be on the Windows operating system?
Related
I am trying to write a simple program that uses Julia to test if all elements of an array are the same. Is there a simple way to do this in Julia?
allunique tests if all elements of an array are unique. In order to test if all elements of an array are the same you can write e.g.:
function allequal(itr)
local x
isfirst = true
for v in itr
if isfirst
x = v
isfirst = false
else
isequal(x, v) || return false
end
end
return true
end
and now you have
julia> allequal([1,2,3])
false
julia> allequal([1,2,1])
false
julia> allequal([1,1,1])
true
You could write a shorter function like e.g.:
f1(itr) = length(Set(itr)) <= 1
but it probably will be slower (I have not run the benchmarks).
or you could write something like:
f2(itr) = length(itr) == 0 ? true : all(isequal(itr[1]), itr)
if your iterable has length defined and supports indexing.
length(itr)==0 || all( ==(itr[1]), itr)
This seems to be 3x faster than the proposed allequal function.
Some benchmarks:
julia> allequal_2(itr) = length(itr)==0 || all( ==(itr[1]), itr);
julia> const vv = ones(10000000)*3;
julia> #btime allequal($vv)
13.212 ms (0 allocations: 0 bytes)
true
julia> #btime allequal_2($vv)
4.178 ms (0 allocations: 0 bytes)
true
What is even more interesting it is 2x faster than the proposed very similar f2 function:
julia> #btime f2($vv)
9.509 ms (0 allocations: 0 bytes)
Say I have a struct,
struct MyStruct
a
b
end
Is there some way to write a function like the following
function doSomething(x::MyStruct,fieldName::String)
y = x.fieldName
return f(y)
end
I could not find anything about this in the documentation / forums.
You can access fields with Symbols, so you can convert the string to a symbol and then use getproperty:
julia> struct MyStruct
a
b
end
julia> function doSomething(x::MyStruct, name::String)
s = Symbol(name)
return getproperty(x, s)
end
doSomething (generic function with 1 method)
julia> doSomething(MyStruct(1, 2), "a")
1
Note, however, that this will probably be very inefficient, since the compiler most likely can't see through this and thus your code might be type-unstable, see https://docs.julialang.org/en/v1/manual/performance-tips/.
If you plan to get the value only few times redrikekre's solution is OK. However, using metaprogramming you can write code without efficiency penalty, see the getDoSomething2() function below.
Consider those three functions:
function doSomethingNative(x)
return x.a
end
function doSomething(x, name::String)
return getproperty(x, Symbol(name))
end
function getDoSomething2(name::String)
field = Symbol(name)
code = quote
(obj) -> obj.$field
end
return eval(code)
end
Now the setup:
using BenchmarkTools
struct MyStruct
a
b
end
x = MyStruct(5,6)
Now the benchmarks:
julia> #btime doSomethingNative($x)
0.001 ns (0 allocations: 0 bytes)
5
julia> #btime doSomething($x,"a")
36.266 ns (0 allocations: 0 bytes)
5
julia> const doSomething2 = getDoSomething2("a");
julia> #btime doSomething2($x)
0.001 ns (0 allocations: 0 bytes)
5
If you run #code_native doSomethingNative(x) and #code_native doSomething2(x) you will see that the assembly output is identical.
Let x = randn(100, 2). I want to write x to its own file. This file will contain x, and only x, and x will only ever be of type Matrix{Float64}. In the past, I have always used HDF5 for this, but it occurs to me that this is over-kill, since in this setup I will only have one array per file. Note that JLD uses HDF5, and so is also over-kill.
1) What is the fastest method for reading and writing x assuming I will only ever want to read the entire matrix?
2) What is the fastest method for reading and writing x assuming I might want to read a slice of the matrix?
3) What is the fastest method for reading and writing x assuming I might want to read a slice of the matrix, or over-write a slice of the matrix (but not change the matrix size)?
You could use the serialize function, provided you heed the warnings in the documentation about non-guarantees between versions etc.
serialize(stream::IO, value)
Write an arbitrary value to a stream in an opaque format, such that it can be read back by deserialize. The read-back value will be as identical as possible to the original. In general, this process will not work if the reading and writing are done by different
versions of Julia, or an instance of Julia with a different system image. Ptr values are serialized as all-zero bit patterns (NULL).
An 8-byte identifying header is written to the stream first. To avoid writing the header, construct a SerializationState and use it as the first argument to serialize instead. See also Serializer.writeheader.
Really though, JLD (or in fact, its successor, JLD2) is generally the recommended way*.
*Of particular interest to you might be the statements that: "JLD2 saves and loads Julia data structures in a format comprising a subset of HDF5, without any dependency on the HDF5 C library" and that "it typically outperforms the previous JLD package (sometimes by multiple orders of magnitude) and often outperforms Julia's built-in serializer".
Based on the suggestions made by Tasos above, I put together a rudimentary speed test for both writes and reads using 4 different methods:
h5 (using the HDF5 package)
jld (using the JLD2 package)
slz (using serialize and deserialize)
dat (write to a binary file, using the first 128 bits to store the dimension of the matrix)
I've pasted the test code at the bottom of this answer. The results are:
julia> #time f_write_test(N, "h5")
0.191555 seconds (2.11 k allocations: 76.380 MiB, 26.39% gc time)
julia> #time f_write_test(N, "jld")
0.774857 seconds (8.33 k allocations: 77.058 MiB, 0.32% gc time)
julia> #time f_write_test(N, "slz")
0.108687 seconds (2.61 k allocations: 76.495 MiB, 1.91% gc time)
julia> #time f_write_test(N, "dat")
0.087488 seconds (1.61 k allocations: 76.379 MiB, 1.08% gc time)
julia> #time f_read_test(N, "h5")
0.051646 seconds (5.81 k allocations: 76.515 MiB, 14.80% gc time)
julia> #time f_read_test(N, "jld")
0.071249 seconds (10.04 k allocations: 77.136 MiB, 57.60% gc time)
julia> #time f_read_test(N, "slz")
0.038967 seconds (3.11 k allocations: 76.527 MiB, 22.17% gc time)
julia> #time f_read_test(N, "dat")
0.068544 seconds (1.81 k allocations: 76.405 MiB, 59.21% gc time)
So for writes, the write to binary option outperforms even serialize, and is twice as fast as HDF5 and almost an order of magnitude faster than JLD2.
For reads, deserialize has the best performance, while HDF5, JLD2 and reading from binary are all fairly close in performance, with HDF5 being slightly ahead.
I haven't included a test for writing to slices, but may come back to this in the future. Obviously writing to slices is impossible using serialize (not to mention the versioning/system image issues that serialize also faces), and I'm not really sure how to do it using JLD2. My gut feel writing a slice to binary will easily beat HDF5 if the slice is contiguous on disk, but will probably be significantly slower than HDF5 if it is non-contiguous and if the HDF5 method optimally exploits chunking. If HDF5 doesn't exploit chunking (which implies knowing at write time what slices you will want), then I suspect the binary method will come out ahead.
In summary, I'm going to go with the binary method, as I think that at this stage it is clearly the overall winner.
I suspect that eventually, JLD2 will probably be the method of choice, but there is a fair way to go here (the package itself is very new so not much time for the community to work on optimisations etc).
Test code follows:
using JLD2, HDF5
f_write_h5(fp::String, x::Matrix{Float64}) = h5write(fp, "G/D", x)
f_write_jld(fp::String, x::Matrix{Float64}) = #save fp x
f_write_slz(fp::String, x::Matrix{Float64}) = open(fid->serialize(fid, x), fp, "w")
f_write_dat_inner(fid1::IOStream, x::Matrix{Float64}) = begin ; write(fid1, size(x,1)) ; write(fid1, size(x,2)) ; write(fid1, x) ; end
f_write_dat(fp::String, x::Matrix{Float64}) = open(fid1->f_write_dat_inner(fid1, x), fp, "w")
f_read_h5(fp::String) = h5read(fp, "G/D")
f_read_jld(fp::String) = #load fp x
f_read_slz(fp::String) = open(deserialize, fp, "r")
f_read_dat_inner(fid1::IOStream) = begin ; d1 = read(fid1, Int) ; d2 = read(fid1, Int) ; read(fid1, Float64, (d1, d2)) ; end
f_read_dat(fp::String) = open(f_read_dat_inner, fp, "r")
function f_write_test(N::Int, filetype::String)
dp = "/home/colin/Temp/"
filetype == "h5" && [ f_write_h5("$(dp)$(n).$(filetype)", randn(1000, 100)) for n = 1:N ]
filetype == "jld" && [ f_write_jld("$(dp)$(n).$(filetype)", randn(1000, 100)) for n = 1:N ]
filetype == "slz" && [ f_write_slz("$(dp)$(n).$(filetype)", randn(1000, 100)) for n = 1:N ]
filetype == "dat" && [ f_write_dat("$(dp)$(n).$(filetype)", randn(1000, 100)) for n = 1:N ]
#[ rm("$(dp)$(n).$(filetype)") for n = 1:N ]
nothing
end
function f_read_test(N::Int, filetype::String)
dp = "/home/colin/Temp/"
filetype == "h5" && [ f_read_h5("$(dp)$(n).$(filetype)") for n = 1:N ]
filetype == "jld" && [ f_read_jld("$(dp)$(n).$(filetype)") for n = 1:N ]
filetype == "slz" && [ f_read_slz("$(dp)$(n).$(filetype)") for n = 1:N ]
filetype == "dat" && [ f_read_dat("$(dp)$(n).$(filetype)") for n = 1:N ]
[ rm("$(dp)$(n).$(filetype)") for n = 1:N ]
nothing
end
f_write_test(1, "h5")
f_write_test(1, "jld")
f_write_test(1, "slz")
f_write_test(1, "dat")
f_read_test(1, "h5")
f_read_test(1, "jld")
f_read_test(1, "slz")
f_read_test(1, "dat")
N = 100
#time f_write_test(N, "h5")
#time f_write_test(N, "jld")
#time f_write_test(N, "slz")
#time f_write_test(N, "dat")
#time f_read_test(N, "h5")
#time f_read_test(N, "jld")
#time f_read_test(N, "slz")
#time f_read_test(N, "dat")
Julia has two build-in functions readdlm & writedlm for doing this:
julia> x = randn(5, 5)
5×5 Array{Float64,2}:
-1.2837 -0.641382 0.611415 0.965762 -0.962764
0.106015 -0.344429 1.40278 0.862094 0.324521
-0.603751 0.515505 0.381738 -0.167933 -0.171438
-1.79919 -0.224585 1.05507 -0.753046 0.0545622
-0.110378 -1.16155 0.774612 -0.0796534 -0.503871
julia> writedlm("txtmat.txt", x, use_mmap=true)
julia> readdlm("txtmat.txt", use_mmap=true)
5×5 Array{Float64,2}:
-1.2837 -0.641382 0.611415 0.965762 -0.962764
0.106015 -0.344429 1.40278 0.862094 0.324521
-0.603751 0.515505 0.381738 -0.167933 -0.171438
-1.79919 -0.224585 1.05507 -0.753046 0.0545622
-0.110378 -1.16155 0.774612 -0.0796534 -0.503871
Definitely not the fastest way(use Mmap.mmap directly as DanGetz suggested in the comment if performance is a big deal), but it seems this is the simplest way and the output file is human-readable.
I have pi approximation code very similar to that on official page:
function piaprox()
sum = 1.0
for i = 2:m-1
sum = sum + (1.0/(i*i))
end
end
m = parse(Int,ARGS[1])
opak = parse(Int,ARGS[2])
#time for i = 0:opak
piaprox()
end
When I try to compare time of C and Julia, then Julia is significantly slower, almost 38 sec for m = 100000000 (time of C is 0.1608328933 sec). Why this is happening?
julia> m=100000000
julia> function piaprox()
sum = 1.0
for i = 2:m-1
sum = sum + (1.0/(i*i))
end
end
piaprox (generic function with 1 method)
julia> #time piaprox()
28.482094 seconds (600.00 M allocations: 10.431 GB, 3.28% gc time)
I would like to mention two very important paragraphs from Performance Tips section of julia documentation:
Avoid global variables A global variable might have its value, and
therefore its type, change at any point. This makes it difficult for
the compiler to optimize code using global variables. Variables should
be local, or passed as arguments to functions, whenever possible.....
The macro #code_warntype (or its function variant code_warntype()) can
sometimes be helpful in diagnosing type-related problems.
julia> #code_warntype piaprox();
Variables:
sum::Any
#s1::Any
i::Any
It's clear from #code_warntype output that compiler could not recognize types of local variables in piaprox(). So we try to declare types and remove global variables:
function piaprox(m::Int)
sum::Float64 = 1.0
i::Int = 0
for i = 2:m-1
sum = sum + (1.0/(i*i))
end
end
julia> #time piaprox(100000000 )
0.009023 seconds (11.10 k allocations: 399.769 KB)
julia> #code_warntype piaprox(100000000);
Variables:
m::Int64
sum::Float64
i::Int64
#s1::Int64
EDIT
as #user3662120 commented, the super fast behavior of the answer is result of a mistake, without a return value LLVM might ignore the for loop, by adding a return line the #time result would be:
julia> #time piaprox(100000000)
0.746795 seconds (11.11 k allocations: 400.294 KB, 0.45% gc time)
1.644934057834575
Does Julia support the reflection just like java?
What I need is something like this:
str = ARGS[1] # str is a string
# invoke the function str()
The Good Way
The recommended way to do this is to convert the function name to a symbol and then look up that symbol in the appropriate namespace:
julia> fn = "time"
"time"
julia> Symbol(fn)
:time
julia> getfield(Main, Symbol(fn))
time (generic function with 2 methods)
julia> getfield(Main, Symbol(fn))()
1.448981716732318e9
You can change Main here to any module to only look at functions in that module. This lets you constrain the set of functions available to only those available in that module. You can use a "bare module" to create a namespace that has only the functions you populate it with, without importing all name from Base by default.
The Bad Way
A different approach that is not recommended but which many people seem to reach for first is to construct a string for code that calls the function and then parse that string and evaluate it. For example:
julia> eval(parse("$fn()")) # NOT RECOMMENDED
1.464877410113412e9
While this is temptingly simple, it's not recommended since it is slow, brittle and dangerous. Parsing and evaling code is inherently much more complicated and thus slower than doing a name lookup in a module – name lookup is essentially just a hash table lookup. In Julia, where code is just-in-time compiled rather than interpreted, eval is much slower and more expensive since it doesn't just involve parsing, but also generating LLVM code, running optimization passes, emitting machine code, and then finally calling a function. Parsing and evaling a string is also brittle since all intended meaning is discarded when code is turned into text. Suppose, for example, someone accidentally provides an empty function name – then the fact that this code is intended to call a function is completely lost by accidental similarity of syntaxes:
julia> fn = ""
""
julia> eval(parse("$fn()"))
()
Oops. That's not what we wanted at all. In this case the behavior is fairly harmless but it could easily be much worse:
julia> fn = "println(\"rm -rf /important/directory\"); time"
"println(\"rm -rf /important/directory\"); time"
julia> eval(parse("$fn()"))
rm -rf /important/directory
1.448981974309033e9
If the user's input is untrusted, this is a massive security hole. Even if you trust the user, it is still possible for them to accidentally provide input that will do something unexpected and bad. The name lookup approach avoids these issues:
julia> getfield(Main, Symbol(fn))()
ERROR: UndefVarError: println("rm -rf /important/directory"); time not defined
in eval(::Module, ::Any) at ./boot.jl:225
in macro expansion at ./REPL.jl:92 [inlined]
in (::Base.REPL.##1#2{Base.REPL.REPLBackend})() at ./event.jl:46
The intent of looking up a name and then calling it as a function is explicit, instead of implicit in the generated string syntax, so at worst one gets an error about a strange name being undefined.
Performance
If you're going to call a dynamically specified function in an inner loop or as part of some recursive computation, you will want to avoid doing a getfield lookup every time you call the function. In this case all you need to do is make a const binding to the dynamically specified function before defining the iterative/recursive procedure that calls it. For example:
fn = "deg2rad" # converts angles in degrees to radians
const f = getfield(Main, Symbol(fn))
function fast(n)
t = 0.0
for i = 1:n
t += f(i)
end
return t
end
julia> #time fast(10^6) # once for JIT compilation
0.010055 seconds (2.97 k allocations: 142.459 KB)
8.72665498661791e9
julia> #time fast(10^6) # now it's fast
0.003055 seconds (6 allocations: 192 bytes)
8.72665498661791e9
julia> #time fast(10^6) # see?
0.002952 seconds (6 allocations: 192 bytes)
8.72665498661791e9
The binding f must be constant for optimal performance, since otherwise the compiler can't know that you won't change f to point at another function at any time (or even something that's not a function), so it has to emit code that looks f up dynamically on every loop iteration – effectively the same thing as if you manually call getfield in the loop. Here, since f is const, the compiler knows f can't change so it can emit fast code that just calls the right function directly. But the compiler can sometimes do even better than that – in this case it actually inlines the implementation of the deg2rad function, which is just a multiplication by pi/180:
julia> #code_llvm fast(100000)
define double #julia_fast_51089(i64) #0 {
top:
%1 = icmp slt i64 %0, 1
br i1 %1, label %L2, label %if.preheader
if.preheader: ; preds = %top
br label %if
L2.loopexit: ; preds = %if
br label %L2
L2: ; preds = %L2.loopexit, %top
%t.0.lcssa = phi double [ 0.000000e+00, %top ], [ %5, %L2.loopexit ]
ret double %t.0.lcssa
if: ; preds = %if.preheader, %if
%t.04 = phi double [ %5, %if ], [ 0.000000e+00, %if.preheader ]
%"#temp#.03" = phi i64 [ %2, %if ], [ 1, %if.preheader ]
%2 = add i64 %"#temp#.03", 1
%3 = sitofp i64 %"#temp#.03" to double
%4 = fmul double %3, 0x3F91DF46A2529D39 ; deg2rad(x) = x*(pi/180)
%5 = fadd double %t.04, %4
%6 = icmp eq i64 %"#temp#.03", %0
br i1 %6, label %L2.loopexit, label %if
}
If you need to do this with many different dynamically specified functions, then you can even pass the function to be called in as an argument:
function fast(f,n)
t = 0.0
for i = 1:n
t += f(i)
end
return t
end
julia> #time fast(getfield(Main, Symbol(fn)), 10^6)
0.007483 seconds (1.70 k allocations: 76.670 KB)
8.72665498661791e9
julia> #time fast(getfield(Main, Symbol(fn)), 10^6)
0.002908 seconds (6 allocations: 192 bytes)
8.72665498661791e9
This generates the same fast code as single-argument fast above, but will generate a new version for every different function f that you call it with.