In MATLAB, there are a pair of functions tic and toc
which can be used to start and stop a stopwatch timer.
An example taken from link:
tic
A = rand(12000, 4400);
B = rand(12000, 4400);
toc
C = A'.*B';
toc
I am aware that there is a macro #time in Julia
that has similar functionality.
julia> #time [sin(cos(i)) for i in 1:100000];
elapsed time: 0.00721026 seconds (800048 bytes allocated)
Is there a set of similar functions in Julia?
The #time macro works well for timing statements
that can be written in one or two lines.
For longer portions of code,
I would prefer to use tic-toc functions.
What I tried
When I googled "julia stopwatch",
I found one useful link and four unrelated links.
Introducing Julia/Metaprogramming - Wikibooks, open ...
Meta-programming is when you write Julia code to process and modify Julia code. ... The #time macro inserts a "start the stopwatch" command at the beginning ...
Our Invisible Stopwatch promo - YouTube
Video for julia stopwatch
Julia Larson on Twitter: "This #Mac OSX timer/stopwatch is ...
Timing episodes of The French Chef with a stopwatch
julia griffith | Oiselle Running Apparel for Women
I don't know why I hadn't thought of just trying tic() and toc().
tic() and toc() have been deprecated as of https://github.com/JuliaLang/julia/commit/1b023388f49e13e7a42a899c12602d0fd5d60b0a
You can use #elapsed and #time for longer chunks by wrapping them in an environment, like so:
t = #elapsed begin
...
end
There is also TickTock.jl, which has reimplemented tic() and toc() and tick() and tock().
using TickTock
tick()
# Started timer at 2017-12-13T22:30:59.632
tock()
# 55.052638936 ms: 55 seconds, 52 milliseconds
From a search of the Julia documentation
tic()
Set a timer to be read by the next call to toc() or toq(). The
macro call #time expr can also be used to time evaluation.
You can also use time() to get similar output. For example:
t = time()
# code block here
# ...
# ...
dt = time() - t
Related
I have a piece of Python code which a want to convert to Julia. I the python code I use the schedule package. What is the equivalent in Julia, I looked at the "Tasks and Parallel Computing" part in the Julia documentation but I can't find something similar. The code in Python is:
def main():
schedule.every(0.25).seconds.do(read_modbus, 1, 1000, 100, 1)
while True:
schedule.run_pending()
time.sleep(0.05)
Will a Timer work? This form of Timer calls your function in a Task so you need to yield control from you main loop occasionally to allow the timer task to run. You can yield by calling yield, sleep, wait, or doing IO, here I show waiting on the timer.
tstart = time()
ncalls = 0
read_modbus() = (global ncalls+=1;#show (time()-tstart)/ncalls,ncalls)
t=Timer((timer)->read_modbus(),0,0.25)
while true
wait(t) # wait for timer to go off
println("mainloop $ncalls")
end
I noticed that a scheduling library for Julia was missing so I wrote one https://github.com/scls19fr/ExtensibleScheduler.jl
Here is a basic example using a blocking scheduler.
using ExtensibleScheduler
function read_modbus(p1, p2, p3, p4)
println("Read_modbus with $p1 $p2 $p3 $p4")
end
sched = BlockingScheduler()
add(sched, Action(read_modbus, 1, 1000, 100, 1), Trigger(Dates.Millisecond(250)))
run(sched)
Althought, this is a work in progress but contributers are welcome.
Currently (2017-12), there is only an implementation of a blocking scheduler but it should be possible to add multithreading.
I have read the documentation for the #async and #sync macros but still cannot figure out how and when to use them, nor can I find many resources or examples for them elsewhere on the internet.
My immediate goal is to find a way to set several workers to do work in parallel and then wait until they have all finished to proceed in my code. This post: Waiting for a task to be completed on remote processor in Julia contains one successful way to accomplish this. I had thought it should be possible using the #async and #sync macros, but my initial failures to accomplish this made me wonder if I am understanding properly how and when to use these macros.
According to the documentation under ?#async, "#async wraps an expression in a Task." What this means is that for whatever falls within its scope, Julia will start this task running but then proceed to whatever comes next in the script without waiting for the task to complete. Thus, for instance, without the macro you will get:
julia> #time sleep(2)
2.005766 seconds (13 allocations: 624 bytes)
But with the macro, you get:
julia> #time #async sleep(2)
0.000021 seconds (7 allocations: 657 bytes)
Task (waiting) #0x0000000112a65ba0
julia>
Julia thus allows the script to proceed (and the #time macro to fully execute) without waiting for the task (in this case, sleeping for two seconds) to complete.
The #sync macro, by contrast, will "Wait until all dynamically-enclosed uses of #async, #spawn, #spawnat and #parallel are complete." (according to the documentation under ?#sync). Thus, we see:
julia> #time #sync #async sleep(2)
2.002899 seconds (47 allocations: 2.986 KB)
Task (done) #0x0000000112bd2e00
In this simple example then, there is no point to including a single instance of #async and #sync together. But, where #sync can be useful is where you have #async applied to multiple operations that you wish to allow to all start at once without waiting for each to complete.
For example, suppose we have multiple workers and we'd like to start each of them working on a task simultaneously and then fetch the results from those tasks. An initial (but incorrect) attempt might be:
using Distributed
cell(N) = Vector{Any}(undef, N)
addprocs(2)
#time begin
a = cell(nworkers())
for (idx, pid) in enumerate(workers())
a[idx] = remotecall_fetch(sleep, pid, 2)
end
end
## 4.011576 seconds (177 allocations: 9.734 KB)
The problem here is that the loop waits for each remotecall_fetch() operation to finish, i.e. for each process to complete its work (in this case sleeping for 2 seconds) before continuing to start the next remotecall_fetch() operation. In terms of a practical situation, we're not getting the benefits of parallelism here, since our processes aren't doing their work (i.e. sleeping) simultaneously.
We can correct this, however, by using a combination of the #async and #sync macros:
#time begin
a = cell(nworkers())
#sync for (idx, pid) in enumerate(workers())
#async a[idx] = remotecall_fetch(sleep, pid, 2)
end
end
## 2.009416 seconds (274 allocations: 25.592 KB)
Now, if we count each step of the loop as a separate operation, we see that there are two separate operations preceded by the #async macro. The macro allows each of these to start up, and the code to continue (in this case to the next step of the loop) before each finishes. But, the use of the #sync macro, whose scope encompasses the whole loop, means that we won't allow the script to proceed past that loop until all of the operations preceded by #async have completed.
It is possible to get an even more clear understanding of the operation of these macros by further tweaking the above example to see how it changes under certain modifications. For instance, suppose we just have the #async without the #sync:
#time begin
a = cell(nworkers())
for (idx, pid) in enumerate(workers())
println("sending work to $pid")
#async a[idx] = remotecall_fetch(sleep, pid, 2)
end
end
## 0.001429 seconds (27 allocations: 2.234 KB)
Here, the #async macro allows us to continue in our loop even before each remotecall_fetch() operation finishes executing. But, for better or worse, we have no #sync macro to prevent the code from continuing past this loop until all of the remotecall_fetch() operations finish.
Nevertheless, each remotecall_fetch() operation is still running in parallel, even once we go on. We can see that because if we wait for two seconds, then the array a, containing the results, will contain:
sleep(2)
julia> a
2-element Array{Any,1}:
nothing
nothing
(The "nothing" element is the result of a successful fetch of the results of the sleep function, which does not return any values)
We can also see that the two remotecall_fetch() operations start at essentially the same time because the print commands that precede them also execute in rapid succession (output from these commands not shown here). Contrast this with the next example where the print commands execute at a 2 second lag from each other:
If we put the #async macro on the whole loop (instead of just the inner step of it), then again our script will continue immediately without waiting for the remotecall_fetch() operations to finish. Now, however, we only allow for the script to continue past the loop as a whole. We don't allow each individual step of the loop to start before the previous one finished. As such, unlike in the example above, two seconds after the script proceeds after the loop, there is the results array still has one element as #undef indicating that the second remotecall_fetch() operation still has not completed.
#time begin
a = cell(nworkers())
#async for (idx, pid) in enumerate(workers())
println("sending work to $pid")
a[idx] = remotecall_fetch(sleep, pid, 2)
end
end
# 0.001279 seconds (328 allocations: 21.354 KB)
# Task (waiting) #0x0000000115ec9120
## This also allows us to continue to
sleep(2)
a
2-element Array{Any,1}:
nothing
#undef
And, not surprisingly, if we put the #sync and #async right next to each other, we get that each remotecall_fetch() runs sequentially (rather than simultaneously) but we don't continue in the code until each has finished. In other words, this would be, I believe, essentially the equivalent of if we had neither macro in place, just like sleep(2) behaves essentially identically to #sync #async sleep(2)
#time begin
a = cell(nworkers())
#sync #async for (idx, pid) in enumerate(workers())
a[idx] = remotecall_fetch(sleep, pid, 2)
end
end
# 4.019500 seconds (4.20 k allocations: 216.964 KB)
# Task (done) #0x0000000115e52a10
Note also that it is possible to have more complicated operations inside the scope of the #async macro. The documentation gives an example containing an entire loop within the scope of #async.
Update: Recall that the help for the sync macros states that it will "Wait until all dynamically-enclosed uses of #async, #spawn, #spawnat and #parallel are complete." For the purposes of what counts as "complete" it matters how you define the tasks within the scope of the #sync and #async macros. Consider the below example, which is a slight variation on one of the examples given above:
#time begin
a = cell(nworkers())
#sync for (idx, pid) in enumerate(workers())
#async a[idx] = remotecall(sleep, pid, 2)
end
end
## 0.172479 seconds (93.42 k allocations: 3.900 MB)
julia> a
2-element Array{Any,1}:
RemoteRef{Channel{Any}}(2,1,3)
RemoteRef{Channel{Any}}(3,1,4)
The earlier example took roughly 2 seconds to execute, indicating that the two tasks were run in parallel and that the script waiting for each to complete execution of their functions before proceeding. This example, however, has a much lower time evaluation. The reason is that for the purposes of #sync the remotecall() operation has "finished" once it has sent the worker the job to do. (Note that the resulting array, a, here, just contains RemoteRef object types, which just indicate that there is something going on with a particular process which could in theory be fetched at some point in the future). By contrast, the remotecall_fetch() operation has only "finished" when it gets the message from the worker that its task is complete.
Thus, if you are looking for ways to ensure that certain operations with workers have completed before moving on in your script (as for instance is discussed in this post: Waiting for a task to be completed on remote processor in Julia) it is necessary to think carefully about what counts as "complete" and how you will measure and then operationalize that in your script.
I have a piece of code in Julia in which a solver iterates many, many times as it seeks a solution to a very complex problem. At present, I have to provide a number of iterations for the code to do, set low enough that I don't have to wait hours for the code to halt in order to save the current state, but high enough that I don't have to keep activating the code every 5 minutes.
Is there a way, with the current state of Julia (0.2), to detect a keystroke instructing the code to either end without saving (in case of problems) or end with saving? I require a method such that the code will continue unimpeded unless such a keystroke event has happened, and that will interrupt on any iteration.
Essentially, I'm looking for a command that will read in a keystroke if a keystroke has occurred (while the terminal that Julia is running in has focus), and run certain code if the keystroke was a specific key. Is this possible?
Note: I'm running julia via xfce4-terminal on Xubuntu, in case that affects the required command.
You can you an asynchronous task to read from STDIN, blocking until something is available to read. In your main computation task, when you are ready to check for input, you can call yield() to lend a few cycles to the read task, and check a global to see if anything was read. For example:
input = ""
#async while true
global input = readavailable(STDIN)
end
for i = 1:10^6 # some long-running computation
if isempty(input)
yield()
else
println("GOT INPUT: ", input)
global input = ""
end
# do some other work here
end
Note that, since this is cooperative multithreading, there are no race conditions.
You may be able to achieve this by sending an interrupt (Ctrl+C). This should work from the REPL without any changes to your code – if you want to implement saving you'll have to handle the resulting InterruptException and prompt the user.
I had some trouble with the answer from steven-g-johnson, and ended up using a Channel to communicate between tasks:
function kbtest()
# allow 'q' pressed on the keyboard to break the loop
quitChannel = Channel(10)
#async while true
kb_input = readline(stdin)
if contains(lowercase(kb_input), "q")
put!(quitChannel, 1)
break
end
end
start_time = time()
while (time() - start_time) < 10
if isready(quitChannel)
break
end
println("in loop # $(time() - start_time)")
sleep(1)
end
println("out of loop # $(time() - start_time)")
end
This requires pressing and then , which works well for my needs.
So, today I decided to try Julia and I came across something odd I couldn't quite understand the reason for nor find a decent answer to with my search queries so here I am...
First, I wanted to have something to benchmark Python against, I settled for this very simple piece of code.
def test():
start = time()
a = 1
while a < 10000000:
a+=1
print(time() - start)
This took ~0.9s to execute in Python 3.3 on my machine. Then I ran the following in Julia.
start = time()
a = 1
while a < 10000000
a+=1
end
print(time() - start)
It took ~0.7s to execute. So I concluded that simple arithmetic performance in Julia is ~= to Python.
However, when I made it into a function, I stumbled upon the weirdness I wasn't expecting which turned my result on its head.
function test_arithmetic()
start = time()
a = 1
while a < 10000000
a+=1
end
print(time() - start)
end
test_arithmetic()
This codesnippet took only ~0.1s to execute, why is this?
The reason is that a global variable can have its type changed at any point, which makes it hard for the compiler to optimize. This is mentioned in the Performance Tips section of the manual.
I found the answer, it has to do with how it is faster to store local variables compared to globals.
The equivalent question for Python (which led me to test if the same applied to Julia) can be found here.
It turns out that this code-snippet runs in ~0.7s as well.
function test_arithmetic()
start = time()
global a = 1
while a < 10000000
a+=1
end
print(time() - start)
end
test_arithmetic()
I hacked together an Emacs function in order to send
tOne <- proc.time()[3]
before my "send-to-R" key, followed by
tTwo <- proc.time()[3]
afterwards, then printing the difference. The printing gets quite messy though.
Is there a better way in R to automatically time everything send to R? (such as in F# #time "on")
EDIT: Currently, it sends some extra newlines since the inferior buffer needs the strings to be sent:
> > a<-ocorrelate.parallel(replicate(2, rnorm(100000)), 0.5)
>
+ user 0.072 sys 0.020 elapsed 14.925
> > a<-ocorrelate.parallel(replicate(2, rnorm(100000)), 0.5)
>
+ user 0.088 sys 0.032 elapsed 16.868
> >
Function:
(defun ess-timed-cc (vis)
(interactive "P")
(process-send-string "R" "tone <- proc.time()[1:3];")
(ess-eval-region-or-function-or-paragraph-and-step vis)
(process-send-string "R" "ttwo <- proc.time()[1:3]; cat(paste(c(\"\",
format(ttwo-tone)), c(\"user\", \"sys\", \"elapsed\", \"\n\")));")
(other-window 1)
(inferior-ess-send-input)
(inferior-ess-send-input)
(goto-char (point-max))
(other-window -1)
)
You can turn on profiling in R and it will tell you the relative amount of time spent in each function, that may be what you want. See ?Rprof for details.
You could also use addTaskCallback to add a callback to show you the time since the last expression finished, though this time would include any idle time and the time to type the expression, not just the run time. If you have all the commands already in a file and just send them to the command line then this should work reasonably well.
There may also be some hooks that you could set that would start and stop the timing, but not all functions have hooks.
For the emacs solution you could use that to wrap the call in system.time instead of calling proc.time twice and subtracting.
You could also use the trace function to insert the 2 calls to proc.time at the beginning and end of each function that you wanted to time. This would require a vector of the names of the functions that you wanted to time, but ls could help with that.