I'm very new to Julia but I've got a some background in Scheme/Rust/F#.
Today I wanted to make yesterday's AoC nicer without an explicit number of nested loops.
I arrived at this working solution, but I don't like the last if. In the languages mentioned above I would call a function (or use a computation expression) that gives me the first result that is not None. For Julia, I expected something to do that. It does, but unexpectedly in an eager fashion.
So When I tried return something(find(r, n, start + 1, which), find(r, n - 1, start + 1, extended)), that also evaluated the second argument when the first already had a result—and thus crashed.
Is there a macro/lazy version or something that I didn't find? How are you supposed to handle a case like that?
I also thought about (short-circuited) or'ing them together, but I guess Julia's strictness in that matter spoils that.
using DataStructures
function find(r::Array{Int}, n, start = 1, which = nil())::Union{Int,Nothing}
if start <= length(r)
extended = cons(start, which)
with_current = sum(i -> r[i], extended)
if with_current == 2020 && n == 1
return prod(i -> r[i], extended)
else
# Unfortunately no :(
#return something(find(r, n, start + 1, which), find(r, n - 1, start + 1, extended))
re = find(r, n, start + 1, which)
if isnothing(re)
return find(r, n - 1, start + 1, extended)
else
re
end
end
end
end
Let me comment more on it why it is not possible given the discussion in the comments.
In Julia function arguments are evaluated eagerly, so Julia evaluates both find(r, n, start + 1, which) and find(r, n - 1, start + 1, extended) before passing them to something function.
Now, with macros you have (I am not writing in a fully general case for simplicity and I hope I got the hygiene right :)):
julia> macro something(x, y)
quote
local vx = $(esc(x))
isnothing(vx) ? $(esc(y)) : vx
end
end
#something (macro with 1 method)
julia> #something 1 2
1
julia> #something nothing 2
2
julia> #something 1 sqrt(-1)
1
julia> #something nothing sqrt(-1)
ERROR: DomainError with -1.0:
sqrt will only return a complex result if called with a complex argument. Try sqrt(Complex(x)).
(in a full-blown version of the macro varargs and Some should be handled to replicate something exactly)
Piqued by Bogumił's answer I wanted to write my first Julia macro. It took some time and numerous attempts to figure out syntax, hygiene and escaping but I'm quite happy now.
I thought it might be worth sharing and provide opportunity for suggestions/improvements.
A lazy #something analog to Base.something
function _something_impl(thing)
:(something($(esc(thing))))
end
function _something_impl(thing, rest...)
quote
local evalued = $(esc(thing))
if isnothing(evalued)
$(_something_impl(rest...))
else
something(evalued)
end
end
end
macro something(things...)
_something_impl(things...)
end
Version without exceptions
As I found exceptions raised from a macro like this not quite suitable, I also made a version that falls back to nothing.
function _something_nothing_impl(thing)
quote
local evaluated = $(esc(thing))
if isa(evaluated, Some)
evaluated.value
else
evaluated
end
end
end
function _something_nothing_impl(thing, rest...)
quote
local evalued = $(esc(thing))
if isnothing(evalued)
$(_something_nothing_impl(rest...))
else
something(evalued)
end
end
end
macro something_nothing(things...)
_something_nothing_impl(things...)
end
Now I guess the recursive middle function could also generated by a macro. :)
Related
I have a function that optionally uses threads for its main loop, doing so when an argument usingthreads is true. At the moment, the code looks like this:
function dosomething(usingthreads::Bool)
n = 1000
if usingthreads
Threads.#threads for i = 1:n
#20 lines of code here
end
else
for i = 1:n
#same 20 lines of code repeated here
end
end
end
Less nasty than the above would be to put the "20 lines" in a separate function. Is there another way?
You could use a macro that changes its behavior depending on the result of Threads.nthreads():
macro maybe_threaded(ex)
if Threads.nthreads() == 1
return esc(ex)
else
return esc(:(Threads.#threads $ex))
end
end
Without threading, this macro will be a no-op:
julia> #macroexpand #maybe_threaded for i in 1:5
print(i)
end
:(for i = 1:5
#= REPL[2]:2 =#
print(i)
end)
But when threading is enabled and e.g. JULIA_NUM_THREADS=4 it will expand to the threaded version:
julia> #maybe_threaded for i in 1:5
print(i)
end
41325
Edit: Upon rereading the question, I realize this doesn't really answer it but it might be useful anyway.
You can use ThreadsX as suggested in this discourse link.
The answer from the thread (all credit to oxinabox):
using ThreadsX
function foo(multi_thread=true)
_foreach = multi_thread ? ThreadsX.foreach : Base.foreach
_foreach(1:10) do ii
#show ii
end
end
I'm writing a genetic program in order to test the fitness of randomly generated expressions. Shown here is the function to generate the expression as well a the main function. DIV and GT are defined elsewhere in the code:
function create_single_full_tree(depth, fs, ts)
"""
Creates a single AST with full depth
Inputs
depth Current depth of tree. Initially called from main() with max depth
fs Function Set - Array of allowed functions
ts Terminal Set - Array of allowed terminal values
Output
Full AST of typeof()==Expr
"""
# If we are at the bottom
if depth == 1
# End of tree, return function with two terminal nodes
return Expr(:call, fs[rand(1:length(fs))], ts[rand(1:length(ts))], ts[rand(1:length(ts))])
else
# Not end of expression, recurively go back through and create functions for each new node
return Expr(:call, fs[rand(1:length(fs))], create_single_full_tree(depth-1, fs, ts), create_single_full_tree(depth-1, fs, ts))
end
end
function main()
"""
Main function
"""
# Define functional and terminal sets
fs = [:+, :-, :DIV, :GT]
ts = [:x, :v, -1]
# Create the tree
ast = create_single_full_tree(4, fs, ts)
#println(typeof(ast))
#println(ast)
#println(dump(ast))
x = 1
v = 1
eval(ast) # Error out unless x and v are globals
end
main()
I am generating a random expression based on certain allowed functions and variables. As seen in the code, the expression can only have symbols x and v, as well as the value -1. I will need to test the expression with a variety of x and v values; here I am just using x=1 and v=1 to test the code.
The expression is being returned correctly, however, eval() can only be used with global variables, so it will error out when run unless I declare x and v to be global (ERROR: LoadError: UndefVarError: x not defined). I would like to avoid globals if possible. Is there a better way to generate and evaluate these generated expressions with locally defined variables?
Here is an example for generating an (anonymous) function. The result of eval can be called as a function and your variable can be passed as parameters:
myfun = eval(Expr(:->,:x, Expr(:block, Expr(:call,:*,3,:x) )))
myfun(14)
# returns 42
The dump function is very useful to inspect the expression that the parsers has created. For two input arguments you would use a tuple for example as args[1]:
julia> dump(parse("(x,y) -> 3x + y"))
Expr
head: Symbol ->
args: Array{Any}((2,))
1: Expr
head: Symbol tuple
args: Array{Any}((2,))
1: Symbol x
2: Symbol y
typ: Any
2: Expr
[...]
Does this help?
In the Metaprogramming part of the Julia documentation, there is a sentence under the eval() and effects section which says
Every module has its own eval() function that evaluates expressions in its global scope.
Similarly, the REPL help ?eval will give you, on Julia 0.6.2, the following help:
Evaluate an expression in the given module and return the result. Every Module (except those defined with baremodule) has its own 1-argument definition of eval, which evaluates expressions in that module.
I assume, you are working in the Main module in your example. That's why you need to have the globals defined there. For your problem, you can use macros and interpolate the values of x and y directly inside the macro.
A minimal working example would be:
macro eval_line(a, b, x)
isa(a, Real) || (warn("$a is not a real number."); return :(throw(DomainError())))
isa(b, Real) || (warn("$b is not a real number."); return :(throw(DomainError())))
return :($a * $x + $b) # interpolate the variables
end
Here, #eval_line macro does the following:
Main> #macroexpand #eval_line(5, 6, 2)
:(5 * 2 + 6)
As you can see, the values of macro's arguments are interpolated inside the macro and the expression is given to the user accordingly. When the user does not behave,
Main> #macroexpand #eval_line([1,2,3], 7, 8)
WARNING: [1, 2, 3] is not a real number.
:((Main.throw)((Main.DomainError)()))
a user-friendly warning message is provided to the user at parse-time, and a DomainError is thrown at run-time.
Of course, you can do these things within your functions, again by interpolating the variables --- you do not need to use macros. However, what you would like to achieve in the end is to combine eval with the output of a function that returns Expr. This is what the macro functionality is for. Finally, you would simply call your macros with an # sign preceding the macro name:
Main> #eval_line(5, 6, 2)
16
Main> #eval_line([1,2,3], 7, 8)
WARNING: [1, 2, 3] is not a real number.
ERROR: DomainError:
Stacktrace:
[1] eval(::Module, ::Any) at ./boot.jl:235
EDIT 1. You can take this one step further, and create functions accordingly:
macro define_lines(linedefs)
for (name, a, b) in eval(linedefs)
ex = quote
function $(Symbol(name))(x) # interpolate name
return $a * x + $b # interpolate a and b here
end
end
eval(ex) # evaluate the function definition expression in the module
end
end
Then, you can call this macro to create different line definitions in the form of functions to be called later on:
#define_lines([
("identity_line", 1, 0);
("null_line", 0, 0);
("unit_shift", 0, 1)
])
identity_line(5) # returns 5
null_line(5) # returns 0
unit_shift(5) # returns 1
EDIT 2. You can, I guess, achieve what you would like to achieve by using a macro similar to that below:
macro random_oper(depth, fs, ts)
operations = eval(fs)
oper = operations[rand(1:length(operations))]
terminals = eval(ts)
ts = terminals[rand(1:length(terminals), 2)]
ex = :($oper($ts...))
for d in 2:depth
oper = operations[rand(1:length(operations))]
t = terminals[rand(1:length(terminals))]
ex = :($oper($ex, $t))
end
return ex
end
which will give the following, for instance:
Main> #macroexpand #random_oper(1, [+, -, /], [1,2,3])
:((-)([3, 3]...))
Main> #macroexpand #random_oper(2, [+, -, /], [1,2,3])
:((+)((-)([2, 3]...), 3))
Thanks Arda for the thorough response! This helped, but part of me thinks there may be a better way to do this as it seems too roundabout. Since I am writing a genetic program, I will need to create 500 of these ASTs, all with random functions and terminals from a set of allowed functions and terminals (fs and ts in the code). I will also need to test each function with 20 different values of x and v.
In order to accomplish this with the information you have given, I have come up with the following macro:
macro create_function(defs)
for name in eval(defs)
ex = quote
function $(Symbol(name))(x,v)
fs = [:+, :-, :DIV, :GT]
ts = [x,v,-1]
return create_single_full_tree(4, fs, ts)
end
end
eval(ex)
end
end
I can then supply a list of 500 random function names in my main() function, such as ["func1, func2, func3,.....". Which I can eval with any x and v values in my main function. This has solved my issue, however, this seems to be a very roundabout way of doing this, and may make it difficult to evolve each AST with each iteration.
I'd like to write an iterator that behaves exactly like ipairs, except which takes a second argument. The second argument would be a table of the indices that ipairs should loop over.
I'm wondering if my current approach is inefficient, and how I could improve it with closures.
I'm also open to other methods of accomplishing the same thing. But I like iterators because they're easy to use and debug.
I'll be making references to and using some of the terminology from Programming in Lua (PiL), especially the chapter on closures (chapter 7 in the link).
So I'd like to have this,
ary = {10,20,30,40}
for i,v in selpairs(ary, {1,3}) do
ary[i] = v+5
print(string.format("ary[%d] is now = %g", i, ary[i]))
end
which would output this:
ary[1] is now = 15
ary[3] is now = 35
My current approach is this : (in order: iterator, factory, then generic for)
iter = function (t, s)
s = s + 1
local i = t.sel[s]
local v = t.ary[i]
if v then
return s, i, v
end
end
function selpairs (ary, sel)
local t = {}
t.ary = ary
t.sel = sel
return iter, t, 0
end
ary = {10,20,30,40}
for _,i,v in selpairs(ary, {1,3}) do
ary[i] = v+5
print(string.format("ary[%d] is now = %g", i, ary[i]))
end
-- same output as before
It works. sel is the array of 'selected' indices. ary is the array you want to perform the loop on. Inside iter, s indexes sel, and i indexes ary.
But there are a few glaring problems.
I must always discard the first returned argument s (_ in the for loop). I never need s, but it has to be returned as the first argument since it is the "control variable".
The "invariant state" is actually two invariant states (ary and sel) packed into a single table. Pil says that this is more expensive, and recommends using closures. (Hence my writing this question).
The rest can of this can be ignored. I'm just providing more context for what I'm wanting to use selpairs for.
I'm mostly concerned with the second problem. I'm writing this for a library I'm making for generating music. Doing simple stuff like ary[i] = v+5 won't really be a problem. But when I do stuff like accessing object properties and checking bounds, then I get concerned that the 'invariant state as a table' approach may be creating unnecessary overhead. Should I be concerned about this?
If anything, I'd like to know how to write this with closures just for the knowledge.
Of course, I've tried using closures, but I'm failing to understand the scope of "locals in enclosing functions" and how it relates to a for loop calling an iterator.
As for the first problem, I imagine I could make the control variable a table of s, i, and v. And at the return in iter, unpack the table in the desired order.
But I'm guessing that this is inefficient too.
Eventually, I'd like to write an iterator which does this, except nested into itself. My main data structure is arrays of arrays, so I'd hope to make something like this:
ary_of_arys = {
{10, 20, 30, 40},
{5, 6, 7, 8},
{0.9, 1, 1.1, 1.2},
}
for aoa,i,v in selpairs_inarrays(ary_of_arys, {1,3}, {2,3,4}) do
ary_of_arys[aoa][i] = v+5
end
And this too, could use the table approach, but it'd be nice to know how to take advantage of closures.
I've actually done something similar: A function that basically does the same thing by taking a function as it's fourth and final argument. It works just fine, but would this be less inefficient than an iterator?
You can hide "control variable" in an upvalue:
local function selpairs(ary, sel)
local s = 0
return
function()
s = s + 1
local i = sel[s]
local v = ary[i]
if v then
return i, v
end
end
end
Usage:
local ary = {10,20,30,40}
for i, v in selpairs(ary, {1,3}) do
ary[i] = v+5
print(string.format("ary[%d] is now = %g", i, ary[i]))
end
Nested usage:
local ary_of_arys = {
{10, 20, 30, 40},
{5, 6, 7, 8},
{0.9, 1, 1.1, 1.2},
}
local outer_indices = {1,3}
local inner_indices = {2,3,4}
for aoa, ary in selpairs(ary_of_arys, outer_indices) do
for i, v in selpairs(ary, inner_indices) do
ary[i] = v+5 -- This is the same as ary_of_arys[aoa][i] = v+5
end
end
Not sure if I understand what you want to achive but why not simply write
local values = {"a", "b", "c", "d"}
for i,key in ipairs {3,4,1} do
print(values[key])
end
and so forth, instead of implementing all that interator stuff? I mean your use case is rather simple. It can be easily extended to more dimensions.
And here's a co-routine based possibility:
function selpairs(t,selected)
return coroutine.wrap(function()
for _,k in ipairs(selected) do
coroutine.yield(k,t[k])
end
end)
end
Like in R:
a <- 2
or even better
a ← 2
which should translate to
a = 2
and if possible respect method overloading.
= is overloaded (not in the multiple dispatch sense) a lot in Julia.
It binds a new variable. As in a = 3. You won't be able to use ← instead of = in this context, because you can't overload binding in Julia.
It gets lowered to setindex!. As in, a[i] = b gets lowered to setindex!(a, b, i). Unfortunately, setindex! takes 3 variables while ← can only take 2 variables. So you can't overload = with 3 variables.
But, you can use only 2 variables and overload a[:] = b, for example. So, you can define ←(a,b) = (a[:] = b) or ←(a,b) = setindex!(a,b,:).
a .= b gets lowered to (Base.broadcast!)(Base.identity, a, b). You can overload this by defining ←(a,b) = (a .= b) or ←(a,b) = (Base.broadcast!)(Base.identity, a, b).
So, there are two potentially nice ways of using ←. Good luck ;)
Btw, if you really want to use ← to do binding (like in 1.), the only way to do it is using macros. But then, you will have to write a macro in front of every single assignment, which doesn't look very good.
Also, if you want to explore how operators get lowered in Julia, do f(a,b) = (a .= b), for example, and then #code_lowered f(x,y).
No. = is not an operator in Julia, and cannot be assigned to another symbol.
Disclaimer: You are fully responsible if you will try my (still beginner's) experiments bellow! :P
MacroHelper is module ( big thanks to #Alexander_Morley and #DanGetz for help ) I plan to play with in future and we could probably try it here :
julia> module MacroHelper
# modified from the julia source ./test/parse.jl
function parseall(str)
pos = start(str)
exs = []
while !done(str, pos)
ex, pos = parse(str, pos) # returns next starting point as well as expr
ex.head == :toplevel ? append!(exs, ex.args) : push!(exs, ex)
end
if length(exs) == 0
throw(ParseError("end of input"))
elseif length(exs) == 1
return exs[1]
else
return Expr(:block, exs...) # convert the array of expressions
# back to a single expression
end
end
end;
With module above you could define simple test "language":
julia> module TstLang
export #tst_cmd
import MacroHelper
macro tst_cmd(a)
b = replace("$a", "←", "=") # just simply replacing ←
# in real life you would probably like
# to escape comments, strings etc
return MacroHelper.parseall(b)
end
end;
And by using it you could probably get what you want:
julia> using TstLang
julia> tst```
a ← 3
println(a)
a +← a + 3 # maybe not wanted? :P
```
3
9
What about performance?
julia> function test1()
a = 3
a += a + 3
end;
julia> function test2()
tst```
a ← 3
a +← a + 3
```
end;
julia> test1(); #time test1();
0.000002 seconds (4 allocations: 160 bytes)
julia> test2(); #time test2();
0.000002 seconds (4 allocations: 160 bytes)
If you like to see syntax highlight (for example in atom editor) then you need to use it differently:
function test3()
#tst_cmd begin
a ← 3
a ← a + a + 3 # parser don't allow you to use "+←" here!
end
end;
We could hope that future Julia IDEs could syntax highlight cmd macros too. :)
What could be problem with "solution" above? I am not so experienced julian so many things. :P (in this moment something about "macro hygiene" and "global scope" comes to mind...)
But what you want is IMHO good for some domain specific languages and not to redefine basic of language! It is because readability very counts and if everybody will redefine everything then it will end in Tower of Babel...
Im looking for a function like Pythons
"foobar, bar, foo".count("foo")
Could not find any functions that seemed able to do this, in a obvious way. Looking for a single function or something that is not completely overkill.
Julia-1.0 update:
For single-character count within a string (in general, any single-item count within an iterable), one can use Julia's count function:
julia> count(i->(i=='f'), "foobar, bar, foo")
2
(The first argument is a predicate that returns a ::Bool).
For the given example, the following one-liner should do:
julia> length(collect(eachmatch(r"foo", "bar foo baz foo")))
2
Julia-1.7 update:
Starting with Julia-1.7 Base.Fix2 can be used, through ==('f') below, as to shorten and sweeten the syntax:
julia> count(==('f'), "foobar, bar, foo")
2
What about regexp ?
julia> length(matchall(r"ba", "foobar, bar, foo"))
2
I think that right now the closest built-in thing to what you're after is the length of a split (minus 1). But it's not difficult to specifically create what you're after.
I could see a searchall being generally useful in Julia's Base, similar to matchall. If you don't care about the actual indices, you could just use a counter instead of growing the idxs array.
function searchall(s, t; overlap::Bool=false)
idxfcn = overlap ? first : last
r = findnext(s, t, firstindex(t))
idxs = typeof(r)[] # Or to only count: n = 0
while r !== nothing
push!(idxs, r) # n += 1
r = findnext(s, t, idxfcn(r) + 1)
end
idxs # return n
end
Adding an answer to this which allows for interpolation:
julia> a = ", , ,";
julia> b = ",";
julia> length(collect(eachmatch(Regex(b), a)))
3
Actually, this solution breaks for some simple cases due to use of Regex. Instead one might find this useful:
"""
count_flags(s::String, flag::String)
counts the number of flags `flag` in string `s`.
"""
function count_flags(s::String, flag::String)
counter = 0
for i in 1:length(s)
if occursin(flag, s)
s = replace(s, flag=> "", count=1)
counter+=1
else
break
end
end
return counter
end
Sorry to post another answer instead of commenting previous one, but i've not managed how to deal with code blocks in comments :)
If you don't like regexps, maybe a tail recursive function like this one (using the search() base function as Matt suggests) :
function mycount(what::String, where::String)
function mycountacc(what::String, where::String, acc::Int)
res = search(where, what)
res == 0:-1 ? acc : mycountacc(what, where[last(res) + 1:end], acc + 1)
end
what == "" ? 0 : mycountacc(what, where, 0)
end
This is simple and fast (and does not overflow the stack):
function mycount2(where::String, what::String)
numfinds = 0
starting = 1
while true
location = search(where, what, starting)
isempty(location) && return numfinds
numfinds += 1
starting = location.stop + 1
end
end
one liner: (Julia 1.3.1):
julia> sum([1 for i = eachmatch(r"foo", "foobar, bar, foo")])
2
Since Julia 1.3, there has been a count method that does exactly this.
count(
pattern::Union{AbstractChar,AbstractString,AbstractPattern},
string::AbstractString;
overlap::Bool = false,
)
Return the number of matches for pattern in string.
This is equivalent to calling length(findall(pattern, string)) but more
efficient.
If overlap=true, the matching sequences are allowed to overlap indices in the
original string, otherwise they must be from disjoint character ranges.
│ Julia 1.3
│
│ This method requires at least Julia 1.3.
julia> count("foo", "foobar, bar, foo")
2
julia> count("ana", "bananarama")
1
julia> count("ana", "bananarama", overlap=true)
2