I'm trying to convert the following to pointfree style: a function that partially applies a value to the transformer function add before passing in the collection to be iterated over. (Using Ramda.js)
R.compose(
R.map,
R.add
)(1, [1,2,3])
The problem is that R.add is arity 2, as is R.map. I want the application order to be as follows:
add(1)
map(add(1))
map(add(1), [1,2,3])
[add(1,1), add(1,2), add(1,3)]
But what happens instead is this:
add(1, [1,2,3])
map(add(1, [1,2,3]))
<partially applied map, waiting for collection>
Anyone know of a way to specify this behavior?
A plain compose or pipe won't do this because either will absorb all the arguments supplied into the first function. Ramda includes two additional functions that help with this, converge and useWith. In this case useWith is the one that will help:
useWith(map, [add, identity])(1, [1, 2, 3]); //=> [2, 3, 4]
While identity is not absolutely required here, it gives the generated function the correct arity.
Figured it out. If anyone's curious, here's the gist. (You can try it in the console on RamdaJS.com.)
0) For a baseline, here's the pointed version.
func0 = x => R.map(R.add(x))
addOne = func0(1)
addOne([1,2,3]) // [2,3,4]
1) Here's the pointfree core, but it has the ordering problem from the question above.
func1 = R.compose(R.map, R.add)
addOne = func1(1)
addOne([1,2,3]) // [2,3,4])
func1(1, [1,2,3]) // function
2) If the composition is unary (arity 1) 2 invocations are needed apply all params.
func2 = R.unary(R.compose(R.map, R.add))
addOne = func2(1)
addOne([1,2,3]) // [2,3,4])
3) We want one invocation to apply both params, so we uncurry 2.
func3 = R.uncurryN(2, func2)
func3(1, [1,2,3]) // [2,3,4])
4) To prove func2 is composable, let's double the results.
func4 = R.compose(
R.map(R.multiply(2)),
R.uncurryN(2, func2)
)
func4(1, [1,2,3]) // [4,6,8])
5) Substitution gives us a completely pointfree function.
func5 = R.compose(
R.map(R.multiply(2)),
R.uncurryN(2, R.unary(R.compose(
R.map,
R.add
)))
)
func5(1, [1,2,3]) // [4,6,8])
Related
My question is instead of F = svd(A), can one first allocate an appropriate memory for an SVD structure, and then do F .= svd(A) ?
What I had in mind is something like the following:
function main()
F = Vector{SVD}(undef,10)
# how to preallocate F?
test(F)
end
function test(F::Vector{SVD})
for i in 1:10
F .= svd(rand(3,3))
end
end
Your code almost works. But what you probably wanted was this:
using LinearAlgebra
function main()
F = Vector{SVD}(undef, 10)
test(F)
end
function test(F::Vector{SVD})
for i in 1:10
F[i] = svd(rand(3, 3))
end
return F
end
The line that you had in the for loop was this:
F .= svd(rand(3,3))
which does the same operation on every loop, since you were not indexing into F. In particular, this operation was trying to broadcast a single SVD object into all the fields of F on each iteration of the loop. (And that broadcast operation failed because by default structs are treated as iterable objects with a length method, but SVD does not have a length method.)
However, I would recommend against pre-allocating a vector in this situation. First, let's look at the type of F:
julia> typeof(Vector{SVD}(undef, 10))
Array{SVD,1}
The problem with this vector is that it is parameterized by an abstract type. There is a section in the Performance Tips chapter of the manual that advises against this. SVD is an abstract type because the types of its parameters have not been specified. To make it concrete, you need to specify the types of the parameters, like this:
julia> SVD{Float64,Float64,Array{Float64,2}}
SVD{Float64,Float64,Array{Float64,2}}
julia> Vector{SVD{Float64,Float64,Array{Float64,2}}}(undef, 2)
2-element Array{SVD{Float64,Float64,Array{Float64,2}},1}:
#undef
#undef
As you can see, it is difficult to correctly specify the concrete type when you are working with complicated types like SVD. Additionally, if you do so, your code will not be as generic as it could be.
A better approach for a problem like this is to use mapping, broadcasting, or a list comprehension. Then the correct output type will automatically be inferred. Here are some examples:
List comprehension
julia> [svd(rand(3, 3)) for _ in 1:2]
2-element Array{SVD{Float64,Float64,Array{Float64,2}},1}:
SVD{Float64,Float64,Array{Float64,2}}([-0.6357040496635746 -0.2941425771794837 -0.7136949667270628; -0.45459999623274916 -0.6045700314848496 0.654090147040599; -0.6238743500629883 0.7402534845042064 0.2506104028424691], [1.4535849689665463, 0.7212190827260345, 0.05010669163393896], [-0.5975505057447164 -0.588792736048385 -0.5442945039782142; 0.7619724725128861 -0.6283345569895092 -0.15682358121595258; -0.2496624605679292 -0.5084474392397449 0.8241054891903787])
SVD{Float64,Float64,Array{Float64,2}}([-0.5593632049776268 0.654338345992878 -0.5088753618327984; -0.6687620652652163 -0.7189576326033171 -0.18936003428293915; -0.4897653570633183 0.23439550227070827 0.8397551092645418], [1.8461274187259178, 0.21226179692488983, 0.14194607536315287], [-0.29089551972856004 -0.7086270946133293 -0.6428276887173754; -0.9203610429640889 0.023709029028269546 0.390350397126212; 0.2613720474647311 -0.7051847436823973 0.6590896221923739])
Map
julia> map(_ -> svd(rand(3, 3)), 1:2)
2-element Array{SVD{Float64,Float64,Array{Float64,2}},1}:
SVD{Float64,Float64,Array{Float64,2}}([-0.5807809149601634 0.5635242755434755 0.5874809951745127; -0.6884131975465821 0.0451903888051729 -0.7239095925620322; -0.43448912329507794 -0.8248625459025509 0.3616918330643316], [1.488618654040125, 0.4122166626927311, 0.004235624485479941], [-0.6721098925787947 -0.2684664121709399 -0.6900681689759235; -0.7384292974335966 0.31185073633575333 0.5978890289498324; -0.05468514413847799 -0.9114136842196914 0.4078414290231468])
SVD{Float64,Float64,Array{Float64,2}}([-0.3677873424759118 0.8090638526628051 -0.4584191892023337; -0.43071684640222546 -0.5851169278783189 -0.6871107472129654; -0.8241452960126802 -0.055261768200600137 0.5636760310989947], [1.6862363968739773, 0.5899255050748418, 0.24246688716190598], [-0.3751742784957875 -0.7172409091515735 -0.5872050229643736; 0.8600668700980193 -0.505618838823938 0.06807766730822862; -0.3457300098559026 -0.4794945964927631 0.8065703268899])
Broadcasting
julia> g = (rand(3, 3) for _ in 1:2)
Base.Generator{UnitRange{Int64},var"#17#18"}(var"#17#18"(), 1:2)
julia> svd.(g)
2-element Array{SVD{Float64,Float64,Array{Float64,2}},1}:
SVD{Float64,Float64,Array{Float64,2}}([-0.7988295268840152 0.5443221484534134 -0.256095266807727; -0.5436890668169485 -0.8354777569473182 -0.0798693700362902; -0.257436566171119 0.07543418554831638 0.963346302244777], [1.8188722412547844, 0.3934389096422389, 0.2020398396772306], [-0.7147404794808727 -0.37763644211761316 -0.5886737335538281; -0.6944558966482991 0.4830041206449164 0.5333273169925189; -0.08292800854873916 -0.7899985677359054 0.607474450798845])
SVD{Float64,Float64,Array{Float64,2}}([-0.5910620103531503 0.3599866268397522 0.7218416228050514; -0.7367495542691711 0.12340124384185132 -0.664809918173956; -0.3283988340440176 -0.9247603805931685 0.1922821996018057], [1.826019614357666, 0.5333148215847028, 0.11639139812894106], [-0.6415954756495915 -0.6888196183142843 -0.33746522643279503; -0.5845558664639438 0.7239484700883465 -0.3663236978948133; -0.4966383841474222 0.037764349353666515 0.8671356118331964])
Furthermore, mapping, broadcasting, and list comprehensions should be just as efficient as pre-allocating the vector. If you're doing a simple mapping, then it's usually easier and more readable to use mapping, broadcasting, or list comprehensions. Pre-allocating vectors is a tool I reserve for writing custom algorithms from scratch.
A final note. In most cases, type parameters are considered an implementation detail and are not a part of the public API for a type. As such, it's best to use generic programming approaches that do not rely on fixing the types for type parameters. Of course there are some exceptions to this rule of thumb, like Array{T,N} and Dict{K,V}.
There's a differnent way of preallocation -- you can reuse the input array by always overwriting it, with both the rand call and svd's internal needs:
function test!(F::Vector{SVD})
A = Matrix{Float64}(undef, 3, 3)
for i in 1:10
rand!(A)
F[i] = svd!(A)
end
end
Cameron's advice still holds. I'd probably use something like
function test()
A = Matrix{Float64}(undef, 3, 3)
return map(1:10) do i
svd!(rand!(A))
end
end
given that the number of loops seems not be the critical part.
After really annoying and long debugging, I found that apply() does not pass arguments to functions via ...! Here is a simplified example (this is a simplified example to demonstrate the error):
A <- matrix(0, nrow = 2, ncol = 2)
apply(A, 1, function (x, ...) { cat("argument arg1: "); print(arg1); }, arg1 = 10)
# Argument arg1: Error in print(arg1) (from #1) : object 'arg1' not found
Do you have any idea why or what to do with this? Workaround is obvious, to list all arguments instead of using ..., which is anoying since I use this as a wrapper for other more complex functions. Any thoughts?
The problem isn't that that argument is not being passed to the function (it is), the problem is you are not "catching" it via a named parameter. This works for example
apply(A, 1, function (x, ..., arg1) { cat("argument arg1: "); print(arg1); }, arg1 = 10)
And we can use that variable as arg1 in the function because we caught it. Otherwise it's left inside the ... so you can pass it along to another function. For example we can just pass everything to list like this...
apply(A, 1, function (x, ...) { print(list(...)) }, arg1 = 10)
So since your function uses ... those values that aren't named stay "in" the dots. In order to get them out you need to capture them as arguments.
When you want to pass more than one argument, you may use mappy() instead.
mapply(your_function, arg1, arg2, argn
I'm writing a genetic program in order to test the fitness of randomly generated expressions. Shown here is the function to generate the expression as well a the main function. DIV and GT are defined elsewhere in the code:
function create_single_full_tree(depth, fs, ts)
"""
Creates a single AST with full depth
Inputs
depth Current depth of tree. Initially called from main() with max depth
fs Function Set - Array of allowed functions
ts Terminal Set - Array of allowed terminal values
Output
Full AST of typeof()==Expr
"""
# If we are at the bottom
if depth == 1
# End of tree, return function with two terminal nodes
return Expr(:call, fs[rand(1:length(fs))], ts[rand(1:length(ts))], ts[rand(1:length(ts))])
else
# Not end of expression, recurively go back through and create functions for each new node
return Expr(:call, fs[rand(1:length(fs))], create_single_full_tree(depth-1, fs, ts), create_single_full_tree(depth-1, fs, ts))
end
end
function main()
"""
Main function
"""
# Define functional and terminal sets
fs = [:+, :-, :DIV, :GT]
ts = [:x, :v, -1]
# Create the tree
ast = create_single_full_tree(4, fs, ts)
#println(typeof(ast))
#println(ast)
#println(dump(ast))
x = 1
v = 1
eval(ast) # Error out unless x and v are globals
end
main()
I am generating a random expression based on certain allowed functions and variables. As seen in the code, the expression can only have symbols x and v, as well as the value -1. I will need to test the expression with a variety of x and v values; here I am just using x=1 and v=1 to test the code.
The expression is being returned correctly, however, eval() can only be used with global variables, so it will error out when run unless I declare x and v to be global (ERROR: LoadError: UndefVarError: x not defined). I would like to avoid globals if possible. Is there a better way to generate and evaluate these generated expressions with locally defined variables?
Here is an example for generating an (anonymous) function. The result of eval can be called as a function and your variable can be passed as parameters:
myfun = eval(Expr(:->,:x, Expr(:block, Expr(:call,:*,3,:x) )))
myfun(14)
# returns 42
The dump function is very useful to inspect the expression that the parsers has created. For two input arguments you would use a tuple for example as args[1]:
julia> dump(parse("(x,y) -> 3x + y"))
Expr
head: Symbol ->
args: Array{Any}((2,))
1: Expr
head: Symbol tuple
args: Array{Any}((2,))
1: Symbol x
2: Symbol y
typ: Any
2: Expr
[...]
Does this help?
In the Metaprogramming part of the Julia documentation, there is a sentence under the eval() and effects section which says
Every module has its own eval() function that evaluates expressions in its global scope.
Similarly, the REPL help ?eval will give you, on Julia 0.6.2, the following help:
Evaluate an expression in the given module and return the result. Every Module (except those defined with baremodule) has its own 1-argument definition of eval, which evaluates expressions in that module.
I assume, you are working in the Main module in your example. That's why you need to have the globals defined there. For your problem, you can use macros and interpolate the values of x and y directly inside the macro.
A minimal working example would be:
macro eval_line(a, b, x)
isa(a, Real) || (warn("$a is not a real number."); return :(throw(DomainError())))
isa(b, Real) || (warn("$b is not a real number."); return :(throw(DomainError())))
return :($a * $x + $b) # interpolate the variables
end
Here, #eval_line macro does the following:
Main> #macroexpand #eval_line(5, 6, 2)
:(5 * 2 + 6)
As you can see, the values of macro's arguments are interpolated inside the macro and the expression is given to the user accordingly. When the user does not behave,
Main> #macroexpand #eval_line([1,2,3], 7, 8)
WARNING: [1, 2, 3] is not a real number.
:((Main.throw)((Main.DomainError)()))
a user-friendly warning message is provided to the user at parse-time, and a DomainError is thrown at run-time.
Of course, you can do these things within your functions, again by interpolating the variables --- you do not need to use macros. However, what you would like to achieve in the end is to combine eval with the output of a function that returns Expr. This is what the macro functionality is for. Finally, you would simply call your macros with an # sign preceding the macro name:
Main> #eval_line(5, 6, 2)
16
Main> #eval_line([1,2,3], 7, 8)
WARNING: [1, 2, 3] is not a real number.
ERROR: DomainError:
Stacktrace:
[1] eval(::Module, ::Any) at ./boot.jl:235
EDIT 1. You can take this one step further, and create functions accordingly:
macro define_lines(linedefs)
for (name, a, b) in eval(linedefs)
ex = quote
function $(Symbol(name))(x) # interpolate name
return $a * x + $b # interpolate a and b here
end
end
eval(ex) # evaluate the function definition expression in the module
end
end
Then, you can call this macro to create different line definitions in the form of functions to be called later on:
#define_lines([
("identity_line", 1, 0);
("null_line", 0, 0);
("unit_shift", 0, 1)
])
identity_line(5) # returns 5
null_line(5) # returns 0
unit_shift(5) # returns 1
EDIT 2. You can, I guess, achieve what you would like to achieve by using a macro similar to that below:
macro random_oper(depth, fs, ts)
operations = eval(fs)
oper = operations[rand(1:length(operations))]
terminals = eval(ts)
ts = terminals[rand(1:length(terminals), 2)]
ex = :($oper($ts...))
for d in 2:depth
oper = operations[rand(1:length(operations))]
t = terminals[rand(1:length(terminals))]
ex = :($oper($ex, $t))
end
return ex
end
which will give the following, for instance:
Main> #macroexpand #random_oper(1, [+, -, /], [1,2,3])
:((-)([3, 3]...))
Main> #macroexpand #random_oper(2, [+, -, /], [1,2,3])
:((+)((-)([2, 3]...), 3))
Thanks Arda for the thorough response! This helped, but part of me thinks there may be a better way to do this as it seems too roundabout. Since I am writing a genetic program, I will need to create 500 of these ASTs, all with random functions and terminals from a set of allowed functions and terminals (fs and ts in the code). I will also need to test each function with 20 different values of x and v.
In order to accomplish this with the information you have given, I have come up with the following macro:
macro create_function(defs)
for name in eval(defs)
ex = quote
function $(Symbol(name))(x,v)
fs = [:+, :-, :DIV, :GT]
ts = [x,v,-1]
return create_single_full_tree(4, fs, ts)
end
end
eval(ex)
end
end
I can then supply a list of 500 random function names in my main() function, such as ["func1, func2, func3,.....". Which I can eval with any x and v values in my main function. This has solved my issue, however, this seems to be a very roundabout way of doing this, and may make it difficult to evolve each AST with each iteration.
Right now I have an SML function:
method([1,1,1,1,2,2,2,3,3,3]);
returns:
val it = [[2,2,2],[3,3,3]] : int list list
but I need it to return:
val it = [[1,1,1,1],[2,2,2],[3,3,3]] : int list list
This is my current code:
- fun method2(L: int list) =
= if tl(L) = [] then [hd(L)] else
= if hd(tl(L)) = hd(L) then hd(L)::method(tl(L)) else [hd(L)];
- fun method(L: int list) =
= if tl(L) = [] then [] else
= if hd(tl(L)) = hd(L) then method(tl(L)) else
= method2(tl(L))::method(tl(L));
As you can see it misses the first method2 call. Any ideas on how I can fix this? I am completely stumped.
Your problem is here if hd(tl(L)) = hd(L) then method(tl(L)) else. This is saying if the head of the tail is equal to the head, then continue processing, but don't add it to the result list. this will skip the first contiguous chunk of equal values. I would suggest separating the duties of these functions a bit more. The way to do this is to have method2 strip off the next contiguous chunk of values, and return a pair, where the first element will have the contiguous chunk removed, and the second element will have the remaining list. For example, method2([1, 1, 1, 2, 2, 3, 3]) = ([1, 1, 1], [2, 2, 3, 3]) and method2([2, 2, 3, 3]) = ([2, 2], [3, 3]). Now, you can just keep calling method2 until the second part of the pair is nil.
I'm not quite sure what you are trying to do with your code. I would recommend creating a tail recursive helper function which is passed three arguments:
1) The list of lists you are trying to build up
2) The current list you are building up
3) The list you are processing
In your example, a typical call somewhere in the middle of the computation would look like:
helper([[1,1,1,1]], [2,2],[2,3,3,3])
The recursion would work by looking at the head of the last argument ([2,3,3,3]) as well as the head of the list which is currently being built up ([2,2]) and, since they are the same -- the 2 at the end of the last argument is shunted to the list being built up:
helper([[1,1,1,1]], [2,2,2],[3,3,3])
in the next step in the recursion the heads are then compared and found to be different (2 != 3), so the helper function will put the middle list at the front of the list of lists:
helper([[2,2,2], [1,1,1,1]], [3],[3,3])
the middle list is re-initialized to [3] so it will start growing
eventually you reach something like this:
helper([[2,2,2], [1,1,1,1]], [3,3,3],[])
the [3,3,3] is then tacked onto the list of lists and the reverse of this list is returned.
Once such a helper function is defined, the main method checks for an empty list and, if not empty, initializes the first call to the helper function. The following code fleshes out theses ideas -- using pattern-matching style rather than hd and tl (I am not a big fan of using those functions explicitly -- it makes the code too Lisp-like). If this is homework then you should probably thoroughly understand how it works and then translate it to code involving hd and tl since your professor would regard it as plagiarized if you use things you haven't yet studied and haven't made it your own work:
fun helper (xs, ys, []) = rev (ys::xs)
| helper (xs, y::ys, z::zs) =
if y = z
then helper(xs, z :: y :: ys, zs)
else helper((y::ys)::xs,[z],zs);
fun method [] = []
| method (x::xs) = helper([],[x],xs);
Suppose we have a Vector of tuples (Int64, Int64) in julia:
In [1] xx = [(1, 2), (3, 4), (5, 6)]
typeof(xx) == Vector{(Int64, Int64)}
Out[1] true
Now I want to construct a new vector of the first indices of the tuples.
In [2] indices = [x[1] for x in xx]
typeof(indices)
Out[2] Array{Any, 1}
I expect it to be an Array{Int64, 1} type. How can I fix this?
edit: I am using 0.3.9.
function f()
xx = [(1, 2), (3, 4), (5, 6)]
inds = [ x[1] for x in xx ]
return(inds)
end
y = f()
typeof(y)
The last line of code returns Array{Int64, 1}.
The problem here is that you are working in global scope. For Julia's type inference to be able to do its magic, you need to work in a local scope. In other words, wrap all your code in functions. This rule is very, very, important, but, having come from a MatLab background myself, I can see why people forget it. Just remember, 90% of questions saying "Why is my Julia code slow?" occur because the user was working in global scope, not local scope.
ps, even in local scope, type inference of loop comprehensions can stumble in particularly complex cases. This is a known issue and is being worked on. If you want to provide the compiler with some "help" you can do something like:
inds = Int[ x[1] for x in xx ]
You can also use map and preserve the type:
#passing a lambda that takes the 1st element, and the iterable
inds = map( (x)-> x[1], xx)