Apologies in advance if this is a really silly question.
I have 2 arrays - StartAmount is a 165×11 Matrix{Float64}: that contains 11 "types" of monetary values for 165 lives, and IncreaseFactors is a 100×11 Matrix{Float64}: that contains 11 increase factors for each of 100 years. I want to multiply these by each other such that my result is a 165x11x100 array, with each of the 11 "types" of amount increased for 100 years. When I try StartAmount .* IncreaseFactors I get ERROR: DimensionMismatch("arrays could not be broadcast to a common size; got a dimension with lengths 165 and 100"). I have already set up a 165x11x100 array of zeros, but I don't know how to populate it!
Any suggestions on how to manipulate my two arrays to create the 3D array I am after?
Thanks!
Tullio.jl can abstract away many indexing notations nicely.
using Tullio
#tullio out[i,j,k] := StartAmount[i,j] * IncreaseFactors[k,j]
165×11×100 Array{Float64, 3}:
[:, :, 1] =
...
It essentially does what the following 3-loop would do but without caring about indices:
out = [StartAmount[i,j] * IncreaseFactors[k,j] for i=1:165, j=1:11, k=1:100]
Related
I picked up Julia to do some numerical analysis stuff and was trying to implement a full pivot LU decomposition (as in, trying to get an LU decomposition that is as stable as possible). I thought that the best way of doing so was finding the maximum value for each column and then resorting the columns in descending order of their maximum values.
Is there a way of avoiding swapping every element of two columns and instead doing something like changing two references/pointers?
Following up on #longemen3000's answer, you can use views to swap columns. For example:
julia> A = reshape(1:12, 3, 4)
3×4 reshape(::UnitRange{Int64}, 3, 4) with eltype Int64:
1 4 7 10
2 5 8 11
3 6 9 12
julia> V = view(A, :, [3,2,4,1])
3×4 view(reshape(::UnitRange{Int64}, 3, 4), :, [3, 2, 4, 1]) with eltype Int64:
7 4 10 1
8 5 11 2
9 6 12 3
That said, whether this is a good strategy depends on access patterns. If you'll use elements of V once or a few times, this view strategy is a good one. In contrast, if you access elements of V many times, you may be better off making a copy or moving values in-place, since that's a price you pay once whereas here you pay an indirection cost every time you access a value.
Just for "completeness", in case you actually want to swap columns in-place,
function swapcols!(X::AbstractMatrix, i::Integer, j::Integer)
#inbounds for k = 1:size(X,1)
X[k,i], X[k,j] = X[k,j], X[k,i]
end
end
is simple and fast.
In fact, in an individual benchmark for small matrices this is even faster than the view approach mentioned in the other answers (views aren't always free):
julia> A = rand(1:10,4,4);
julia> #btime view($A, :, $([3,2,1,4]));
31.919 ns (3 allocations: 112 bytes)
julia> #btime swapcols!($A, 1,3);
8.107 ns (0 allocations: 0 bytes)
in julia there is the #view macro, that allows you to create an array that is just a reference to another array, for example:
A = [1 2;3 4]
Aview = #view A[:,1] #view of the first column
Aview[1,1] = 10
julia> A
2×2 Array{Int64,2}:
10 2
3 4
with that said, when working with concrete number types (Float64,Int64,etc), julia uses contiguous blocks of memory with the direct representation of the number type. that is, a julia array of numbers is not an array of pointers were each element of an array is a pointer to a value. if the values of an array can be represented by a concrete binary representation (an array of structs, for example) then an array of pointers is used.
I'm not a computer science expert, but i observed that is better to have your data tightly packed that using a lot of pointers when doing number crunching.
Another different case is Sparse Arrays. the basic julia representation of an sparse array is an array of indices and an array of values. here you can simply swap the indices instead of copying the values
I am looking for help with pseudo code (unless you are a user of Game Maker 8.0 by Mark Overmars and know the GML equivalent of what I need) for how to generate a list / array of unique combinations of a set of X number of integers which size is variable. It can be 1-5 or 1-1000.
For example:
IntegerList{1,2,3,4}
1,2
1,3
1,4
2,3
2,4
3,4
I feel like the math behind this is simple I just cant seem to wrap my head around it after checking multiple sources on how to do it in languages such as C++ and Java. Thanks everyone.
As there are not many details in the question, I assume:
Your input is a natural number n and the resulting array contains all natural numbers from 1 to n.
The expected output given by the combinations above, resembles a symmetric relation, i. e. in your case [1, 2] is considered the same as [2, 1].
Combinations [x, x] are excluded.
There are only combinations with 2 elements.
There is no List<> datatype or dynamic array, so the array length has to be known before creating the array.
The number of elements in your result is therefore the binomial coefficient m = n over 2 = n! / (2! * (n - 2)!) (which is 4! / (2! * (4 - 2)!) = 24 / 4 = 6 in your example) with ! being the factorial.
First, initializing the array with the first n natural numbers should be quite easy using the array element index. However, the index is a property of the array elements, so you don't need to initialize them in the first place.
You need 2 nested loops processing the array. The outer loop ranges i from 1 to n - 1, the inner loop ranges j from 2 to n. If your indexes start from 0 instead of 1, you have to take this into consideration for the loop limits. Now, you only need to fill your target array with the combinations [i, j]. To find the correct index in your target array, you should use a third counter variable, initialized with the first index and incremented at the end of the inner loop.
I agree, the math behind is not that hard and I think this explanation should suffice to develop the corresponding code yourself.
I have an array of size 330x534x223 containing a distribution of values, like a 3d image. It's size is prohibitively large for my purposes, so I'm looking to resample it by a factor of 20 in each dimension.
Is there any conceivable way to do this? I've tried checking the docs
Thank you
If you're thinking in terms of "thumbnail images," then just taking every 20th element may not be satisfying. For something that will simultaneously smooth and subsample with very good performance, I recommend restrict from JuliaImages (which cuts by a factor of 2, and you can call it repeatedly).
How about this?
julia> subsample(a, n) = getindex(a, (indices(a,i)[1:n:end] for i=1:ndims(a))...)
subsample (generic function with 1 method)
julia> a = reshape(1:10^6, (100,100,100));
julia> subsample(a, 50)
2×2×2 Array{Int64,3}:
[:, :, 1] =
1 5001
51 5051
[:, :, 2] =
500001 505001
500051 505051
This should also work for arrays with unconventional indexing like OffsetArrays.
For large arrays, the overhead of this implementation compared to the direct indexing with ranges is negligible.
Edit:
A type stable version:
subinds(n, inds) = (first(inds)[1:n:end], subinds(n, Base.tail(inds))...)
subinds(n, ::Tuple{}) = ()
subsample(a, n) = getindex(a, subinds(n, indices(a))...)
If I do
mat = rand(8,8)
sum(mat, 1)
the return type is a Matrix with a single row, whereas sum(mat, 2) gives a Matrix with a single column. This surprises me, as singleton dimensions are generally dropped in 0.5, so I would expect the return type of both operations would be a Vector. Why is the singleton dimension not dropped here?
I might expect this was in order to preserve the orientation (e.g. sum(mat, 1) is a row Vector), but the behaviour is the same on 0.6, which has explicit 1-d RowVectors, so this does not appear to be an explanation.
Thanks!
Yes, reductions like sum preserve the dimensionality of the array. This is intentional as it enables broadcasting the result back across the original array. This means that you can, for example, normalize the columns of an array with ./:
julia> A = rand(1:100, 4, 3)
4×3 Array{Int64,2}:
94 50 32
46 15 78
34 29 41
79 22 58
julia> A ./ sum(A, 1)
4×3 Array{Float64,2}:
0.371542 0.431034 0.15311
0.181818 0.12931 0.373206
0.134387 0.25 0.196172
0.312253 0.189655 0.277512
While the two-dimensional case might be able to be handled by RowVectors, that approach does not generalize to higher dimensions.
That said, there are other cases where dropping dimensions would be similarly useful. This is an open design question on the issue tracker.
I was delighted to learn that Julia allows a beautifully succinct way to form inner products:
julia> x = [1;0]; y = [0;1];
julia> x'y
1-element Array{Int64,1}:
0
This alternative to dot(x,y) is nice, but it can lead to surprises:
julia> #printf "Inner product = %f\n" x'y
Inner product = ERROR: type: non-boolean (Array{Bool,1}) used in boolean context
julia> #printf "Inner product = %f\n" dot(x,y)
Inner product = 0.000000
So while i'd like to write x'y, it seems best to avoid it, since otherwise I need to be conscious of pitfalls related to scalars versus 1-by-1 matrices.
But I'm new to Julia, and probably I'm not thinking in the right way. Do others use this succinct alternative to dot, and if so, when is it safe to do so?
There is a conceptual problem here. When you do
julia> x = [1;0]; y = [0;1];
julia> x'y
0
That is actually turned into a matrix * vector product with dimensions of 2x1 and 1 respectively, resulting in a 1x1 matrix. Other languages, such as MATLAB, don't distinguish between a 1x1 matrix and a scalar quantity, but Julia does for a variety of reasons. It is thus never safe to use it as alternative to the "true" inner product function dot, which is defined to return a scalar output.
Now, if you aren't a fan of the dots, you can consider sum(x.*y) of sum(x'y). Also keep in mind that column and row vectors are different: in fact, there is no such thing as a row vector in Julia, more that there is a 1xN matrix. So you get things like
julia> x = [ 1 2 3 ]
1x3 Array{Int64,2}:
1 2 3
julia> y = [ 3 2 1]
1x3 Array{Int64,2}:
3 2 1
julia> dot(x,y)
ERROR: `dot` has no method matching dot(::Array{Int64,2}, ::Array{Int64,2})
You might have used a 2d row vector where a 1d column vector was required.
Note the difference between 1d column vector [1,2,3] and 2d row vector [1 2 3].
You can convert to a column vector with the vec() function.
The error message suggestion is dot(vec(x),vec(y), but sum(x.*y) also works in this case and is shorter.
julia> sum(x.*y)
10
julia> dot(vec(x),vec(y))
10
Now, you can write x⋅y instead of dot(x,y).
To write the ⋅ symbol, type \cdot followed by the TAB key.
If the first argument is complex, it is conjugated.
Now, dot() and ⋅ also work for matrices.
Since version 1.0, you need
using LinearAlgebra
before you use the dot product function or operator.