Tensor mode n product in Julia - julia

I need tensor mode n product.
The defination of tenosr mode n product can be seen here.
https://www.alexejgossmann.com/tensor_decomposition_tucker/
I found python code.
I would like to convert this code into julia.
def mode_n_product(x, m, mode):
x = np.asarray(x)
m = np.asarray(m)
if mode <= 0 or mode % 1 != 0:
raise ValueError('`mode` must be a positive interger')
if x.ndim < mode:
raise ValueError('Invalid shape of X for mode = {}: {}'.format(mode, x.shape))
if m.ndim != 2:
raise ValueError('Invalid shape of M: {}'.format(m.shape))
return np.swapaxes(np.swapaxes(x, mode - 1, -1).dot(m.T), mode - 1, -1)
I have found another answer using Tensortoolbox.jl
using TensorToolbox
X=rand(5,4,3);
A=rand(2,5);
ttm(X,A,n) #X times A[1] by mode n

One way is:
using TensorOperations
#tensor y[i1, i2, i3, out, i5] := x[i1, i2, i3, s, i5] * a[out, s]
This is literally the formula given at your link to define this, except that I changed the name of the summed index to s; you can you any index names you like, they are just markers. The sum is implicit, because s does not appear on the left.
There is nothing very special about putting the index out back in the same place. Like your python code, #tensor permutes the dimensions of x in order to use ordinary matrix multiplication, and then permutes again to give y the requested order. The fewer permutations needed, the faster this will be.
Alternatively, you can try using LoopVectorization, Tullio; #tullio y[i1, i2, ... with the same notation. Instead of permuting in order to call a library matrix multiplication function, this writes a pure-Julia version which works with the array as it arrives.

Related

Cumulative Integration Options With Julia

I have two 1-D arrays in which I would like to calculate the approximate cumulative integral of 1 array with respect to the scalar spacing specified by the 2nd array. MATLAB has a function called cumtrapz that handles this scenario. Is there something similar that I can try within Julia to accomplish the same thing?
The expected result is another 1-D array with the integral calculated for each element.
There is a numerical integration package for Julia (see the link) that defines cumul_integrate(X, Y) and uses the trapezoidal rule by default.
If this package didn't exist, though, you could easily write the function yourself and have a very efficient implementation out of the box because the loop does not come with a performance penalty.
Edit: Added an #assert to check matching vector dimensions and fixed a typo.
function cumtrapz(X::T, Y::T) where {T <: AbstractVector}
# Check matching vector length
#assert length(X) == length(Y)
# Initialize Output
out = similar(X)
out[1] = 0
# Iterate over arrays
for i in 2:length(X)
out[i] = out[i-1] + 0.5*(X[i] - X[i-1])*(Y[i] + Y[i-1])
end
# Return output
out
end

How to inherit from a regular matrix

using ShiftedArrays
struct CircularMatrix{T} <: AbstractArray{T,2}
data::Array{T,2}
view::CircShiftedArray
currentIndex::Int
function CircularMatrix{T}(dims...) where T
data = zeros(T, dims...)
CircularMatrix(data, ShiftedArrays.circshift(data, (0, -1)), 1)
end
end
Base.size(M::CircularMatrix) = size(M.data)
Base.eltype(::Type{CircularMatrix{T}}) where {T} = T
function shift_forward!(M::CircularMatrix)
M.shift_forward!(1)
end
function shift_forward!(M::CircularMatrix, n)
# replace the view with a view shifted forwards.
M.currentIndex += n
M.view = ShiftedArrays.circshift(M.data, (n, M.currentIndex))
end
#inline Base.#propagate_inbounds function Base.getindex(M::CircularMatrix, i) = M.view[i]
#inline Base.#propagate_inbounds function Base.setindex!(M::CircularMatrix, data, i) = M.view[i] = data
How can I make CircularMatrix act just like a regular matrix.
So that I can access it like
m = CircularMatrix{Int}(4,4)
m[1, 1] = 5
x = view(m, 1, :)
Your matrix type is defined to be a subtype of AbstractArray{T, 2}. You need to implement a few methods in the informal array interface of Julia for your type to make functions and features that work on AbstractArray{T, 2} to also work on your custom type, that is, to make your CircularMatrix an iterable, indexable, completely functioning matrix.
The methods to implement are
size(M::CircularMatrix)
getindex(M::CircularMatrix, i::Int)
getindex(M::CircularMatrix, I::Vararg{Int, N})
setindex!(M::CircularMatrix, v, i::Int)
setindex!(M::CircularMatrix, v, I::Vararg{Int, N})
You already implement 1, 2 and 4 but have not yet set your indexing style. You might not need 3 and 5 if you choose linear indexing style. You only need to set IndexStyle to be IndexLinear() and maybe a few modifications, then everything should just work for your matrix.
1. size(M::CircularMatrix)
The first one is size. size(A::CircularMatrix) returns a Tuple of dimensions of A. I believe for your matrix probably something like the following
Base.size(M::CircularMatrix) = size(M.data)
2. getindex(M::CircularMatrix, i::Int)
This method is needed if you choose linear indexing style. getindex(M, i::Int) should give you the value at linear index i. You already implement it in your code. If you choose linear indexing, you need to set IndexStyle for your type and then you simply skip 3 and 5. Julia will automatically convert multiple index accesses, e.g. a[3, 5], to a linear index access.
Base.IndexStyle(::Type{<:CircularMatrix}) = IndexLinear()
Base.#propogate_inbounds function Base.getindex(M::CircularMatrix, i::Int)
#boundscheck checkbounds(M, i)
#inbounds M.view[i]
end
It might be better to use #inbounds here on the second line. If the caller doesn't use #inbounds, we check the bounds first and this hopefully makes the subsequent bounds check unnecessary. You might want to omit this during development, though.
3. getindex(M::CircularMatrix, I::Vararg{Int, N})
The third one is for Cartesian indexing style. If you choose this style you need to implement this method. Vararg{Int, N} in the signature stands for "exactly N Int arguments". Here N should be equal to the dimensionality of CircularMatrix. Since this is a matrix, N should be two. If you choose this style, you need to define something like the following
Base.#propogate_inbounds function Base.getindex(A::CircularMatrix, I::Vararg{Int, 2})
#boundscheck checkbounds(A, I...)
#inbounds A.view[# convert I[1]` and `I[2]` to a linear index in `view`]
end
or since your dimensionality is not parametric and a matrix is 2D, simply
Base.#propogate_inbounds function Base.getindex(A::CircularMatrix, i::Int, j::Int)
#boundscheck checkbounds(A, i, j)
#inbounds A.view[# convert i` and `j` to a linear index in `view`]
end
4. setindex!(M::CircularMatrix, v, i::Int)
The fourth one is similar to the second. This method should set the value at linear index i, if you choose linear indexing style.
5. setindex!(M::CircularMatrix, v, I::Vararg{Int, N})
The fifth one should be similar to the third, if you choose Cartesian indexing style.
After the implementations for 1, 2, and 4 and setting IndexStyle, you should have a custom matrix type that just works.
m[1, 1] = 5
x = view(m, 1, :)
for e in
...
end
for i in eachindex(m)
...
end
display(m)
println(m)
length(m)
ndims(m)
map(f, A)
....
These should all work.
A few notes
There is a documentation for Abstract Arrays interface here with a few examples. You can also see Optional Methods to implement.
There is a JuliaArray organization on GitHub that provides lots of useful custom array implementations including StaticArrays, OffsetArrays, etc. and also a JuliaMatrices organization that provides custom matrix types. You might want to take a look at their implementations.
#inline is redundant if you use Base.#propogate_inbounds.
#propagate_inbounds
Tells the compiler to inline a function while retaining the caller's
inbounds context.
You do not need to define eltype for your matrix, since there is already a definition for AbstractArray{T, N} which returns T.

Slicing and broadcasting multidimensional arrays in Julia : meshgrid example

I recently started learning Julia by coding a simple implementation of Self Organizing Maps. I want the size and dimensions of the map to be specified by the user, which means I can't really use for loops to work on the map arrays because I don't know in advance how many layers of loops I will need. So I absolutely need broadcasting and slicing functions that work on arrays of arbitrary dimensions.
Right now, I need to construct an array of indices of the map. Say my map is defined by an array of size mapsize = (5, 10, 15), I need to construct an array indices of size (3, 5, 10, 15) where indices[:, a, b, c] should return [a, b, c].
I come from a Python/NumPy background, in which the solution is already given by a specific "function", mgrid :
indices = numpy.mgrid[:5, :10, :15]
print indices.shape # gives (3, 5, 10, 15)
print indices[:, 1, 2, 3] gives [1, 2, 3]
I didn't expect Julia to have such a function on the get-go, so I turned to broadcasting. In NumPy, broadcasting is based on a set of rules that I find quite clear and logical. You can use arrays of different dimensions in the same expression as long as the sizes in each dimension match or one of it is 1 :
(5, 10, 15) broadcasts to (5, 10, 15)
(10, 1)
(5, 1, 15) also broadcasts to (5, 10, 15)
(1, 10, 1)
To help with this, you can also use numpy.newaxis or None to easily add new dimensions to your array :
array = numpy.zeros((5, 15))
array[:,None,:] has shape (5, 1, 15)
This helps broadcast arrays easily :
A = numpy.arange(5)
B = numpy.arange(10)
C = numpy.arange(15)
bA, bB, bC = numpy.broadcast_arrays(A[:,None,None], B[None,:,None], C[None,None,:])
bA.shape == bB.shape == bC.shape = (5, 10, 15)
Using this, creating the indices array is rather straightforward :
indices = numpy.array(numpy.broadcast_arrays(A[:,None,None], B[None,:,None], C[None,None,:]))
(indices == numpy.mgrid[:5,:10,:15]).all() returns True
The general case is of course a bit more complicated, but can be worked around using list comprehension and slices :
arrays = [ numpy.arange(i)[tuple([None if m!=n else slice(None) for m in range(len(mapsize))])] for n, i in enumerate(mapsize) ]
indices = numpy.array(numpy.broadcast_arrays(*arrays))
So back to Julia. I tried to apply the same kind of rationale and ended up achieving the equivalent of the arrays list of the code above. This ended up being rather simpler than the NumPy counterpart thanks to the compound expression syntax :
arrays = [ (idx = ones(Int, length(mapsize)); idx[n] = i;reshape([1:i], tuple(idx...))) for (n,i)=enumerate(mapsize) ]
Now I'm stuck here, as I don't really know how to apply the broadcasting to my list of generating arrays here... The broadcast[!] functions ask for a function f to apply, and I don't have any. I tried using a for loop to try forcing the broadcasting:
indices = Array(Int, tuple(unshift!([i for i=mapsize], length(mapsize))...))
for i=1:length(mapsize)
A[i] = arrays[i]
end
But this gives me an error : ERROR: convert has no method matching convert(::Type{Int64}, ::Array{Int64,3})
Am I doing this the right way? Did I overlook something important? Any help is appreciated.
If you're running julia 0.4, you can do this:
julia> function mgrid(mapsize)
T = typeof(CartesianIndex(mapsize))
indices = Array(T, mapsize)
for I in eachindex(indices)
indices[I] = I
end
indices
end
It would be even nicer if one could just say
indices = [I for I in CartesianRange(CartesianIndex(mapsize))]
I'll look into that :-).
Broadcasting in Julia has been modelled pretty much on broadcasting in NumPy, so you should hopefully find that it obeys more or less the same simple rules (not sure if the way to pad dimensions when not all inputs have the same number of dimensions is the same though, since Julia arrays are column-major).
A number of useful things like newaxis indexing and broadcast_arrays have not been implemented (yet) however. (I hope they will.) Also note that indexing works a bit differently in Julia compared to NumPy: when you leave off indices for trailing dimensions in NumPy, the remaining indices default to colons. In Julia they could be said to default to ones instead.
I'm not sure if you actually need a meshgrid function, most things that you would want to use it for could be done by using the original entries of your arrays array with broadcasting operations. The major reason that meshgrid is useful in matlab is because it is terrible at broadcasting.
But it is quite straightforward to accomplish what you want to do using the broadcast! function:
# assume mapsize is a vector with the desired shape, e.g. mapsize = [2,3,4]
N = length(mapsize)
# Your line to create arrays below, with an extra initial dimension on each array
arrays = [ (idx = ones(Int, N+1); idx[n+1] = i;reshape([1:i], tuple(idx...))) for (n,i) in enumerate(mapsize) ]
# Create indices and fill it one coordinate at a time
indices = zeros(Int, tuple(N, mapsize...))
for (i,arr) in enumerate(arrays)
dest = sub(indices, i, [Colon() for j=1:N]...)
broadcast!(identity, dest, arr)
end
I had to add an initial singleton dimension on the entries of arrays to line up with the axes of indices (newaxis had been useful here...).
Then I go through each coordinate, create a subarray (a view) on the relevant part of indices, and fill it. (Indexing will default to returning subarrays in Julia 0.4, but for now we have to use sub explicitly).
The call to broadcast! just evaluates the identity function identity(x)=x on the input arr=arrays[i], broadcasts to the shape of the output. There's no efficiency lost in using the identity function for this; broadcast! generates a specialized function based on the given function, number of arguments, and number of dimensions of the result.
I guess this is the same as the MATLAB meshgrid functionality. I've never really thought about the generalization to more than two dimensions, so its a bit harder to get my head around.
First, here is my completely general version, which is kinda crazy but I can't think of a better way to do it without generating code for common dimensions (e.g. 2, 3)
function numpy_mgridN(dims...)
X = Any[zeros(Int,dims...) for d in 1:length(dims)]
for d in 1:length(dims)
base_idx = Any[1:nd for nd in dims]
for i in 1:dims[d]
cur_idx = copy(base_idx)
cur_idx[d] = i
X[d][cur_idx...] = i
end
end
#show X
end
X = numpy_mgridN(3,4,5)
#show X[1][1,2,3] # 1
#show X[2][1,2,3] # 2
#show X[3][1,2,3] # 3
Now, what I mean by code generation is that, for the 2D case, you can simply do
function numpy_mgrid(dim1,dim2)
X = [i for i in 1:dim1, j in 1:dim2]
Y = [j for i in 1:dim1, j in 1:dim2]
return X,Y
end
and for the 3D case:
function numpy_mgrid(dim1,dim2,dim3)
X = [i for i in 1:dim1, j in 1:dim2, k in 1:dim3]
Y = [j for i in 1:dim1, j in 1:dim2, k in 1:dim3]
Z = [k for i in 1:dim1, j in 1:dim2, k in 1:dim3]
return X,Y,Z
end
Test with, e.g.
X,Y,Z=numpy_mgrid(3,4,5)
#show X
#show Y
#show Z
I guess mgrid shoves them all into one tensor, so you could do that like this
all = cat(4,X,Y,Z)
which is still slightly different:
julia> all[1,2,3,:]
1x1x1x3 Array{Int64,4}:
[:, :, 1, 1] =
1
[:, :, 1, 2] =
2
[:, :, 1, 3] =
3
julia> vec(all[1,2,3,:])
3-element Array{Int64,1}:
1
2
3

posmax: like argmax but gives the position(s) of the element x for which f[x] is maximal

Mathematica has a built-in function ArgMax for functions over infinite domains, based on the standard mathematical definition.
The analog for finite domains is a handy utility function.
Given a function and a list (call it the domain of the function), return the element(s) of the list that maximize the function.
Here's an example of finite argmax in action:
Canonicalize NFL team names
And here's my implementation of it (along with argmin for good measure):
(* argmax[f, domain] returns the element of domain for which f of
that element is maximal -- breaks ties in favor of first occurrence. *)
SetAttributes[{argmax, argmin}, HoldFirst];
argmax[f_, dom_List] := Fold[If[f[#1]>=f[#2], #1, #2]&, First[dom], Rest[dom]]
argmin[f_, dom_List] := argmax[-f[#]&, dom]
First, is that the most efficient way to implement argmax?
What if you want the list of all maximal elements instead of just the first one?
Second, how about the related function posmax that, instead of returning the maximal element(s), returns the position(s) of the maximal elements?
#dreeves, you're correct in that Ordering is the key to the fastest implementation of ArgMax over a finite domain:
ArgMax[f_, dom_List] := dom[[Ordering[f /# dom, -1]]]
Part of the problem with your original implementation using Fold is that you end up evaluating f twice as much as necessary, which is inefficient, especially when computing f is slow. Here we only evaluate f once for each member of the domain. When the domain has many duplicated elements, we can further optimize by memoizing the values of f:
ArgMax[f_, dom_List] :=
Module[{g},
g[e___] := g[e] = f[e]; (* memoize *)
dom[[Ordering[g /# dom, -1]]]
]
This was about 30% faster in some basic tests for a list of 100,000 random integers between 0 and 100.
For a posmax function, this somewhat non-elegant approach is the fastest thing I can come up with:
PosMax[f_, dom_List] :=
Module[{y = f/#dom},
Flatten#Position[y, Max[y]]
]
Of course, we can apply memoization again:
PosMax[f_, dom_List] :=
Module[{g, y},
g[e___] := g[e] = f[e];
y = g /# dom;
Flatten#Position[y, Max[y]]
]
To get all the maximal elements, you could now just implement ArgMax in terms of PosMax:
ArgMax[f_, dom_List] := dom[[PosMax[f, dom]]]
For posmax, you can first map the function over the list and then just ask for the position of the maximal element(s). Ie:
posmax[f_, dom_List] := posmax[f /# dom]
where posmax[list] is polymorphically defined to just return the position of the maximal element(s).
It turns out there's a built-in function, Ordering that essentially does this.
So we can define the single-argument version of posmax like this:
posmax[dom_List] := Ordering[dom, -1][[1]]
I just tested that against a loop-based version and a recursive version and Ordering is many times faster.
The recursive version is pretty so I'll show it off here, but don't ever try to run it on large inputs!
(* posmax0 is a helper function for posmax that returns a pair with the position
and value of the max element. n is an accumulator variable, in lisp-speak. *)
posmax0[{h_}, n_:0] := {n+1, h}
posmax0[{h_, t___}, n_:0] := With[{best = posmax0[{t}, n+1]},
If[h >= best[[2]], {n+1, h}, best]]
posmax[dom_List] := First#posmax0[dom, 0]
posmax[f_, dom_List] := First#posmax0[f /# dom, 0]
posmax[_, {}] := 0
None of this addresses the question of how to find all the maximal elements (or positions of them).
That doesn't normally come up for me in practice, though I think it would be good to have.

dynamic programming pseudocode for Travelling Salesman

this is a dynamic programming pseudocode for TSP (Travelling Salesman Problem). i understood its optimal substructure but i can't figure out what the code in red brackets do.
i am not asking anyone to write the actual code, i just need explanation on what is happening so i can write my own.... thanks:)
here is a link for the pseudocode, i couln't uploaded over here.
http://www.imagechicken.com/viewpic.php?p=1266328410025325200&x=jpg
Here is some less mathematical pseudo-code. I don't know if this will explain what's happening, but it may help you read it. This isn't a functional algorithm (lots of := all over), so I'm going to use Python pseudo-code.
# I have no idea where 'i' comes from. It's not defined anywhere
for k in range(2,n):
C[set(i,k), k] = d(1,k)
shortest_path = VERY_LARGE_NUMBER
# I have to assume that n is the number of nodes in the graph G
# other things that are not defined:
# d_i,j -- I will assume it's the distance from i to j in G
for subset_size in range(3,n):
for index_subset in subsets_of_size(subset_size, range(1,n)):
for k in index_subset:
C[S,k] = argmin(lambda m: C[S-k,m] + d(G,m,k), S - k)
shortest_path = argmin(lambda k: C[set(range(1,n)),k] + d(G,1,k), range(2,n))
return shortest_path
# also needed....
def d(G, i, j):
return G[i][j]
def subsets_of_size(n, s): # returns a list of sets
# complicated code goes here
pass
def argmin(f, l):
best = l[0]
bestVal = f(best)
for x in l[1:]:
newVal = f(x)
if newVal < bestVal:
best = x
bestVal = newVal
return best
Some notes:
The source algorithm is not complete. At least, its formatting is weird in the inner loop, and it rebinds k in the second argmin. So the whole thing is probably wrong; I've not tried to run this code.
arguments to range should probably all be increased by 1 since Python counts from 0, not 1. (and in general counting from 1 is a bad idea).
I assume that G is a dictionary of type { from : { to : length } }. In other words, an adjacency list representation.
I inferred that C is a dictionary of type { (set(int),int) : int }. I could be wrong.
I use a set as keys to C. In real Python, you must convert to a frozen_set first. The conversion is just busywork so I left it out.
I can't remember the set operators in Python. I seem to remember it uses | and & instead of + and -.
I didn't write subsets_of_size. It's fairly complicated.

Resources