I am looking for a way to calculate outer products among more than three vectors in Julia.
Let a, b, c and d are vectors that each size is I, J, K and L, respectively. Then, their outer products of them are defined as a tensor T whose size is I x J x K x L, and each element is defined as
T[i,j,k,l] = a[i]*b[j]*c[k]*d[l]
Is there any helpful function in Julia to get T from vectors?
What you're looking for here is kron. Taking the Kronecker product of vectors will give vectors, so you can get the desired tensor by reshaping, so in your case
reshape(kron(d,c,b,a),(I,J,K,L))
or more generally
reshape(kron(d,c,b,c),length.((a,b,c,d))). I usually define a function to remember the ordering for me
outer(v...) = reshape(kron(reverse(v)...),length.(v))
So this is how you take the outer product of vectors, of course you could ask whether you can do similar for tensors more generally. Here is a little more tedious to deal with the sizes, but yea you can do the same trick
outer(v...) = reshape(kron(reverse(vec.(v))...),tuple(vcat(collect.(size.(v))...)...))
You can just broadcast * after reshaping:
julia> a,b,c,d = (rand(1+i) for i in 1:4);
julia> t4 = a .* permutedims(b) .* reshape(c,1,1,:) .* reshape(d,1,1,1,:);
julia> summary(t4)
"2×3×4×5 Array{Float64, 4}"
My package translates the notation in the question to precisely this broadcast:
julia> using TensorCast
julia> #cast t4c[i,j,k,l] := a[i] * b[j] * c[k] * d[l];
julia> t4c == t4
true
This operation is similar to kron, but somewhat confusingly the order of its arguments isn't what you'd expect for column-major arrays.
julia> kron(b,a) ≈ vec(a .* b') ≈ #cast t2[(i,j)] := a[i] * b[j]
true
julia> kron(d,c,b,a) ≈ #cast _[(i,j,k,l)] := a[i] * b[j] * c[k] * d[l]
true
Related
I'm attempting to translate the equivalent of the following Python code (from SMT GEKPLS) into Julia:
def differences(X, Y):
D = X[:, np.newaxis, :] - Y[np.newaxis, :, :]
return D.reshape((-1, X.shape[1]))
So, given an input like this:
X = np.array([[1.0,1.0,1.0], [2.0,2.0,2.0]])
Y = np.array([[1.0,2.0,3.0], [4.0,5.0,6.0], [7.0,8.0,9.0]])
diff = differences(X,Y)
We get an output (diff) that looks like this:
[[ 0. -1. -2.]
[-3. -4. -5.]
[-6. -7. -8.]
[ 1. 0. -1.]
[-2. -3. -4.]
[-5. -6. -7.]]
What is an efficient way to do this with Julia code? I expect the X and Y input matrices to be quite large.
After some thinking, I came to this function:
function differences(X, Y)
Rx = repeat(X, inner=(size(Y, 1), 1))
Ry = repeat(Y, size(X, 1))
Rx - Ry
end
I hope I was helpful.
Here's a version that avoids repeat, which creates unnecessary data duplication:
function diffs_row(X, Y)
N = size(X, 2)
return reshape(reshape(X', 1, N, :) .- Y', N, :)'
end
The reason for all the adjoints ' is that it isn't really natural to operate row-wise in Julia. Julia arrays are column-major so reshape will retrieve data column-wise. If you decide instead to change the orientation of the data, you could write
function diffs_col(X, Y)
N = size(X, 1)
return reshape(reshape(X, N, 1, :) .- Y, N, :)
end
instead.
One often sees this when translating numpy code to Julia. Numpy is natively row-major, so the translation becomes a bit awkward. You should consider changing your data layout to be column major in many cases.
This might be faster than other alternatives, while still being easy to understand.
[x .- y for x ∈ X for y ∈ Y]
6-element Vector{Vector{Float64}}:
[0.0, -1.0, -2.0]
[-3.0, -4.0, -5.0]
[-6.0, -7.0, -8.0]
[1.0, 0.0, -1.0]
[-2.0, -3.0, -4.0]
[-5.0, -6.0, -7.0]
The one thing I disliked about numpy is that one has to exactly remember each function in conjunction with a combination of input parameters. In Julia, the traditional loop can serve as an efficient drop-in replacement for most algorithms.
Addendum: The above might be the fastest solution as I said, provided that working with a Vector{Vector{Float64}} is not an issue. If it is, here is another solution that outputs a Matrix{Float64} while being fast as well.
function diffr(X,Y)
i, l, m, n = 0, length(first(X)), length(X), length(Y)
Z = Matrix{Float64}(undef, m*n, l)
for x in X, y in Y
Z[i+=1,:] .= x .- y
end
Z
end
And here is a performance comparison of all posted solutions on my computer.
#btime [x.-y for x∈$X for y∈$Y] # 312.245 ns (9 allocations: 656 bytes)
#btime diffr($X, $Y) # 73.868 ns (1 allocation: 208 bytes)
#btime differences($X, $Y) # 439.000 ns (12 allocations: 896 bytes)
#btime diffs_row($X, $Y) # 463.131 ns (11 allocations: 784 bytes)
Right now I use the package FFTW in order to get some Fourier Transforms I am interested in. However, I'm wondering if there is already a package of FFT that can do the transformation in a vector space which is of the form kron(C2, Rn), where C2 means a 2x2 system and Rn represents the "spatial" subspace in which one is interested in getting the Fourier Transform. In other words, does it exist a routine that implements:
kron(Id2x2, FFT)[kron(C2, Rn)] = kron(C2, FFT(Rn))
Of course the real problem I am interested is in the "two particle case" where the vector space (Hilbert space) is kron(kron(C2, Rn),kron(C2, Rn)), so in this case the routine would need an operator like kron(kron(Id2x2, FFT), kron(Id2x2, FFT)).
Note 1: I haven't tried to do the problem taking partial traces, but in my case this option simply may not work because the states are sparse, i.e. it might be ineficient.
Note 2: Note that (unless I'm mistaken) for kron(C2, Rn) one could do "twice" the fft (one in each sector of C2). However this might also be ineficient for large vector spaces.
Here's an example of what I think you are asking. res is computed by FFT from mat = kron(C2, Rn), and this is (as you say) a wasteful way of doing kron(C2, fft(Rn)) since it the FFT along the k dimension is re-done for each of the 2×2 other dimensions. But the point, presumably, is to do this for "entangled" states in the product space -- a generic likemat = rand(8,2) cannot be decomposed into factors kron(likeC2, likeRn).
(If instead you are really only interested in "un-entangled" product states, then you should probably just work with their components. Combining with kron will then always be wasteful. The package Kronecker.jl may help for some things, but I don't think it knows about fft.)
This uses my package to handle kron-like operations; you could just write out the necessary reshapes yourself, too.
julia> C2 = [1 2; 3 4]; Rn = [1,10,0,0];
julia> mat = kron(C2,Rn)
8×2 Matrix{Int64}:
1 2
10 20
0 0
0 0
3 4
30 40
0 0
0 0
julia> using TensorCast, FFTW
# notation: kron is a reshape of a tensor product, to combine i & k
julia> kron(C2,Rn) == #cast out[(k,i),j] := C2[i,j] * Rn[k]
true
# reshape mat to put the index from Rn in its own dimension:
julia> #cast tri[k,i,j] := mat[(k,i),j] (i in 1:2);
julia> summary(tri)
"4×2×2 Array{Int64, 3}"
# then fft(tri, 1) is the FFT along only that, reshape back:
julia> #cast res[(ktil,i),j] := fft(tri, 1)[ktil,i,j]
8×2 Matrix{ComplexF64}:
11.0+0.0im 22.0+0.0im
1.0-10.0im 2.0-20.0im
-9.0+0.0im -18.0+0.0im
1.0+10.0im 2.0+20.0im
33.0+0.0im 44.0+0.0im
3.0-30.0im 4.0-40.0im
-27.0+0.0im -36.0+0.0im
3.0+30.0im 4.0+40.0im
julia> res ≈ kron(C2, fft(Rn))
true
julia> res ≈ fft(mat, 1)
false
julia> fft(Rn)
4-element Vector{ComplexF64}:
11.0 + 0.0im
1.0 - 10.0im
-9.0 + 0.0im
1.0 + 10.0im
# if fft() understood the dims keyword, it could be tidier:
julia> _fft(x; dims) = fft(x, dims);
julia> #cast _res[(k,i),j] := _fft(k) mat[(k,i),j] (i in 1:2);
julia> _res ≈ res
true
I came across this question in a coding competition. Given a number n, concatenate the binary representation of first n positive integers and return the decimal value of the resultant number formed. Since the answer can be large return answer modulo 10^9+7.
N can be as large as 10^9.
Eg:- n=4. Number formed=11011100(1=1,10=2,11=3,100=4). Decimal value of 11011100=220.
I found a stack overflow answer to this question but the problem is that it only contains a O(n) solution.
Link:- concatenate binary of first N integers and return decimal value
Since n can be up to 10^9 we need to come up with solution that is better than O(n).
Here's some Python code that provides a fast solution; it uses the same ideas as in Abhinav Mathur's post. It requires Python >= 3.8, but it doesn't use anything particularly fancy from Python, and could easily be translated into another language. You'd need to write algorithms for modular exponentiation and modular inverse if they're not already available in the target language.
First, for testing purposes, let's define the slow and obvious version:
# Modulus that results are reduced by,
M = 10 ** 9 + 7
def slow_binary_concat(n):
"""
Concatenate binary representations of 1 through n (inclusive).
Reinterpret the resulting binary string as an integer.
"""
concatenation = "".join(format(k, "b") for k in range(n + 1))
return int(concatenation, 2) % M
Checking that we get the expected result:
>>> slow_binary_concat(4)
220
>>> slow_binary_concat(10)
462911642
Now we'll write a faster version. First, we split the range [1, n) into subintervals such that within each subinterval, all numbers have the same length in binary. For example, the range [1, 10) would be split into four subintervals: [1, 2), [2, 4), [4, 8) and [8, 10). Here's a function to do that splitting:
def split_by_bit_length(n):
"""
Split the numbers in [1, n) by bit-length.
Produces triples (a, b, 2**k). Each triple represents a subinterval
[a, b) of [1, n), with a < b, all of whose elements has bit-length k.
"""
a = 1
while n > a:
b = 2 * a
yield (a, min(n, b), b)
a = b
Example output:
>>> list(split_by_bit_length(10))
[(1, 2, 2), (2, 4, 4), (4, 8, 8), (8, 10, 16)]
Now for each subinterval, the value of the concatenation of all numbers in that subinterval is represented by a fairly simple mathematical sum, which can be computed in exact form. Here's a function to compute that sum modulo M:
def subinterval_concat(a, b, l):
"""
Concatenation of values in [a, b), all of which have the same bit-length k.
l is 2**k.
Equivalently, sum(i * l**(b - 1 - i)) for i in range(a, b)) modulo M.
"""
n = b - a
inv = pow(l - 1, -1, M)
q = (pow(l, n, M) - 1) * inv
return (a * q + (q - n) * inv) % M
I won't go into the evaluation of the sum here: it's a bit off-topic for this site, and it's hard to express without a good way to render formulas. If you want the details, that's a topic for https://math.stackexchange.com, or a page of fairly simple algebra.
Finally, we want to put all the intervals together. Here's a function to do that.
def fast_binary_concat(n):
"""
Fast version of slow_binary_concat.
"""
acc = 0
for a, b, l in split_by_bit_length(n + 1):
acc = (acc * pow(l, b - a, M) + subinterval_concat(a, b, l)) % M
return acc
A comparison with the slow version shows that we get the same results:
>>> fast_binary_concat(4)
220
>>> fast_binary_concat(10)
462911642
But the fast version can easily be evaluated for much larger inputs, where using the slow version would be infeasible:
>>> fast_binary_concat(10**9)
827129560
>>> fast_binary_concat(10**18)
945204784
You just have to note a simple pattern. Taking up your example for n=4, let's gradually build the solution starting from n=1.
1 -> 1 #1
2 -> 2^2(1) + 2 #6
3 -> 2^2[2^2(1)+2] + 3 #27
4 -> 2^3{2^2[2^2(1)+2]+3} + 4 #220
If you expand the coefficients of each term for n=4, you'll get the coefficients as:
1 -> (2^3)*(2^2)*(2^2)
2 -> (2^3)*(2^2)
3 -> (2^3)
4 -> (2^0)
Let the N be total number of bits in the string representation of our required number, and D(x) be the number of bits in x. The coefficients can then be written as
1 -> 2^(N-D(1))
2 -> 2^(N-D(1)-D(2))
3 -> 2^(N-D(1)-D(2)-D(3))
... and so on
Since the value of D(x) will be the same for all x between range (2^t, 2^(t+1)-1) for some given t, you can break the problem into such ranges and solve for each range using mathematics (not iteration). Since the number of such ranges will be log2(Given N), this should work in the given time limit.
As an example, the various ranges become:
1. 1 (D(x) = 1)
2. 2-3 (D(x) = 2)
3. 4-7 (D(x) = 3)
4. 8-15 (D(x) = 4)
I have written a method that approximates a definite integral by the composite Simpson's rule.
#=
f integrand
a lower integration bound
b upper integration bound
n number of iterations or panels
h step size
=#
function simpson(f::Function, a::Number, b::Number, n::Number)
n % 2 == 0 || error("`n` must be even")
h = (b - a) / n
s = f(a) + f(b)
s += 4*sum(f(a .+ collect(1:2:n) .* h))
s += 2*sum(f(a .+ collect(2:2:n-1) .* h))
return h/3 * s
end
For "simple" functions, like e^(-x^2), the simpson function works.
Input: simpson(x -> simpson(x -> exp.(-x.^2), 0, 5, 100)
Output: 0.8862269254513949
However, for the more complicated function f(x)
gArgs(x) = (30 .+ x, 0)
f(x) = exp.(-x.^2) .* maximum(generator.(gArgs.(x)...)[1])
where generator(θ, plotsol) is a function that takes in a defect θ in percent and a boolean value plotsol (either 0 or 1) that determines whether the generator should be plotted, and returns a vector with the magnetization in certain points in the generator.
When I try to compute the integral by running the below code
gArgs(x) = (30 .+ x, 0)
f(x) = exp.(-x.^2) .* maximum(generator.(gArgs.(x)...)[1])
println(simpson(x -> f(x), 0, 5, 10))
I encounter the error MethodError: no method matching generator(::Float64). With slight variants of the expression for f(x) I run into different errors like DimensionMismatch("array could not be broadcast to match destination") and InexactError: Bool(33.75). In the end, I think the cause of the error boils down to that I cannot figure out how to properly enter an expression for the integrand f(x). Could someone help me figure out how to enter f(x) correctly? Let me know if anything is unclear in my question.
Given an array x , gArgs.(x) returns an array of Tuples and you are trying to broadcast over an array of tuples. But the behavior of broadcasting with tuples is a bit different. Tuples are not treated as a single element and they themselves broadcast.
julia> println.(gArgs.([0.5, 1.5, 2.5, 3.5, 4.5])...)
30.531.532.533.534.5
00000
This is not what you expected, is it?
You can also see the problem with the following example;
julia> (2, 5) .!= [(2, 5)]
2-element BitArray{1}:
true
true
I believe f is a function that actually takes a scalar and returns a scalar. Instead of making f work on arrays, you should leave the broadcasting to the caller. You are very likely to be better of implementing f element-wise. This is the more Julia way of doing things and will make your job much easier.
That said, I believe your implementation should work with the following modifications, if you do not have an error in generator.
function simpson(f::Function, a::Number, b::Number, n::Number)
n % 2 == 0 || error("`n` must be even")
h = (b - a) / n
s = f(a) + f(b)
s += 4*sum(f.(a .+ collect(1:2:n) .* h)) # broadcast `f`
s += 2*sum(f.(a .+ collect(2:2:n-1) .* h)) # broadcast `f`
return h/3 * s
end
# define `gArg` and `f` element-wise and `generator`, too.
gArgs(x) = (30 + x, 0) # get rid of broadcasting dot. Shouldn't `0` be `false`?
f(x) = exp(-x^2) * maximum(generator(gArgs(x)...)[1]) # get rid of broadcasting dots
println(simpson(f, 0, 5, 10)) # you can just write `f`
You should also define the generator function element-wise.
With the aid of broadcasting, the following code will work whether x, y, and z are scalars, vectors of size n, or any combination thereof.
b = zeros(n)
b .= x.*y.*z .+ x
However, I'd like a for-loop. The following for-loop only works when x is a vector of size n, y is a scalar, and z is a scalar.
for i = 1:n
b[i] = x[i]*y*z + x[i]
end
To write the equivalent of b .= x.*y.*z .+ x as a for-loop for any case, I can only think of writing a for-loop for every combination of x, y, and z within if-statements. This can get messy with more variables in more complicated math expressions.
Is there a more elegant way to do what I'd like than using many if-statements?
You could define a wrapper type that indexing into it will give array indexing if wrapped variable is array and repeats the same value for all indices for scalars. I have an example below but it probably is not as efficient as using broadcast. And it is not checking if array lengths are consistent. However, a custom wrapper type would alleviate the situation.
julia> function f(x,y,z)
lx,ly,lz = length(x),length(y),length(z)
maxlen = max(lx,ly,lz)
cx = cycle(x)
cy = cycle(y)
cz = cycle(z)
b = zeros(maxlen)
#inbounds for (xi,yi,zi,i) in zip(cx,cy,cz,1:maxlen)
b[i] = xi*yi*zi+xi
end
return b
end
f (generic function with 1 method)
julia> f(1:3,21,2)
3-element Array{Float64,1}:
43.0
86.0
129.0