What makes a convolution kernel separable? How would I be able to tell what those separable parts were in order to do two 1D convolutions instead of a 2D convolution>
Thanks
If the 2D filter kernel has a rank of 1 then it is separable. You can test this in e.g. Matlab or Octave:
octave-3.2.3:1> sobel = [-1 0 1 ; -2 0 2 ; -1 0 1];
octave-3.2.3:2> rank(sobel)
ans = 1
octave-3.2.3:3>
See also: http://blogs.mathworks.com/steve/2006/11/28/separable-convolution-part-2/ - this covers using SVD (Singular Value Decomposition) to extract the two 1D kernels from a separable 2D kernel.
See also this question on DSP.stackexchange.com: Fast/efficient way to decompose separable integer 2D filter coefficients
you can also split the matrix into symmetric and skew parts and separate each part, which can be effective for larger 2d convolutions.
Related
I am trying to generate two vectors with a given cosine similarity. Input would be the degree of cosine similarity (or angle as it depends on it anyway) and the number of dimensions (D) in the vectors, and output would be two vectors of D dimensions with that given similarity between them Now, I know how to use the cosine similarity function to calculate the similarity but I'm lost when trying it the other way around.
Is there such a procedure or algorithm and how is it called?
For a given starting vector u and cosine similarity c:
Generate 2 points in n-D space; call them a and b
Project b onto the plane orthogonal to u and containing a
Subtract a from the result to obtain a vector orthogonal to u; call it h
Use [u,h] as a basis and basic trigonometry to generate the desired vector v
The above method is dimension-agnostic as it only uses dot products. The resultant vectors {v} are of unit length and uniformly distributed around u.
I am trying to match the result of doing a 2D FFT using the already implemented calls in FFTW and my own version of 2D FFT via 1D FFTw calls and mpi communication.
So, resuming, I've followed the theory:
1 - FFT in y dimension
2 - transpose the matrix
3 - MPI_Alltoall communication
4- FFT in x dimension
5- transpose back
6 - MPI_Alltoall communication
I've tried with a small number of processors (8- 12) and it seems to work fine. Correctness has been carried out using RMS between the 2D FFTW call and my own result. However, as I increase the number of cores and size of the matrix, it seems that I am loosing precision, ie. RMS fails because the error is larger than expected (I set the error to 1.0e-10):
Given a matrix of 512x512 and considering RMS of tolerance of 1.0e-6:
"ERROR: Position 1 0 expected im-part 10936907150.600960 and got 10936907150.600958"
Given a matrix of 2048x2048 and considering RMS of tolerance of 1.0e-6:
"ERROR:Position 1 0 expected real part -4294967296.000107 and got -4294967295.999999"
Why would I lose precision if all I am using are double types?
So we have a matrix like
12,32
24,12
...
with length 2xN and another
44,32
44,19
...
with length 2xN and there is some function f(x, y) that returns z[1], z[2]. That 2 matrices that we were given represent known value pairs for x,y and z[1],z[2]. What are interpolation formulas that would help in such case?
If you solve the problem for one return value, you can find two functions f_1(x,y) and f_2(x,y) by interpolation, and compose your function as f(x, y) = [f_1(x,y), f_2(x,y)]. Just pick any method for solving the interpolation function suitable for your problem.
For the actual interpolation problem in two dimensions, there are a lot of ways you can handle this. If simple is what you require, you can go with linear interpolation. If you are OK with piecewise functions, you can go for bezier curves, or splines. Or, if data is uniform, you could get away with a simple polynomial interpolation (well, not quite trivial when in 2D, but easy enough).
EDIT: More information and some links.
A piecewise solution is possible using Bilinear interpolation (wikipedia).
For polynomial interpolation, if your data is on a grid, you can use the following algorithm (I cannot find the reference for it, it is from memory).
If the data points are on a k by l grid, rewrite your polynomial as follows:
f(x,y) = cx_1(x)*y^(k-1) + cx_2(x)*y^(k-2) + ... + cx_k(x)
Here, each coefficient cx_i(x) is also a polynomial of degree l. The first step is to find k polynomials of degree l by interpolating each row or column of the grid. When this is done, you have l coefficient sets (or, in other words, l polynomials) as interpolation points for each cx_i(x) polynomials as cx_i(x0), cx_i(x1), ..., cx_i(xl) (giving you a total of l*k points). Now, you can determine these polynomials using the above constants as the interpolation points, which give you the resulting f(x,y).
The same method is used for bezier curves or splines. The only difference is that you use control points instead of polynomial coefficients. You first get a set of splines that will generate your data points, and then you interpolate the control points of these intermediate curves to get the control points of the surface curve.
Let me add an example to clarify the above algorithm. Let's have the following data points:
0,0 => 1
0,1 => 2
1,0 => 3
1,1 => 4
We start by fitting two polynomials: one for data points (0,0) and (0,1), and another for (1, 0) and (1, 1):
f_0(x) = x + 1
f_1(x) = x + 3
Now, we interpolate in the other direction to determine the coefficients.When we read these polynomial coefficients vertically, we need two polynomials. One evaluates to 1 at both 0 and 1; and another that evaluates to 1 at 0, and 3 at 1:
cy_1(y) = 1
cy_2(y) = 2*y + 1
If we combine these into f(x,y), we get:
f(x,y) = cy_1(y)*x + cy_2(y)
= 1*x + (2*y + 1)*1
= x + 2*y + 1
I wonder if it is possible (and if it is then how) to re-present an arbitrary M3 matrix transformation as a sequence of simpler transformations (such as translate, scale, skew, rotate)
In other words: how to calculate MTranslate, MScale, MRotate, MSkew matrices from the MComplex so that the following equation would be true:
MComplex = MTranslate * MScale * MRotate * MSkew (or in an other order)
Singular Value Decomposition (see also this blog and this PDF). It turns an arbitrary matrix into a composition of 3 matrices: orthogonal + diagonal + orthogonal. The orthogonal matrices are rotation matrices; the diagonal matrix represents skewing along the primary axes = scaling.
The translation throws a monkey wrench into the game, but what you should do is take out the translation part of the matrix so you have a 3x3 matrix, run SVD on that to give you the rotation+skewing, then add the translation part back in. That way you'll have a rotation + scale + rotation + translate composition of 4 matrices. It's probably possible to do this in 3 matrices (rotation + scaling along some set of axes + translation) but I'm not sure exactly how... maybe a QR decomposition (Q = orthogonal = rotation, but I'm not sure if the R is skew-only or has a rotational part.)
Yes, but the solution will not be unique. Also you should rather put translation at the end (the order of the rest doesn't matter)
For any given square matrix A there exists infinitely many matrices B and C so that A = B*C. Choose any invertible matrix B (which means that B^-1 exists or det(B) != 0) and now C = B^-1*A.
So for your solution first decompose MC into MT and MS*MR*MSk*I, choosing MT to be some invertible transposition matrix. Then decompose the rest into MS and MR*MSk*I so that MS is arbitrary scaling matrix. And so on...
Now if at the end of the fun I is an identity matrix (with 1 on diagonal, 0 elsewhere) you're good. If it is not, start over, but choose different matrices ;-)
In fact, using the method above symbolically you can create set of equations that will yield you a parametrized formulas for all of these matrices.
How useful these decompositions would be for you, well - that's another story.
If you type this into Mathematica or Maxima they'll compute this for you in no time.
Say i'm trying to evaluate the Polynomial:
x^2 + 1
Using the Fast Fourier transform method for evaluating co-efficients. Now i can change this into matrix/vector form using the co-effcient as inputs for the fast fourier transform:
so:
x^2 + 1 = <1, 0, 1, 0>
This is done by using the coefficient value e.g 1 = 1, 0x^1 = 0, X^2 = 1 and so on
Now we get to the bit where i'm totally confused. I'm meant to use the vandermonde matrix :Vandermonde matrix ~ Wiki to evaluate these values into FFT Form using the matrix:
1 1 1 1
1 i-1-i
1-1 1-i
1-i 1 i
The output of
fft(1,0,1,0)
is
(2,0,2,0)
Now thats the step i don't quite understand, how did we use that matrix to get (2,0,2,0)?
First, your Vandermonde matrix is incorrect. The (4,3) entry should be -1, not 1, since the fourth row should be (-i)0, (-i)1, (-i)2, (-i)3. Note in particular that
(-i)*(-i) = (-1)2 * i2 = i2 = -1.
With this correction, the result follows from multiplying the Vandermonde matrix by the column vector (1,0,1,0).
Maybe you could explain what your overall goal is here. I have never heard of FFTs being used to evaluate polynomials. They are used to multiply polynomials, or to convolve signals (an equivalent task), but I wouldn't bother unless the polynomials/signals have a large number of terms. x2 + 1 isn't large. 16 terms is not large, and even 64 or 256 terms is probably better done by straightforward O(N2) techniques.
Discrete Fourier Transforms use the matrix Mij = ωij where ω is the Nth complex root of 1 and column/row numbering goes from 0 to N-1.
Fast Fourier Transforms never use this matrix directly, they are heavily optimized to use a divide-and-conquer technique (Cooley-Tukey algorithm) to calculate the end result through stages of 2x2 DFTs in series and parallel.
If you write your vector as [0,1,0,1] instead of [1,0,1,0], I think you will see that if you multiply that by the matrix you gave, you'll get [0,2,0,2]. (Although you have an error, it's
1 1 1 1
1 i-1-i
1-1 1-1
1-i-1 i
) There must be some convention in the program you are using which reverses the order of the vector's coefficients.