How do I deal with sparse matrices in JuMP?
For example, suppose I want to impose a constrain of the form:
A * x == 0
where A is a sparse matrix and x a vector of variables. I assume that the sparsity of A could be exploited to make the optimization faster. How can I take advantage of this in JuMP?
JuMP already benefits from sparse matrix in different ways, I've not checked the source but refer to a cited paper from JuMP.jl:
In the case of LP, the input data structures are the vectors c and b
and the matrix A in sparse format, and the routines to generate these
data structures are called matrix generators
One point to note is that, the main task of algebraic modeling languages (AMLs) like JuMP is to generate input data structures for solvers. AMLs like JuMP do not solve generated problems themselves but they call standard appropriate solvers to do the task.
Related
I have a 10000x10000 array in Julia, say A=rand(10000,10000). How can I store that large array so I can work with it in a IDE like Atom/Juno, performing matrices operations, determinants, eigenvalues and so on? Or even if I transfer that array to R, is it a way to work with that large array in R?
If your data is sparse (not all cells have values) you can store it as a sparse Matrix, which will greatly improve the memory footprint (see https://docs.julialang.org/en/v1/stdlib/SparseArrays/). Whether or not it fits into memory also depends on what the elements of the Matrix are. E.g. can you represent the values with Int8 or do you need 64-bit precision elements? A Matrix is not just a Matrix.
On a more general note, if your objects become so big they don't fit into memory, you can write them to disk and "memory-map" them, in that way you can use on-disk Matrices for anything you can use a normal Matrix for. You can check the documentation here: https://docs.julialang.org/en/v1/stdlib/Mmap
I currently use explicit matrix storage for my generalized Eigenvalue equation of the form $AX = \lambda BX$ with eigenvalue lambda and eigenvector $X$. $A$ and $B$ are pentadiagonal by blocks, Hermitian and every block is Hermitian as well.
The problem is that for large simulations memory usage gets out of hand. I would therefore like to switch to Shell matrices. An added advantage would be that then I can avoid the duplication of a lot of information, as A and B are both filled through finite differences. I.e., the first derivative of a function X can be approximated by $X_i' = \frac{X_{i+1}-X_{i-1}}{\Delta}$, so that the same piece of information appears in two places. It gets (much) worse for higher orders.
When I try to implement this in Fortran, using multiple MPI processes that each contain a part of the rows of $A$ and $B$, I stumble upon the following issue: To perform matrix multiplication, one needs the vector information of $X$ from other ranks at the end of each rank's interval, due to the off-diagonal elements of $A$ and $B$.
I found a conceptual solution by using MPI all to all commands that pass the information from these "ghosted" regions to the ranks next-door. However, I fear that this might not be most portable, and also not too elegant.
Is there any way to automate this process of taking the information from ghost zones in Petsc / Slepc?
I am trying to solve a 5x5 Cholesky decomposition (for a variance-covariance matrix) all in terms of unknowns (no constants).
A simplified version, for the sake of giving an example, would be a 2x2 decomposition:
[[a,0],[b,c]]*[[a,b],[0,c]]=[[U1,U2],[U2,U3]]
Is there a software (I'm proficient in R, so if R can do it that would be great) that could solve the above to yield an answer of the left-hand variables in terms of the right-hand variables? i.e. this would be the final answer:
a = sqrt(U1)
b = U2/sqrt(U1)
c = sqrt(U3+U2/U1)
Take a look at this Wikipedia section.
The symbolic definition of the (i,j)th entry of the decomposition is defined recursively in terms of the entries above and to the left. You could implement these recursions using Matlab's Symbolic Math Toolbox and then apply them (symbolically) to obtain your formulas for the 5x5 case. Be warned that you'll probably end up with extremely complicated formulas for some of the unknowns, and - excepting unusual circumstances - it will be fine to implement the decomposition iteratively even for a fixed size 5x5 matrix.
I am trying to run the full SVD of a large (120k x 600k) and sparse (0,1% of non-null values) matrix M. Due to memory limitation all my previous attempts failed (with SVDLIBC, Octave, and R) and I am (almost) resigned to exploring other approaches to my problem (LSA).
However, at the moment, I am only interested in the eigenvalues of the diagonal matrix S and not in the left/right singular vectors (matrices U and V).
Is there a way to compute those singular values without storing in memory the dense matrix M and/or the singular vector matrices U and V?
Any help will be greatly appreciated.
[EDIT] My server configuration: 3,5GHz/3,9GHz (6 cores / 12 threads) 128GB of RAM
Looking for the meaning of that values (elements of matrix S from a SVD decomposition) in wikipedia we get:
The non-zero singular values of M (found on the diagonal entries of Σ)
are the square roots of the non-zero eigenvalues of both M*M and MM*
So you can look for the eigenvalues of the matrix A*A' (120k x 120k) without explicitly build the matrix, of course.
By the way, I dont think you are interested in ALL the eigenvalues (or singular values) for a matrix with such a dimensions. I do not think that any algorithm will give enough accurate results.
How comfortable are you with Fortran? I think you should be able to complete the computations using prebuilt packages available here and/or here. Also, if you're open to C++ and a decomposition using randomized and re-orthonormalized matrices, you can try the code at the google code project called redsvd. (I can't post the link because I don't have the requisite reputation for three links, but you can search for redsvd and find it readily.)
Suppose there are 14 objects, each of which have or do not have 1000 binary features. I have a 14x14 similarity matrix, but not the raw 14x1000 data. Is there a way to reconstruct or generate something similar to the raw data, given the similarity matrix?
I tried Monte Carlo simulations, but unconstrained they would take way too much time to achieve even a low level of consistency with the original similarity matrix.
I saw this relevant question: Similarity matrix -> feature vectors algorithm?. However, they wanted to reduce not increase dimensionality. Also, I am not sure (1) which matrix or matrices to use, and (2) how to convert into a binary matrix.
It's impossible to say for sure unless you describe how the similarity scores were computed.
In general, for the usual kind of similarity scoring this is not possible: information has been lost in the transformation from individual features to aggregate statistics. The best you can hope to do is to arrive at a set of features that are consistent with the similarity scores.
I think that is what you are talking about when you say "similar to" the original. That problem is pretty interesting. Suppose similarity was computed as the dot-product of two feature vectors (ie the count of features for a pair of objects that both have value = 1/true). This is not the only choice: it is consistent with value of 0 (false) meaning no information. But it may generalize to other similarity measures.
In such a case, the problem is really a linear programming problem: a naive approach is to exhaustively search the space of possible objects - not randomly, but guided by the constraints. For example, suppose SIM(A,B) := similarity of object A and object B. Define an order on these vectors.
If SIM(A,B) = N, then choose A=B minimal (like (1,....,1 (N times), 0, .... 0 (1000-N times)), and then choose the minimum C s.t. (A,C), (B,C) have the given values. Once you find an inconsistency, backtrack, and increment.
This will find a consistent answer, although the complexity is very high (but probably better than monte carlo).
Finding a better algorithm is an interesting problem, but more than this I can't say in a SO post - that's probably a topic for a CS thesis!