What are some of the better libraries for large sparse iterative (conjugate gradient, MINRES, GMRES, etc.) linear algebra system solving? I've often coded my own routines, but I'm interested to know which "off-the-shelf" packages people prefer. I've heard of PETSc, TAUCS, IML++, and a few others. I'm wondering how these stack up, and what else is out there. My preference is for ease of use, and freely available software.
Victor Eijkhout's Overview of Iterative Linear System Solver Packages would probably be a good place to start.
You may also wish to look at Trilinos
http://trilinos.sandia.gov/
It is designed by some great software craftsman, using modern
design techniques.
Moreover, from within Trilinos, you can call PetsC if you desire.
NIST has some sparse Linear Algebra software you can download
here: http://math.nist.gov/sparselib++/ and here: http://math.nist.gov/spblas/
I haven't used those packages myself, but I've heard good things about them.
http://www.cise.ufl.edu/research/sparse/umfpack/
UMFPACK is a set of routines for
solving unsymmetric sparse linear
systems, Ax=b, using the Unsymmetric
MultiFrontal method. Written in
ANSI/ISO C, with a MATLAB (Version 6.0
and later) interface. Appears as a
built-in routine (for lu, backslash,
and forward slash) in MATLAB. Includes
a MATLAB interface, a C-callable
interface, and a Fortran-callable
interface. Note that "UMFPACK" is
pronounced in two syllables, "Umph
Pack". It is not "You Em Ef Pack".
I'm using it for FEM code.
I would check out Microsoft's Solver Foundation. It's free to cheap for even pretty big problems. The unlimited version is industrial strength and is based on Gurobi and of course isn't cheap.
http://code.msdn.microsoft.com/solverfoundation
Related
I build structural design optimization software together with civil engineers in a large renewable energy company. The way we have set it up, the domain experts will do most of their own code (think structural validation of steel part, geophysical response models,...), and my team is responsible for integrating it all and building optimization algorithms on top.
Currently everything is matlab because this is what the engineers are taught at school. We are starting to consider rebuilding the system from scratch, and one question that pops up is what language to use.
My prioritized checklist is:
Easy to learn for a non-CS minded civil engineer
Well suited for linear algebra calculation, large algebraic expressions and numerical work in general
Existing numerical libraries with root finding, basic optimizers, etc. -- or solid support for C/Fortran integration.
So far my favorite is Python, which I have seen several of the engineers get up to speed with in a few weeks, but I was wondering if Julia would also fit the bill.
Do you have experience using Julia as a domain expert language, and would you recommend it for this purpose?
(Keeping this specific to Julia to make it an answerable question -- but feel free to chime in with other language options!)
Like mentioned in the comments this is a question for julia discourse.
However, I cannot resist to comment on your list.
Easy to learn for a non-CS minded civil engineer
Julia is slightly more difficult then Python. I have been teaching Julia to several groups of people and basically if your team has a background such as Python or Matlab transformation to Julia is painless and natural.
Well suited for linear algebra calculation, large algebraic expressions and numerical work in general
This is exactly what is this language designed for. Julia totally overcomes each of Python's shortcomings in that areas.
Existing numerical libraries with root finding, basic optimizers, etc. -- or solid support for C/Fortran integration.
Have a look at https://github.com/JuliaMath and https://github.com/JuliaOpt/
Additionally if you are interested in linear and nonlinear programming have a look at JuMP at https://github.com/JuliaOpt/JuMP.jl - this is ingenious and beautiful package
Additionally, C and Fortran work out-of-the-box: https://docs.julialang.org/en/v1.0/manual/calling-c-and-fortran-code/
Finally, it is worth noting that you can also load and directly use Python libraries in Julia via the PyCall.jl package.
I have a linear algebra code that I am trying get to run faster. Its a iterative algorithm with a loop and matrix vector multiplications within in.
So far, I have used MATMUL (Fortran Lib.), DGEMV, Tried writing my own MV code in OpenMP but the algorithm is doing no better in terms of scalability. Speed ups are barely 3.5 - 4 irrespective of how many processors I am allotting to it (I have tried up 64 processors).
The profiling shows significant time being spent in Matrix-Vector and the rest is fairly nominal.
My question is:
I have a shared memory system with tons of RAM and processors. I have tried tweaking OpenMP implementation of the code (including Matrix Vector) but has not helped. Will it help to code in MPI? I am not a pro at MPI but the ability to fine tune the message communication might help a bit but I can't be sure. Any comments?
More generally, from the literature I have read, MPI = Distributed, OpenMP = Shared but can they perform well in the others' territory? Like MPI in Shared? Will it work? Will it be better than the OpenMP implementation if done well?
You're best off just using a linear algebra package that is already well optimized for a multitcore environment and using that for your matrix-vector multiplication. The Atlas package, gotoblas (if you have a nehalem or older; sadly it's no longer being updated), or vendor BLAS implementations (like MKL for intel CPUs, ACML for AMD, or VecLib for apple, which all cost money) all have good, well-tuned, multithreaded BLAS implementations. Unless you have excellent reason to believe that you can do better than those full time development teams can, you're best off using them.
Note that you'll never get the parallel speedup with DGEMV that you do with DGEMM, just because the vector is smaller than another matrix and so there's less work; but you can still do quite well, and you'll find you get much better perforamance with these libraries than you do with anything hand-rolled unless you were already doing multi-level cache blocking.
You can use MPI in a shared environment (though not OpenMP in a distributed one). However, achieving a good speedup depends a lot more on your algorithms and data dependencies than the technology used. Since you have a lot of shared memory, I'd recommend you stick with OpenMP, and carefully examine whether you're making the best use of your resources.
My application has some parabolic partial differential equations...which are inter-related and use some variables which the user inputs via a UI from a desktop application.
Can you guide me through as to which software or library or a particular language would serve the best purpose for the above?
Maybe Python language with:
PyQt for UI
SciPy for scientific computing
Or Matlab, or its free counterpart gnu octave or scilab, of freemat.
Or just crank it up in Wolfram Alpha web UI.
http://www.wolframalpha.com/input/?i=X^2%2B2x%2B1%3D0
Or Wolfram Mathematica 8.
Since you said "equations", I'll assume there's more than one and that they're coupled. It's highly unlikely that you'll find a closed-form solution for a problem that difficult.
When I hear "parabolic PDE", the prototype for me is transient diffusion. That usually means a numerical integration forward in time using explicit Euler (small steps, unstable), implicit, or Crank-Nicholson integration scheme.
I'd discretize using finite element methods and weighted residuals. This is how you turn those PDEs into matrix equations.
Once both of those are decided upon, you'll have a set of linear algebra problems to solve repeatedly for each time step. You can use any good linear algebra library you have available in the language of your choice.
Maybe MATLAB or Octave, its open source cousin, could help you here.
I need to develop applications doing linear algebra + eigenvalue + linear equation solutions over a cluster of pcs ( I have a lot of machines available).
I discovered Scalapack libraries but they seem to me developed long time ago.
Do you know if these are other libs available that I should learn doing math & linear algebra in a cluster?
My language is C++ and off course I am newbie to this topic.
The kind of problem you are mentionning are very different and I doubt there is a single library that would do everything efficiently. Some libraries may also be suited more specifically for linear algebra problems rising from specific applications (like finite elements problems).
Concerning libraries, I have never used Scalapack, but remember that in this field, old does not necessarily means bad. Here are a few other picks you can choose from:
PETSc : linear solvers
SLEPc : eigenvalue solvers
MUMPS and SuperLU: linear solvers
Of course,...have a look at the netlib repository, you might find interesting things, including some libraries on which the above links rely.
Finally, about the language, remember that efficiency will also depend on the use of the appropriate libraries: Blas ,Atlas, Lapack,... which are most probably written in "some language other than C++" and you'll just have to call them with the appropriate wrapper.
Given my previous questions about the the usage of AMPL.
Are there any other programming/scripting languages that are strictly meant for mathmatical processing?
For example: Matlab (it does deviate a bit from a mathematical structure, but its close enough), Mathematica, and AMPL
R / S+ for statistical computing
Other stat languages: SAS, SPSS, STATA, GAUSS, etc.
Octave, an open source clone of Matlab
Fortress, "a language for high-performance computation that provides abstraction and type safety on par with modern programming language principles."
Maple
Maxima
There's always APL, with its builtin matrix operators. Modern APL even supports .NET.
R, Numpy/scipy for Python, Maple, Yacas, even Fortran.
This may be only of historical significance, but Fortan (The IBM Mathematical Formula Translating System) is especially suited to numeric computation and scientific computing.
OPL (Optimization Programming Language) is one of the most comprehensive modelling languages for Mathematical Programming. You can do Linear Programming (LP), Mixed Integer Programming (MIP), Quadratic Programming (QP), Constraint Programming (CP), MIQP, etc.
IBM-ILOG CPLEX Optimization Studio uses this language.
Maple for symbolic math (similar to Mathematica).
SAS, SPSS, R for statistics.
The Operation Research / Management Science magazine has a yearly survey of Simulation Software, and while I can't find the link I believe they have one yearly survey on optimization packages, such as AMPL you are quoting.
Sage is basically Python with a load of packages and a few language extensions put into a "notebook" interface like that of Mathematica. It has interfaces to all sorts of computer algebra systems. And with Numpy and Scipy (which are included) it's a fine replacement for Matlab. And it's open source and actively developed.
Given your previous question, I assume you are looking for an alternative to commercial mathematics packages. If so, you should try Sage, it is open source and is a unified front end for almost all of the open source mathematics/sci.calc. packages out there (list).
The way it works, is that it uses your web browser as a graphical front end for displaying, editing and evaluating Mathematica style notebooks (it is also possible to just use the command line). All the dirty work, such as selecting the appropriate package for the situation, is done transparently in the background.
Sage uses Python as it's main language / syntax, so it's fairly easy to learn, and if you have old Python scripts, they should work straight out of the box. If I didn't have access to a Mathematica license, I would definitely use this.
Interactive Data Language (IDL) is a proprietary language used in astronomy, medicine and other sciences at least in part because of its built-in array operations and mathematical library.
As this question is still open and well indexed in Google, I would definitively add to the list the Julia language.
Aside the technical aspects that make shine this high level/high performance new language, an important consideration is that the community of developers/users is clearly biased toward mathematicians.