I need to develop applications doing linear algebra + eigenvalue + linear equation solutions over a cluster of pcs ( I have a lot of machines available).
I discovered Scalapack libraries but they seem to me developed long time ago.
Do you know if these are other libs available that I should learn doing math & linear algebra in a cluster?
My language is C++ and off course I am newbie to this topic.
The kind of problem you are mentionning are very different and I doubt there is a single library that would do everything efficiently. Some libraries may also be suited more specifically for linear algebra problems rising from specific applications (like finite elements problems).
Concerning libraries, I have never used Scalapack, but remember that in this field, old does not necessarily means bad. Here are a few other picks you can choose from:
PETSc : linear solvers
SLEPc : eigenvalue solvers
MUMPS and SuperLU: linear solvers
Of course,...have a look at the netlib repository, you might find interesting things, including some libraries on which the above links rely.
Finally, about the language, remember that efficiency will also depend on the use of the appropriate libraries: Blas ,Atlas, Lapack,... which are most probably written in "some language other than C++" and you'll just have to call them with the appropriate wrapper.
Related
I build structural design optimization software together with civil engineers in a large renewable energy company. The way we have set it up, the domain experts will do most of their own code (think structural validation of steel part, geophysical response models,...), and my team is responsible for integrating it all and building optimization algorithms on top.
Currently everything is matlab because this is what the engineers are taught at school. We are starting to consider rebuilding the system from scratch, and one question that pops up is what language to use.
My prioritized checklist is:
Easy to learn for a non-CS minded civil engineer
Well suited for linear algebra calculation, large algebraic expressions and numerical work in general
Existing numerical libraries with root finding, basic optimizers, etc. -- or solid support for C/Fortran integration.
So far my favorite is Python, which I have seen several of the engineers get up to speed with in a few weeks, but I was wondering if Julia would also fit the bill.
Do you have experience using Julia as a domain expert language, and would you recommend it for this purpose?
(Keeping this specific to Julia to make it an answerable question -- but feel free to chime in with other language options!)
Like mentioned in the comments this is a question for julia discourse.
However, I cannot resist to comment on your list.
Easy to learn for a non-CS minded civil engineer
Julia is slightly more difficult then Python. I have been teaching Julia to several groups of people and basically if your team has a background such as Python or Matlab transformation to Julia is painless and natural.
Well suited for linear algebra calculation, large algebraic expressions and numerical work in general
This is exactly what is this language designed for. Julia totally overcomes each of Python's shortcomings in that areas.
Existing numerical libraries with root finding, basic optimizers, etc. -- or solid support for C/Fortran integration.
Have a look at https://github.com/JuliaMath and https://github.com/JuliaOpt/
Additionally if you are interested in linear and nonlinear programming have a look at JuMP at https://github.com/JuliaOpt/JuMP.jl - this is ingenious and beautiful package
Additionally, C and Fortran work out-of-the-box: https://docs.julialang.org/en/v1.0/manual/calling-c-and-fortran-code/
Finally, it is worth noting that you can also load and directly use Python libraries in Julia via the PyCall.jl package.
My application has some parabolic partial differential equations...which are inter-related and use some variables which the user inputs via a UI from a desktop application.
Can you guide me through as to which software or library or a particular language would serve the best purpose for the above?
Maybe Python language with:
PyQt for UI
SciPy for scientific computing
Or Matlab, or its free counterpart gnu octave or scilab, of freemat.
Or just crank it up in Wolfram Alpha web UI.
http://www.wolframalpha.com/input/?i=X^2%2B2x%2B1%3D0
Or Wolfram Mathematica 8.
Since you said "equations", I'll assume there's more than one and that they're coupled. It's highly unlikely that you'll find a closed-form solution for a problem that difficult.
When I hear "parabolic PDE", the prototype for me is transient diffusion. That usually means a numerical integration forward in time using explicit Euler (small steps, unstable), implicit, or Crank-Nicholson integration scheme.
I'd discretize using finite element methods and weighted residuals. This is how you turn those PDEs into matrix equations.
Once both of those are decided upon, you'll have a set of linear algebra problems to solve repeatedly for each time step. You can use any good linear algebra library you have available in the language of your choice.
Maybe MATLAB or Octave, its open source cousin, could help you here.
What are some of the better libraries for large sparse iterative (conjugate gradient, MINRES, GMRES, etc.) linear algebra system solving? I've often coded my own routines, but I'm interested to know which "off-the-shelf" packages people prefer. I've heard of PETSc, TAUCS, IML++, and a few others. I'm wondering how these stack up, and what else is out there. My preference is for ease of use, and freely available software.
Victor Eijkhout's Overview of Iterative Linear System Solver Packages would probably be a good place to start.
You may also wish to look at Trilinos
http://trilinos.sandia.gov/
It is designed by some great software craftsman, using modern
design techniques.
Moreover, from within Trilinos, you can call PetsC if you desire.
NIST has some sparse Linear Algebra software you can download
here: http://math.nist.gov/sparselib++/ and here: http://math.nist.gov/spblas/
I haven't used those packages myself, but I've heard good things about them.
http://www.cise.ufl.edu/research/sparse/umfpack/
UMFPACK is a set of routines for
solving unsymmetric sparse linear
systems, Ax=b, using the Unsymmetric
MultiFrontal method. Written in
ANSI/ISO C, with a MATLAB (Version 6.0
and later) interface. Appears as a
built-in routine (for lu, backslash,
and forward slash) in MATLAB. Includes
a MATLAB interface, a C-callable
interface, and a Fortran-callable
interface. Note that "UMFPACK" is
pronounced in two syllables, "Umph
Pack". It is not "You Em Ef Pack".
I'm using it for FEM code.
I would check out Microsoft's Solver Foundation. It's free to cheap for even pretty big problems. The unlimited version is industrial strength and is based on Gurobi and of course isn't cheap.
http://code.msdn.microsoft.com/solverfoundation
Given my previous questions about the the usage of AMPL.
Are there any other programming/scripting languages that are strictly meant for mathmatical processing?
For example: Matlab (it does deviate a bit from a mathematical structure, but its close enough), Mathematica, and AMPL
R / S+ for statistical computing
Other stat languages: SAS, SPSS, STATA, GAUSS, etc.
Octave, an open source clone of Matlab
Fortress, "a language for high-performance computation that provides abstraction and type safety on par with modern programming language principles."
Maple
Maxima
There's always APL, with its builtin matrix operators. Modern APL even supports .NET.
R, Numpy/scipy for Python, Maple, Yacas, even Fortran.
This may be only of historical significance, but Fortan (The IBM Mathematical Formula Translating System) is especially suited to numeric computation and scientific computing.
OPL (Optimization Programming Language) is one of the most comprehensive modelling languages for Mathematical Programming. You can do Linear Programming (LP), Mixed Integer Programming (MIP), Quadratic Programming (QP), Constraint Programming (CP), MIQP, etc.
IBM-ILOG CPLEX Optimization Studio uses this language.
Maple for symbolic math (similar to Mathematica).
SAS, SPSS, R for statistics.
The Operation Research / Management Science magazine has a yearly survey of Simulation Software, and while I can't find the link I believe they have one yearly survey on optimization packages, such as AMPL you are quoting.
Sage is basically Python with a load of packages and a few language extensions put into a "notebook" interface like that of Mathematica. It has interfaces to all sorts of computer algebra systems. And with Numpy and Scipy (which are included) it's a fine replacement for Matlab. And it's open source and actively developed.
Given your previous question, I assume you are looking for an alternative to commercial mathematics packages. If so, you should try Sage, it is open source and is a unified front end for almost all of the open source mathematics/sci.calc. packages out there (list).
The way it works, is that it uses your web browser as a graphical front end for displaying, editing and evaluating Mathematica style notebooks (it is also possible to just use the command line). All the dirty work, such as selecting the appropriate package for the situation, is done transparently in the background.
Sage uses Python as it's main language / syntax, so it's fairly easy to learn, and if you have old Python scripts, they should work straight out of the box. If I didn't have access to a Mathematica license, I would definitely use this.
Interactive Data Language (IDL) is a proprietary language used in astronomy, medicine and other sciences at least in part because of its built-in array operations and mathematical library.
As this question is still open and well indexed in Google, I would definitively add to the list the Julia language.
Aside the technical aspects that make shine this high level/high performance new language, an important consideration is that the community of developers/users is clearly biased toward mathematicians.
I'm not looking for a general discussion on if math is important or not for programming.
Instead I'm looking for real world scenarios where you have actually used some branch of math to solve some particular problem during your career as a software developer.
In particular, I'm looking for concrete examples.
I frequently find myself using De Morgan's theorem when as well as general Boolean algebra when trying to simplify conditionals
I've also occasionally written out truth tables to verify changes, as in the example below (found during a recent code review)
(showAll and s.ShowToUser are both of type bool.)
// Before
(showAll ? (s.ShowToUser || s.ShowToUser == false) : s.ShowToUser)
// After!
showAll || s.ShowToUser
I also used some basic right-angle trigonometry a few years ago when working on some simple graphics - I had to rotate and centre a text string along a line that could be at any angle.
Not revolutionary...but certainly maths.
Linear algebra for 3D rendering and also for financial tools.
Regression analysis for the same financial tools, like correlations between financial instruments and indices, and such.
Statistics, I had to write several methods to get statistical values, like the F Probability Distribution, the Pearson product moment coeficient, and some Linear Algebra correlations, interpolations and extrapolations for implementing the Arbitrage pricing theory for asset pricing and stocks.
Discrete math for everything, linear algebra for 3D, analysis for physics especially for calculating mass properties.
[Linear algebra for everything]
Projective geometry for camera calibration
Identification of time series / statistical filtering for sound & image processing
(I guess) basic mechanics and hence calculus for game programming
Computing sizes of caches to optimize performance. Not as simple as it sounds when this is your critical path, and you have to go back and work out the times saved by using the cache relative to its size.
I'm in medical imaging, and I use mostly linear algebra and basic geometry for anything related to 3D display, anatomical measurements, etc...
I also use numerical analysis for handling real-world noisy data, and a good deal of statistics to prove algorithms, design support tools for clinical trials, etc...
Games with trigonometry and AI with graph theory in my case.
Graph theory to create a weighted graph to represent all possible paths between two points and then find the shortest or most efficient path.
Also statistics for plotting graphs and risk calculations. I used both Normal distribution and cumulative normal distribution calculations. Pretty commonly used functions in Excel I would guess but I actully had to write them myself since there is no built-in support in the .NET libraries. Sadly the built in Math support in .NET seem pretty basic.
I've used trigonometry the most and also a small amount a calculus, working on overlays for GIS (mapping) software, comparing objects in 3D space, and converting between coordinate systems.
A general mathematical understanding is very useful if you're using 3rd party libraries to do calculations for you, as you ofter need to appreciate their limitations.
i often use math and programming together, but the goal of my work IS the math so use software to achive that.
as for the math i use; mostly Calculus (FFT's analysing continuous and discrete signals) with a slash of linar algebra (CORDIC) to do trig on a MCU with no floating point chip.
I used a analytic geometry for simple 3d engine in opengl in hobby project on high school.
Some geometry computation i had used for dynamic printing reports, where was another 90° angle layout than.
A year ago I used some derivatives and integrals for store analysis (product item movement in store).
Bot all the computation can be found on internet or high-school book.
Statistics mean, standard-deviation, for our analysts.
Linear algebra - particularly gauss-jordan elimination and
Calculus - derivatives in the form of difference tables for generating polynomials from a table of (x, f(x))
Linear algebra and complex analysis in electronic engineering.
Statistics in analysing data and translating it into other units (different project).
I used probability and log odds (log of the ratio of two probabilities) to classify incoming emails into multiple categories. Most of the heavy lifting was done by my colleague Fidelis Assis.
Real world scenarios: better rostering of staff, more efficient scheduling of flights, shortest paths in road networks, optimal facility/resource locations.
Branch of maths: Operations Research. Vague definition: construct a mathematical model of a (normally complex) real world business problem, and then use mathematical tools (e.g. optimisation, statistics/probability, queuing theory, graph theory) to interrogate this model to aid in the making of effective decisions (e.g. minimise cost, maximise efficency, predict outcomes etc).
Statistics for scientific data analyses such as:
calculation of distributions, z-standardisation
Fishers Z
Reliability (Alpha, Kappa, Cohen)
Discriminance analyses
scale aggregation, poling, etc.
In actual software development I've only really used quite trivial linear algebra, geometry and trigonometry. Certainly nothing more advanced than the first college course in each subject.
I have however written lots of programs to solve really quite hard math problems, using some very advanced math. But I wouldn't call any of that software development since I wasn't actually developing software. By that I mean that the end result wasn't the program itself, it was an answer. Basically someone would ask me what is essentially a math question and I'd write a program that answered that question. Sure I’d keep the code around for when I get asked the question again, and sometimes I’d send the code to someone so that they could answer the question themselves, but that still doesn’t count as software development in my mind. Occasionally someone would take that code and re-implement it in an application, but then they're the ones doing the software development and I'm the one doing the math.
(Hopefully this new job I’ve started will actually let me to both, so we’ll see how that works out)