Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking for code must demonstrate a minimal understanding of the problem being solved. Include attempted solutions, why they didn't work, and the expected results. See also: Stack Overflow question checklist
Closed 9 years ago.
Improve this question
Im using a genetic algorithm (witness.genetic from package "witness") to fit a model to a dataset. The current system time is taking 61 hours on this machine:
Intel Core i7-2640M CPU # 2.80 GHz 2.80 GHz
RAM: 4 GB
OS: 64-bit windows
I want to speed up this process. In my research so far its seems there are memory issues both with storage of a vector (for a 64-bit ) and processing speed. Will improving my hardware help with speed up a genetic algorithm or the speed limited by the OS. IF the former, what aspects of my hardware should I focus on?
The answer is almost always to write a better algorithm! Moore's law says upgrading a 10 year old computer will give a 32 times speed-up. But designing a O(n log n) algorithm to replace a O(n**2) algorithm will give a 2500 speed up (assuming there are 10k items).
Read Steven Skiena's Algorithm Design Manual for examples.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 years ago.
Improve this question
I am exploring parallel programming in R and I have a good understanding of how the foreach function works, but I don't understand the differences between parallel,doparallel,doMC,doSNOW,SNOW,multicore, etc.
After doing a bunch of reading it seems that these packages work differently depending on the operating system, and I see some packages use the word multicore, and others use cluster (I am not sure if those are different), but beyond that it isn't clear what advantages or disadvantages each have.
I am working Windows, and I want to calculate standard errors using replicate weights in parallel so I don't have to calculate each replicate one at a time (if I have n cores I should be able to do n replicates at once). I was able to implement it using doSNOW, but it looks like plyr and the R community in general uses doMC so I am wondering if using doSNOW is a mistake.
Regards,
Carl
My understanding is that parallel is a conglomeration of snow and multicore, and is meant to incorporate the best parts of both.
For parallel computing on a single machine, I find parallel to have been very effective.
For parallel computing using a cluster of multiple machines, I've never succeeded in completing the cluster set up using parallel, but have succeeded using snow.
I've never used any of the do* packages, so I'm afraid I'm unable to comment.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
I worked out an estimator, and I would like check its performance by doing simulation studies with R. I want to repeat the experiment for 500 times. Unfortunately, the computation involved in the estimator is sophisticated. Each replication will take 15 minutes on my desktop. I am looking for some distributed computation approaches with R. How should I start? I googled this topic. There are some many posts about this.
I'd suggest starting with the foreach package. If you're using mac or linux the following is the simplest way to do parallel computing:
# First we register a parallel backend. This will work on mac and linux.
# Windows is more complicated, try the `snow` package.
library(doMC)
registerDoMC(cores=4) # substitute for number of cores you want to run on.
# now we can run things in parallel using foreach
foreach (i = 1:4) %dopar% {
# What's in here will run on a separate core for each iteration.
}
You should read the vignette for foreach as it's quite different to for (especially nested loops) and it is also quite powerful for combining results at the end and returning them.
First step with any R problem as broad as this should be checking the CRAN Task Views. Oh look:
http://cran.r-project.org/web/views/HighPerformanceComputing.html
Note that StackOverflow isn't really the place for asking broad questions that are best answered with 'read that documentation over there' or 'why don't you try using tool X?'
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
I am looking for a free software for mathematical modeling.
Here is a list of things I might be willing to achieve with this software: Integrating functions, solving differential equations, graph theory, analyzing infinite series, local stability analysis, Taylor series, get eigenvectors, compute the long term behaviour of a system of equations, etc...
Here is a related SE post. I am surprised that nobody is suggesting R. I am currently a R user and already use R for graph theory. Therefore I would appreciate to use R also to make other mathematical modeling. Is R less efficient that Sage, SimPy, Mathematica and others for mathematical modeling? Why? Do you know a manual providing exaplanation for how to make mathematical modeling with R?
Thank you
Sounds like R is your first way to go. It does not make to good sense to compare R with any other tool in such a braod way you are asking for. R packages differ largely in efficiency, some are in fact C tools while others are written in the R language. As a start R can hardly be any wrong and is free.
Matlab might be a stable alternative, Julia is rising but still pre alpha.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 8 years ago.
Improve this question
I am trying to do modular exponentiation of integers with a very large modulus by repetitive squaring (the power is always a power of 2 in my case, so I believe this is the most efficient way). Thanks to a nice property of my modulus, computing remainder is cheap; the hard part is multiplication.
Currently I run GMP on Intel Core 2 Quad. I would like to make efficient use of the four cores of the processor, but GMP does not scale on SMP environments, so I am looking for a substitute arbitrary-precision arithmetic library. I have found some libraries for parallel computation on matrices, but what I really need is a library for integers.
Does what I am looking for exist?
The answer is yes, multi-threaded arbitrary-precision libraries do exist. But I'm not aware of a single one that is actually public. (with comparable speed to GMP)
For example, the arbitrary-precision libraries that are used in the Pi-computing programs, TachusPi and y-cruncher are capable of multi-threaded arithmetic on large numbers.
However, both libraries are closed source and are not available to the public for use.
Affiliation Disclosure: I'm the author of y-cruncher. So I have written one of such multi-threaded arbitrary-precision libraries myself.
Have you check out http://mpir.org? They claim to be doing this with a variant of GMP, and using GPUs.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
I needed an application for solving linear systems of equations (N up to 10), so I got different codes, and compile them, and they seem to work, but I get lots of problems with precision. I mean, the solvers are really very sensitive to small changes of the system.
So, could somebody recommend to me a reliable commandl ine application for this purpose? Or some useful open source code (and easy to compile)
Thanks
GNU Octave is essentially a free version of Matlab (the syntax is identical for basic operations), so you can try things out there and see how they compare to the answers that you're getting.
Having said that, if your answer is very sensitive to the input, it's possible that your problem is ill-conditioned - you can check this by computing the condition number of the matrix in Octave. It's hard to say what to do in that case without knowing more specifics on the problem.
Also, you don't mention which method you're currently using. Gaussian elimination (i.e. "what you learned in math class") is notoriously numerically unstable if you don't use pivoting (see the wikipedia entry for "Pivoting"); adding that might be enough to improve the quality of the results.
An approach is to use the numpy package in Python. You can create a 2d matrix A and a 1d vector b, then solve Ax=b for x using solve(A, x). It's part of the linalg subpackage of numpy.