Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 7 years ago.
Improve this question
I use Mac OS X 10.6.8 (yeah, it's an old MBP but it works)...
Still, i think there's something not working properly. The machine gets dead slow but there should be around 1.12 GB left:
I have the feeling the memory is not allocated to the R program. Here no other software is running, and R is doing some stopword removal operation on a 600 MB corpus.
(I'm posting this here because i dont believe i could make it to apple.stackoverflow but feel free to migrate to a better forum).
How can I "free" the inactive RAM to either the system or the running application?
I am not sure of your actual application, but if you are storing large datasets in memory, then one way to reduce memory usage is to delete them after they are no longer needed using rm(). For example, if I load a large dataset "allData", and then filter it as "filteredData", then it would be best to remove allData by rm(allData) if it is not being used anymore.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I have used mclapply quite a bit and love it. It is a memory hog but very convenient. Alas, now I have a different problem that is not simply embarrassingly parallel.
Can R (esp Unix R) employ multiple CPU cores on a single computer, sharing the same memory space, without resorting to copying full OS processes, so that
there is minimal process overhead; and
modification of global data by one CPU are immediately available to other CPUs?
If yes, can R lock some memory just like files (flock)?
I suspect that the answer is no and learning this definitively would be very useful. If the answer is yes, please point me the right way.
regards,
/iaw
You can use the Rdsm package to use distributed shared memory parallelism, i.e. multiple R processes using the same memory space.
Besides that, you can employ multi-threaded BLAS/LAPACK (e.g. OpenBLAS or Intel MKL) and you can use C/C++ (and probably Fortran) code together with OpenMP. See assembling a matrix from diagonal slices with mclapply or %dopar%, like Matrix::bandSparse for an example.
Have you take a look at Microsoft's R Open (available for Linux), with the custom Math Kernel Library (MKL).
I've seen very good performance improvements without rewriting code.
https://mran.microsoft.com/documents/rro/multithread
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I have a file of approximately 40Gb and i need to perform some analytics activity on it but my ram is limited to 4gb any suggestions how to proceed?
Check out the LaF package. It allows you to do column selection and filtering on files that would not otherwise fit into memory.
Documentation and examples are a little thin on the ground which is a shame given that the syntax is a little idiosyncratic. This may help:
https://stackoverflow.com/a/24716798/1427069
Increase Your Virtual Memory:
increase the size of the paging file
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 years ago.
Improve this question
I've played around with making web-scrapers in the past. I've used them to map all links out of a website and all pages in a website. However I realise 'in theory' I could release this system and let it follow link after link and map the entire internet. Of course the millions of pages that exist on the internet would take a lot longer to scrap than the small collection I'm used to... but how long?
On a typical machine, lets say 8GB Ram, i7 3Ghz, how many links would a machine be able to trawl through?
Map the entire internet? Not on your typical machine. The internet doubles in size about every five years so if your process takes more than five years it is never going to get to the end. #jdphenix calculates 79 years for the size of the internet as it is today so by the time your process ends you will have mapped approximately 0.003% of the internet as it will then be.
So how long on a typical machine? Infinite time. This is not a battle you can win without a serious amount of computing power.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
Is there a way to remove some uneeded locale to reduce the size of Qt Core ?
You'll need to be more specific about what your application requires. Regardless, I'd recommend reading through this thread on the interest mailing list, as it has some interesting information regarding slimming Qt Core. In particular, you can reduce the size of ICU:
I'll leave it for others to pass comment on the standard configure
options and size, but if you're really desperate for every last saving
then removing the locales you don't need can save you 230 KB (on Linux
64bit it reduces my default release build from 5.5MB to 5.2MB), but
it's a manual process:
Download http://unicode.org/Public/cldr/24/core.zip and unzip
Run "../path/to/qt5/qtbase/util/local_database/cldr2qlocalexml.py
core/common/main >> qlocale.xml"
Edit qlocale.xml to remove all the locales you don't need: only
remove groups from inside and nothing else, I
suggest you always keep C and en_US in addition to the locales you
require.
Run "../path/to/qt5/qtbase/util/local_database/qlocalexml2cpp.py
qlocale.xml ../path/to/qt5/qtbase/"
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
I think I set up a TrueCrypt volume on my Win7 machine a few months ago. Now I cannot remember where I set it up or what the password might be. Is there any way that I can detect this volume? I would have maybe gave it a capacity of 2GB or thereabouts so I don't want that storage going to waste.
If I uninstall TrueCrypt, would that release the volume assigned to it?
GF
Try a disk utility that allows you to locate large files or sort files by size (such as windirstat, although there are many others). A file that's 2GB should stand out.
Search your disk for a file with extension tc (*.tc).
There is no volume assigned unless you assign one (a drive letter). So you can uninstall freely.