Is there a quick and easy way to take an existing RData file and convert it to netCDF? I have a large .RData file (~780 MB) with 88 variables, and all I can find online is how to make a netCDF file from scratch with simple examples that don't have a lot of variables. I'm trying to loop the 88 variables, but am getting errors because of the complexity. Is there a function that exists to do this in R?
Thank you.
Related
I've a lot of .hyper files to work with. Most of the time I work with them using Python (using tableauhyperio lib), but I need to read them in R and I could not find a way to do it. Does anyone know some way to read hyper files in R?
Right now I'm reading the data in python and exporting them as csv to read in R the csv files...
I'm trying to unzip a 7zip file with the unzip function. The zip file contains around 155 000 csv files inside. When it starts unzipping the first 25 000 csv files are read within 5 minutes and then it starts to slow down and reads all files in almost an hour. Is this common behavior.
I know not providing a reproducible example makes everyones life more difficult but I cannot share the data with which I'm working. I'm just interested whether there is another way to extract files from zip with r, tweak the function or maybe call outside program or something?
Extracting with 7zip outside of R takes 20 mins tops, which is why I believe this to be an R related issue
I have a sas7bdat file of size around 80 GB. Since my pc has a memory of 4 GB the only way I can see is reading some of its rows. I tried using the sas7bdat package in R which gives the error "big endian files are not supported"
The read_sas() function in haven seems to work but the function supports reading specific columns only while I need to read any subset of rows with all columns. For example, it will be fine if I can read 1% of the data to understand it.
Is there any way to do this? Any package which can work?
Later on I plan to read parts of the file and divide it into 100 or so sections
If you have Windows you can use the SAS Universal Viewer, which is free, and export the dataset to CSV. Then you can import the CSV into R in more readable chunks using this method.
I have about ~1000 tar.gz files (about 2 GB/file compressed) each containing bunch of large .tsv (tab separated) files e.g. 1.tsv, 2.tsv, 3.tsv, 4.tsv etc.
I want to work in R on a subset of the .tsv files (say 1.tsv, 2.tsv) without extracting the .tar.gz files, in order to save space/time.
I tried looking around but couldn't find a library or a routine to stream the tar.gz files through memory and extracting data from them on the fly. In other languages there are ways of doing this efficiently. I would be surprised if one couldn't do this in R
Does anyone know of a way to accomplish this in R? Any help is greatly appreciated! Note: Unzipping/untarring the file is not an option. I want to extract relevant fields and save them in a data.frame without extracting the file
For my processes in R I want to read in a 20 gigabyte file. I got it in a XML file type.
In R I cannot load it in with readOGR since it is to big. It gives me the error cannot allocate vector 99.8 mb.
Since my file is to big the logical next step in my mind would be to split the file. But since I can not open it in R and any other GIS package at hand, I can not split the file before I load it in. I am already using the best PC to my availability.
Is there a solution?
UPDATE BECAUSE OF COMMENT
If I use head() my line looks like underneath. It does not work unfortunately.
headfive <- head(readOGR('file.xml', layer = 'layername'),5)