I am trying to read multple nc4 files in r. below is the code I am using to execute this task;
library(ncdf4)
OSND_gpmr.df<-NULL
GPM_R.files= list.files(path,pattern='*.nc4',full.names=TRUE)
for(i in seq_along(GPM_R.files)) {
nc_data = nc_open(GPM_R.files[i])
GPM_Prec<-ncvar_get(nc_data, 'IRprecipitation')
x=dim(GPM_Prec)
### note start=c(42,28) are the index in image regards to real coordinates of interset
## R reads images from lat,long.
OSND_gpmr.spec =ncvar_get(nc_data, 'IRprecipitation', start = c(42,28), count = c(1,1))
rbind(OSND_gpmr.df,data.frame(OSND_gpmr.spec))->OSND_gpmr.df
nc_close(nc_data)
}
but I consistently get this error:
Error in R_nc4_open: No such file or directory.
But the list of files is correctly recognised as chr [1:1440] as shown in the global environments-Values.
Can someone please help me with what I am doing wrong?
Your working directory might have been different from the files location. Your GPM_R.files list stores only the file names from the given location without file paths. While nc_open() expects filenames with complete path.
Related
I am new to coding and very new to this forum, so I hope my request makes sense.
I am trying to select images listed in a .csv file and to copy them to a new folder. The pictures and the .csv file are both in the folder GRA04. The .csv file contain only one column with the picture names.
I used the following code:
#set working directory
setwd("E:/2019/GRA04")
#create and identify a new folder in R
targetdir <- dir.create("GRA04_age")<br/>
#find the files you want to copy
filestocopy <- read.csv("age.csv", header=FALSE) #read csv as data table (only one column, each raw being a file name)
filestocopy_v <- readLines(filestocopy)#convert data table in character vector
filestocopy_v #shows the character vector
#copy the files to the new folder
file.copy(filestocopy_v, targetdir, recursive = TRUE)
When reaching the line
filestocopy_v <- readLines(filestocopy)
I get this error message:
Error in readLines(filestocopy) : 'con' is not a connection
I looked online for solutions with no luck. I ran this code before (or else something similar... didn't back it up...) and it worked fine, so I am not sure what is happening...
Thanks!
Out of interest, would the following now do what you're trying to achieve?
filestocopy_v <- filestocopy[[1]]
I am trying to deal with extracting a subset from multiple .grb2 files in the same file path, and write them in separate csv files. I am using the following code which does the job and stores the csv files in the same directory as the .grb2 files.
path <- "file path"
input.file.names <- dir(path, pattern =".grb2")
output.file.names <-
paste0(tools::file_path_sans_ext(input.file.names),".csv")
for(i in 1:length(input.file.names)){
GRIB <- brick(input.file.names[i])
GRIB <- as.array(GRIB)
tmp2m.6hr <- GRIB[46,13,c(1:20)]
str(tmp2m.6hr)
tmp2m.data <- data.frame(tmp2m.6hr)
write.csv(tmp2m.data,output.file.names[i])
}
My first question is this: how can I store the csv files in a different directory than the .grb2 files?
My .grb2 files, and thus the resulting csv files, end in four different types, i.e. 00.grb2, 06.grb2, 12.grb2, 18.grb2. The resulting csv files have the following form:
enter image description here
My second question is: how can I merge all my 00.csv, 06.csv, 12.csv, 18.csv files (each category in the same column) in a single csv file in a directory of my choice with the following headrs: 00_tmp2m.6hr, 06_tmp2m.6hr, 12_tmp2m.6hr, 18_tmp2m.6hr, and also create a fifth column with the average of the other four? The result that I want is the following:
enter image description here
As I m not an experienced user this is too complicated for me. I would very much apreciate any assistance with this.
For your fist question, you might try specifying the path using a relative reference to the folder, as in write.csv(paste0("./myfolder/", output.file.names[i])).
Your second question might be easier if you read the data and then write your results as a new file. you might also want to take a look at the optional parameters of write.csv(append = FALSE, ...).
Also, you might get a better answer by creating a minimal example.
Thank you in advance for your're help. I am using R to analyse some data that is initially created in Matlab. I am using the package "R.Matlab" and it is fantastic for 1 file, but I am struggling to import multiple files.
The working script for a single file is as follows...
install.packages("R.matlab")
library(R.matlab)
x<-("folder_of_files")
path <- system.file("/home/ashley/Desktop/Save/2D Stream", package="R.matlab")
pathname <- file.path(x, "Test0000.mat")
data1 <- readMat(pathname)
And this works fantastic. The format of my files is 'Name_0000.mat' where between files the name is a constant and the 4 digits increase, but not necesserally by 1.
My attempt to load multiple files at once was along these lines...
for (i in 1:length(temp))
data1<-list()
{data1[[i]] <- readMat((get(paste(temp[i]))))}
And also in multiple other ways that included and excluded path and pathname from the loop, all of which give me the same error:
Error in get(paste(temp[i])) :
object 'Test0825.mat' not found
Where 0825 is my final file name. If you change the length of the loop it is always just the name of the final one.
I think the issue is that when it pastes the name it looks for that object, which as of yet does not exist so I need to have the pasted text in speach marks, yet I dont know how to do that.
Sorry this was such a long post....Many thanks
I'm trying to design a program to help me convert 1000+ DEM file into USGS raster file, using the method "arcpy.DEMtoRaster_Conversion" in ArcGIS. My idea is to use a OpenFileDialog to allow multiple selection for these files, then use an array to same these names and use these names as the inDEM and save the outRaster in tif format.
file_path = tkFileDialog.askopenfilename(filetypes=(("DEM", "*.dem"),),multiple=1)
this is how I open multiple files in the dialog, but I;m not sure how to save them so as to fulfill the following steps. Can someone help me?
This code will find all dems in a folder and apply the conversion function and save the output tiffs to another folder
#START USER INPUT
datadir="Y:/input_rasters/" #directory where dem files are located
outputdir="Y:/output_rasters/" #existing directory where output tifs are to be saved in
#END USER INPUT
import os
arcpy.env.overwriteOutput = True
arcpy.env.workspace = datadir
arcpy.env.compression = "LZW"
DEMList = arcpy.ListFiles("*.dem")
for f in DEMList:
print "starting %s" %(f)
rastername=os.path.join(datadir, f)
outrastername=os.path.join(outputdir, f[:-4]+".tif")
arcpy.DEMToRaster_conversion(rastername, outrastername)
So I have a single instance of dta to csv conversion, and I need to repeat it for all files in a directory. Great help on SO, but I'm still not quite there. Here's the single instance
#Load Foreign Library
library(foreign)
## Set working directory in which dtw files can be found)
setwd("~/Desktop")
## Single File Convert
write.csv(read.dta("example.dta"), file = "example.csv")
From here, I figure I use something like:
## Get list of all the files
file_list<-dir(pattern = ".dta$", recursive=F, ignore.case = T)
## Get the number of files
n <- length(file_list)
## Loop through each file
for(i in 1:n) file_list[[i]]
But I'm not sure of the proper syntax, expressions, etc. After reviewing the great solutions below, I'm just confused (not necessarily getting errors) and about to do it manually -- quick tips for an elegant way to go through each file in a directory and convert it?
Answers reviewed include:
Convert Stata .dta file to CSV without Stata software
applying R script prepared for single file to multiple files in the directory
Reading multiple files from a directory, R
THANKS!!
Got the answer: Here's the final code:
## CONVERT ALL FILES IN A DIRECTORY
## Load Foreign Library
library(foreign)
## Set working directory in which dtw files can be found)
setwd("~/Desktop")
## Convert all files in wd from DTA to CSV
### Note: alter the write/read functions for different file types. dta->csv used in this specific example
for (f in Sys.glob('*.dta'))
write.csv(read.dta(f), file = gsub('dta$', 'csv', f))
If the files are in your current working directory, one way would be to use Sys.glob to get the names, then loop over this vector.
for (f in Sys.glob('*.dta'))
write.csv(read.dta(f), file = gsub('dta$', 'csv', f))