I am new to R and NetCDF files. I am trying to open data on surface sea temperatures that are in a .nc file from here. My code is as follows:
rm(list=ls())
#install.packages(c("netcdf", "chron", "RColorBrewer", "lattice"))
library(chron)
library(RColorBrewer)
library(lattice)
library(ncdf4)
# set path and filename
ncpath <- "C:/Users/Mihir Sharma/Dropbox/data/"
ncname <- "20140101023458-NCEI-L3C_GHRSST-SSTskin-AVHRR_Pathfinder-PFV5.3_NOAA19_G_2014001_night-v02.0-fv01.0"
ncfname <- paste(ncpath, ncname, ".nc", sep="")
dname <- "tmp" # note: tmp means temperature (not temporary)
# open a NetCDF file
ncin <- nc_open(ncfname)
But I am getting the following error:
Error in nc_open(ncfname) :
Error in nc_open trying to open file C:/Users/Mihir Sharma/Dropbox/1 EPIC/MPA/data/20140101023458-NCEI-L3C_GHRSST-SSTskin-AVHRR_Pathfinder-PFV5.3_NOAA19_G_2014001_night-v02.0-fv01.0.nc
I have followed the code from here and here. what am I doing wrong?
Many thanks,
Mihir
Path problems when using someone else's code:
# set path and filename
ncpath <- "C:/Users/Mihir Sharma/Dropbox/data/"
When you're following code on a blog or tutorial, if it's written well, they'll use a platform independent way to describe, but often they won't.
The platform independent way to write a path is:
file.path("Users", "Mihir Sharma", "Dropbox", "data"). This will always use the correct file separator for your Platform, a value stored in .Platform$file.sep
Related
I need to read a XLS file in R, but I'm having a problem regarding the way my file is generated and the R function readxl. I do not have this issue with Python, and this is my hope that it's possible to solve this problem inside R.
An application we use at my company exports reports in XLS format (not XLSX). This report is generated daily. What I need is to sum the total value of the rows in each file, in order to create a new report containing each day followed by this total value.
When I try to read these files in R using the readxl package, the program returns this error:
Erro: Can't subset columns that don't exist.
x Location 5 doesn't exist.
i There are only 0 columns.
Run rlang::last_error() to see where the error occurred.
Now, the weird thing is that, when I open the XLS file on Excel before running my script, R is able to run properly.
I guesses this was an error caused by something like the file only being completed when I open it... but the same python script does give me the correct result.
I am now assuming this is a bug in the readxl package. Is there another package I could use to run XLS (and not XLSX)? One that does not depend on Java installed on my computer, I mean.
my readxl script:
if (!require("readxl")) {install.packages("readxl"); library("readxl")}
"%,%" <- function(x,y) paste0(x,"\\",y)
year = "2021"
month = "Aug"
column = 5 # VL_COVAR
path <- "F:\\variancia" %,% year %,% month
tiposDF = c("date","numeric","list","numeric","numeric","numeric","list")
file.names <- dir(path, pattern =".xls")
vari <- c()
for (i in 1:length(file.names)){
file <- paste(path,sep="\\",file.names[i])
print(paste("Reading ", file))
dados <- read_excel(file, col_types = tiposDF)
somaVar <- sum(dados[column])
vari <- append(vari,c(somaVar))
}
vari
file <- paste(path,sep="\\",'Covariância.xls_02082021.xls')
print(paste("Reading ", file))
dados <- read_excel(file, col_types = tiposDF)
somaVar <- sum(dados[column])
vari <- append(vari,c(somaVar))
x <- import(file)
View(x)
Thanks everyone!
I was trying to convert a dataset from netcdf to csv format using R.
although I have installed 'raster and 'netcdf4' in R.
but still it doesn't find nc.brick.
rm(list=ls())
library(raster)
library(ncdf4)
nc.brick <- brick(file.choose)
dim(nc.brick)
nc.df <- as.data.frame(nc.brick[[1]],xy=T)
head(nc.df)
write.csv(nc.df,file.choose())
test <- read.csv(file.choose())
Can you see the difference between your usage of file.choose in these two lines:
nc.brick <- brick(file.choose)
write.csv(nc.df,file.choose())
there's your problem.
I would like to read all shapefiles in a directory into the global environment However, when I get a list of files in my working directory, the list includes both .shp and .shp.xml files, the latter of which I do not want. My draft code reads both .shp and .shp.xml files. How can I prevent it from doing so?
Draft code follows:
library(maptools)
# get all files with the .shp extension from working directory
setwd("N:/Dropbox/_BonesFirst/139_Transit_Metros_Points_Subset_by_R")
shps <- dir(getwd(), "*.shp")
# the assign function will take the string representing shp
# and turn it into a variable which holds the spatial points data
for (shp in shps) {
cat("now loading", shp, "...", '\n\r')
assign(shp, readOGR(shp))
}
EDIT: Problems seems to be in the readShapePoints. Either readOGR (from rgdal) or shapefile (from raster) work better.
Get all the files:
# replace with your folder name:
dir <- "c:/files/shpfiles"
ff <- list.files(dir, pattern="\\.shp$", full.names=TRUE)
Now read them. Easiest with raster::shapefile. Do not use readShapefile (obsolete and incomplete)
library(raster)
# first file
shapefile(ff[1])
# all of them into a list
x <- lapply(ff, shapefile)
These days, you could use "terra" and do
library(terra)
v <- vect( lapply(ff, vect) )
Going by the Title of your post, I believe you want to list all shape files from your current directory.You may want to use below.
list.files(, pattern="*.shp", full.names=TRUE)
link for the .shp file dataset
I want to read a .shp file from R tool, am using the following script:
>scot_LL<-readOGR("C:/Documents and Settings/Admin/Desktop/scotlip/lip cancer/New Scot/scot.shp","scot"),
but when I run the script, am getting the below errors:
>Error in file(paste(DSN, .Platform$file.sep, layer, ".dbf", sep = ""), <br/>:
cannot open the connection<br/>:
In addition: Warning message:<br/>:
In file(paste(DSN, .Platform$file.sep, layer, ".dbf", sep = ""), :<br/>:
cannot open file 'C:/Documents and Settings/Admin/Desktop/scotlip/lip cancer/New Scot/scot.dbf': No such file or directory
Can you please help me.
Here is a solution using the maptools package:
library(maptools)
scot_mp <- readShapeSpatial('C:/Documents and Settings/Admin/Desktop/scotlip/lip cancer/New Scot/scot.shp')
proj4string(scot_mp) <- "+proj=longlat +datum=WGS84" # specify projection
plot(scot_mp)
and the result:
EDIT:
Solution using the rgdal package:
scot_mp <- readOGR('C:/Documents and Settings/Admin/Desktop/scotlip/lip cancer/New Scot', 'scot')
The first argument should not necessary contain the name of shape file, but the main problem seems to be that at least the dbf file is not in your folder, all of these are needed and should be in the same folder: scot.shp, scot.dbf, scot.shx. After that it proceeds as the other solution:
proj4string(scot_mp) <- "+proj=longlat +datum=WGS84"
plot(scot_mp)
I am trying to download and extract a .csv file from a webpage using R.
This question is a duplicate of Using R to download zipped data file, extract, and import data.
I cannot get the solution to work, but it may be due to the web address i am using.
I am trying to download the .csv files from http://data.worldbank.org/country/united-kingdom (under the download data drop down)
Using #Dirk's solution from the link above, i tried
temp <- tempfile()
download.file("http://api.worldbank.org/v2/en/country/gbr?downloadformat=csv",temp)
con <- unz(temp, "gbr_Country_en_csv_v2.csv")
dat <- read.table(con, header=T, skip=2)
unlink(temp)
I got the extended link by looking at the page source code, which I expect is causing the problems, although it works if i paste it into the address bar.
The file downloads with the correct Gb
download.file("http://api.worldbank.org/v2/en/country/gbr?downloadformat=csv",temp)
# trying URL 'http://api.worldbank.org/v2/en/country/gbr?downloadformat=csv'
# Content type 'application/zip' length 332358 bytes (324 Kb)
# opened URL
# downloaded 324 Kb
# also tried unzip but get this warning
con <- unzip(temp, "gbr_Country_en_csv_v2.csv")
# Warning message:
# In unzip(temp, "gbr_Country_en_csv_v2.csv") :
# requested file not found in the zip file
But these are the file names when i manually download them.
I'd appreciate some help with where i am going wrong , thanks
I am using Windows 8, R version 3.1.0
In order to get your data to download and uncompress, you need to set mode="wb"
download.file("...",temp, mode="wb")
unzip(temp, "gbr_Country_en_csv_v2.csv")
dd <- read.table("gbr_Country_en_csv_v2.csv", sep=",",skip=2, header=T)
It looks like the default is "w" which assumes a text files. If it was a plain csv file this would be fine. But since it's compressed, it's a binary file, hence the "wb". Without the "wb" part, you can't open the zip at all.
It's almost everything ok. In this case you only need to specify that it's a comma separated file, eg using sep="," in read.table:
temp <- tempfile()
download.file("http://api.worldbank.org/v2/en/country/gbr?downloadformat=csv",
temp)
con <- unz(temp, "gbr_Country_en_csv_v2.csv")
dat <- read.table(con, header=T, skip=2, sep=",")
unlink(temp)
With this little change i can import your csv smoothly.
HTH, Luca
The Word Bank Developmet Indictors can be obtained using the WDI package. For example,
library(WDI)
inds <- WDIsearch(field = "indicator")[, 1]
GB <- WDI("GB", indicator = inds)
See WDIsearch and WDI functions and the rerference manual for more info.