The variable from a netcdf file comes out flipped - r

I have downloaded a nc file from
f=open.ncdf("file.nc")
[1] "file Lfile.nc has 2 dimensions:"
[1] "Longitude Size: 1440"
[1] "Latitude Size: 720"
[1] "------------------------"
[1] "file filr.nc has 8 variables:"
[1] "short ts[Latitude,Longitude] Longname:Skin Temperature (2mm) Missval:NA"
I then wanted to work with the variable soil_moisture_c
A = get.var.ncdf(nc=f,varid="soil_moisture_c",verbose=TRUE)
I then plot A with image(A). I got the map shown below,I even transposed it image(t(a)) but that was changed to other direction,and not how it should be. Anyway,in order to know what is wrong,I used the netcdf viewer Panoply and the map was correctly plotted as you can see below.

The reason is that the NetCDF interface you are using is very low-level, and all you have done is read out the variable without any of its dimension information. The orientation of the grid is really arbitrary, and the coordinate information needs to be understood in a particular context.
library(raster) ## requires ncdf package for this file
d <- raster("LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T185959Z_20040114.nc", varname = "soil_moisture_c")
(I used a different file to yours, but it should work the same).
It turns out that even raster does not get this right without work, but it does make it easy to rectify:
d <- flip(t(d), direction = "x")
That transposed the data and flipped around "x" (longitude), keeping the georeferencing from the original context.
Plot it up with a map from maptools to check:
plot(d)
library(maptools)
data(wrld_simpl)
plot(wrld_simpl, add = TRUE)
There are many other ways to achieve this by reading the dimension information from the file, but this is at least a shortcut to do most of the hard work for you.

Just as a complement to #mdsumner far better solution, here is a way to do that using library ncdf only.
library(ncdf)
f <- open.ncdf("LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040101.nc")
A <- get.var.ncdf(nf,"soil_moisture_c")
All you need is to find your dimensions in order to have a coherent x and y-axis. If you look at your netCDF objects dimensions, here what you see:
str(f$dim)
List of 2
$ Longitude:List of 8
..$ name : chr "Longitude"
..$ len : int 1440
..$ unlim : logi FALSE
..$ id : int 1
..$ dimvarid : num 2
..$ units : chr "degrees_east"
..$ vals : num [1:1440(1d)] -180 -180 -179 -179 -179 ...
..$ create_dimvar: logi TRUE
..- attr(*, "class")= chr "dim.ncdf"
$ Latitude :List of 8
..$ name : chr "Latitude"
..$ len : int 720
..$ unlim : logi FALSE
..$ id : int 2
..$ dimvarid : num 1
..$ units : chr "degrees_north"
..$ vals : num [1:720(1d)] 89.9 89.6 89.4 89.1 88.9 ...
..$ create_dimvar: logi TRUE
..- attr(*, "class")= chr "dim.ncdf"
Hence your dimensions are:
f$dim$Longitude$vals -> Longitude
f$dim$Latitude$vals -> Latitude
Now your Latitude goes from 90 to -90 instead of the opposite, which image prefers, so:
Latitude <- rev(Latitude)
A <- A[nrow(A):1,]
Finally, as you noticed, the x and y of your object A are flipped so you need to transpose it, and your NA values are represented for some reasons by value -32767:
A[A==-32767] <- NA
A <- t(A)
And finally the plot:
image(Longitude, Latitude, A)
library(maptools)
data(wrld_simpl)
plot(wrld_simpl, add = TRUE)
Edit: To do that on your 31 files, let's call your vector of file names ncfiles and yourpath the directory where you stored them (for simplicity i'm going to assume your variable is always called soil_moisture_c and your NAs are always -32767):
ncfiles
[1] "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040101.nc" "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040102.nc"
[3] "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040103.nc" "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040104.nc"
[5] "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040105.nc" "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040106.nc"
[7] "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040107.nc" "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040108.nc"
[9] "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040109.nc" "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040110.nc"
[11] "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040111.nc" "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040112.nc"
[13] "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040113.nc" "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040114.nc"
[15] "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040115.nc" "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040116.nc"
[17] "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040117.nc" "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040118.nc"
[19] "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040119.nc" "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040120.nc"
[21] "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040121.nc" "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040122.nc"
[23] "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040123.nc" "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040124.nc"
[25] "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040125.nc" "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040126.nc"
[27] "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040127.nc" "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040128.nc"
[29] "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040129.nc" "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040130.nc"
[31] "LPRM-AMSR_E_L3_D_SOILM3_V002-20120520T173800Z_20040131.nc"
yourpath
[1] "C:\\Users"
library(ncdf)
library(maptools)
data(wrld_simpl)
for(i in 1:length(ncfiles)){
f <- open.ncdf(paste(yourpath,ncfiles[i], sep="\\"))
A <- get.var.ncdf(f,"soil_moisture_c")
f$dim$Longitude$vals -> Longitude
f$dim$Latitude$vals -> Latitude
Latitude <- rev(Latitude)
A <- A[nrow(A):1,]
A[A==-32767] <- NA
A <- t(A)
close.ncdf(f) # this is the important part
png(paste(gsub("\\.nc","",ncfiles[i]), "\\.png", sep="")) # or any other device such as pdf, jpg...
image(Longitude, Latitude, A)
plot(wrld_simpl, add = TRUE)
dev.off()
}

You can also simply invert the latitudes from the command line using CDO:
cdo invertlat file.nc file_inverted.nc

Related

R Convert column to spatial objects - Polygon

I have some polygon data as an csv file with this format:
"ID","name","geometry"
"1","A","list(c(-0.25291548373789, -0.251828087559847, -0.251065989114876, -0.24944850900495, 51.4652163826438, 51.4652280432439, 51.4652040920587, 51.4650720752044))"
"2","B","list(c(-0.26291548373789, -0.211828087559847, -0.231065989114876, -0.26944850900495, 51.1652163826438, 51.2652280432439, 51.3652040920587, 51.5650720752044))"
And, when I import it to Rstudio looks like this:
Which is what I'm looking for, but the type of data is not what I need:
$ ID : num [1:2] 1 2
$ name : chr [1:2] "A" "B"
$ geometry: chr [1:2] ""list(c(-0.25291548373789, -0.2518280875598..."
What I need is this geometry column to an Spatial Object as Polygon, to be able to use it on Leaflet as:
addPolygons(data=test$geometry)
Any ideas how to achieve this?
Thanks!
test <- st_as_sf(test, wkt = 'geometry')

Convert sf object to dataframe and restore it to original state

I'd like to convert a sf object to a dataframe and restore it to its original state. But when I make the conversion of st_as_text(st_sfc(stands_sel$geometry)) is shows very difficult to retrieve it again. In my example:
library(sf)
# get AOI in shapefile
download.file(
"https://github.com/Leprechault/trash/raw/main/sel_stands_CMPC.zip",
zip_path <- tempfile(fileext = ".zip")
)
unzip(zip_path, exdir = tempdir())
# Open the file
setwd(tempdir())
stands_sel <- st_read("sel_stands_CMPC.shp")
st_crs(stands_sel) = 4326
# Extract geometry as text
geom <- st_as_text(st_sfc(stands_sel$geometry))
# Add the features
features <- st_drop_geometry(stands_sel)
str(features)
# Joining feature + geom
geo_df <- cbind(features, geom)
str(geo_df)
# 'data.frame': 2 obs. of 17 variables:
# $ CD_USO_SOL: num 2433 9053
# $ ID_REGIAO : num 11 11
# $ ID_PROJETO: chr "002" "344"
# $ PROJETO : chr "BARBA NEGRA" "CAMPO SECO"
# $ CD_TALHAO : chr "159A" "016A"
# $ CARACTERIS: chr "Plantio Comercial" "Plantio Comercial"
# $ CARACTER_1: chr "Produtivo" "Produtivo"
# $ CICLO : int 2 1
# $ ROTACAO : int 1 1
# $ DATA_PLANT: chr "2008/04/15" "2010/04/15"
# $ LOCALIDADE: chr "BARRA DO RIBEIRO" "DOM FELICIANO"
# $ ESPACAMENT: chr "3.00 x 2.50" "3.5 x 2.14"
# $ ESPECIE : chr "SALIGNA" "DUNNI"
# $ SISTEMA_PR: chr "MACRO ESTACA - EUCALIPTO" "SEMENTE - EUCALIPTO"
# $ VLR_AREA : num 8.53 28.07
# $ ID_UNIQUE : chr "BARBANEGRA159A" "CAMPOSECO016A"
# $ geom : chr "MULTIPOLYGON (((-51.21423 -30.35172, -51.21426 -30.35178, -51.2143 -30.35181, -51.21432 -30.35186, -51.21433 -3"| __truncated__
# Return to original format again
stands_sf <- geo_df %>%
st_geometry(geom) %>%
sf::st_as_sf(crs = 4326)
#Error in UseMethod("st_geometry") :
Please, any help to restore my stands_sf object to the orinal state?
I think geom isn't in a format st_geometry is expecting. st_as_text converted your geometry into WKT as discussed in the help:
The returned WKT representation of simple feature geometry conforms to the simple features access specification and extensions, known as EWKT, supported by PostGIS and other simple features implementations for addition of SRID to a WKT string.
https://r-spatial.github.io/sf/reference/st_as_text.html
Instead, use st_as_sf(wkt=) to set the new (old) geometry.
st_as_sf(geo_df, wkt = "geom", crs = 4326)

shapefile in R: aggregate geometries by similar attributes

I have a shapefile in SpatialPolygonsDataFrame (myshape) format with several sub-divisions represents by A, B, C, etc. of the same area eg. LIMOEIRO064A, LIMOEIRO064B, etc. for several different areas. I'd like to merge the geometries of LIMOEIRO064 for example. For this I try:
# Packages
library(raster)
library(rgdal)
# Download and unzip the shapefile example
download.file('https://www.dropbox.com/s/2zoproayzqvj1cc/myshape.zip?dl=0',
destfile="myshape.zip",
method="auto")
unzip(paste0(getwd(),"/myshape.zip"))
#Read target shapefile -----------------------------------------------
myshape <- readOGR (".","myshape")
proj4string(myshape) <- CRS("+proj=longlat +ellps=GRS80 +no_defs")
# Create unique ID for each area without sub-units A, B, C, etc. if have in CD_TALHAO attribute
str(myshape#data)
#'data.frame': 419 obs. of 7 variables:
# $ OBJECTID : chr "563774" "563783" "795091" "795092" ...
# $ ID_PROJETO: chr "131" "131" "131" "131" ...
# $ PROJETO : chr "LIMOEIRO" "LIMOEIRO" "LIMOEIRO" "LIMOEIRO" ...
# $ CD_TALHAO : chr "064A" "017B" "V00204" "V00702" ...
# $ SHAPE_AREA: num 1.07e-05 1.67e-05 1.72e-07 2.46e-07 2.07e-06 ...
# $ SHAPE_LEN : num 0.02774 0.01921 0.00401 0.005 0.01916 ...
# $ CODE : chr "LIMOEIRO064A" "LIMOEIRO017B" "LIMOEIROV00204" "LIMOEIROV00702" ...
myshape#data$UNIQUE<-gsub("[a-zA-Z]", "", myshape#data$CD_TALHAO)
# New unique CODE
myshape#data$CODE<-paste0(myshape#data$PROJETO,myshape#data$UNIQUE)
#unique(myshape#data$CODE)
# [1] "LIMOEIRO064" "LIMOEIRO017" "LIMOEIRO00204" "LIMOEIRO00702" "LIMOEIRO06501" "LIMOEIRO02403"
# [7] "LIMOEIRO00201" "LIMOEIRO05002" "LIMOEIRO03516" "LIMOEIRO02203" "LIMOEIRO02904" "LIMOEIRO00405"
# [13] "LIMOEIRO01804" "LIMOEIRO01608" "LIMOEIRO03106" "LIMOEIRO00101" "LIMOEIRO010" "LIMOEIRO035"
# [19] "LIMOEIRO020" "LIMOEIRO001" "LIMOEIRO056" "LIMOEIRO059" "LIMOEIRO06402" "LIMOEIRO01801"
#...
# [295] "LIMOEIRO011" "LIMOEIRO06408"
Now, I'd like to merge shapefiles and geometries with the same CODE identification in a new_myshape but I find options like bind() and union() not work for me. I need something like aggregate by myshape#data$CODE or some option like this.
Please any ideas?
Here is how you can do that with terra
library(terra)
f <- system.file("ex/lux.shp", package="terra")
v <- vect(f)
va <- aggregate(v, "ID_1")
You can use the same approach with raster/sp
p <- shapefile(system.file("external/lux.shp", package="raster"))
pa <- aggregate(p, "ID_1")

List aviable WFS layers and read into data frame with rgdal

I have the following problem according to different sources it should be able to read WFS layer in R using rgdal.
dsn<-"WFS:http://geomap.reteunitaria.piemonte.it/ws/gsareprot/rp-01/areeprotwfs/wfs_gsareprot_1?service=WFS&request=getCapabilities"
ogrListLayers(dsn)
readOGR(dsn,"SIC")
The result of that code should be 1) to list the available WFS layer and 2) to read a specific Layer (SIC) into R as a Spatial(Points)DataFrame.
I tried several other WFS server but it does not work.
I always get the warning:
Cannot open data source
Checking for the WFS driver i get the following result:
> "WFS" %in% ogrDrivers()$name
[1] FALSE
Well it looks like the WFS driver is not implemented in rgdal (anymore?)
Or why are there so many examples "claiming" the opposite?
I also tried the gdalUtils package and well it works but It gives out the whole console message of ogrinfo.exe and not only the available layers.(I guess it "just" calls the ogrinfo.exe and sends the result back to R like using the r shell or system command).
Well does anyone know what I´m making wrong, or if something like that is even possible with rgdal or any similar package?
You can combine the two packages to accomplish your task.
First, convert the layer you need into a local shapefile using gdalUtils. Then, use rgdal as normal. NOTE: you'll see a warning message after the ogr2ogr call but it performed the conversion fine for me. Also, ogr2ogr won't overwrite local files without the overwrite parameter being TRUE (there are other parameters that may be of use as well).
library(gdalUtils)
library(rgdal)
dsn <- "WFS:http://geomap.reteunitaria.piemonte.it/ws/gsareprot/rp-01/areeprotwfs/wfs_gsareprot_1?service=WFS&request=getCapabilities"
ogrinfo(dsn, so=TRUE)
## [1] "Had to open data source read only."
## [2] "INFO: Open of `WFS:http://geomap.reteunitaria.piemonte.it/ws/gsareprot/rp-01/areeprotwfs/wfs_gsareprot_1?service=WFS&request=getCapabilities'"
## [3] " using driver `WFS' successful."
## [4] "1: AreeProtette"
## [5] "2: ZPS"
## [6] "3: SIC"
ogr2ogr(dsn, "sic.shp", "SIC")
sic <- readOGR("sic.shp", "sic", stringsAsFactors=FALSE)
## OGR data source with driver: ESRI Shapefile
## Source: "sic.shp", layer: "sic"
## with 128 features
## It has 23 fields
plot(sic)
str(sic#data)
## 'data.frame': 128 obs. of 23 variables:
## $ gml_id : chr "SIC.510" "SIC.472" "SIC.470" "SIC.508" ...
## $ objectid : chr "510" "472" "470" "508" ...
## $ inspire_id: chr NA NA NA NA ...
## $ codice : chr "IT1160026" "IT1160017" "IT1160018" "IT1160020" ...
## $ nome : chr "Faggete di Pamparato, Tana del Forno, Grotta delle Turbiglie e Grotte di Bossea" "Stazione di Linum narbonense" "Sorgenti del T.te Maira, Bosco di Saretto, Rocca Provenzale" "Bosco di Bagnasco" ...
## $ cod_tipo : chr "B" "B" "B" "B" ...
## $ tipo : chr "SIC" "SIC" "SIC" "SIC" ...
## $ cod_reg_bi: chr "1" "1" "1" "1" ...
## $ des_reg_bi: chr "Alpina" "Alpina" "Alpina" "Alpina" ...
## $ mese_istit: chr "11" "11" "11" "11" ...
## $ anno_istit: chr "1996" "1996" "1996" "1996" ...
## $ mese_ultmo: chr "2" NA NA NA ...
## $ anno_ultmo: chr "2002" NA NA NA ...
## $ sup_sito : chr "29396102.9972" "82819.1127" "7272687.002" "3797600.3563" ...
## $ perim_sito: chr "29261.8758" "1227.8846" "17650.289" "9081.4963" ...
## $ url1 : chr "http://gis.csi.it/parchi/schede/IT1160026.pdf" "http://gis.csi.it/parchi/schede/IT1160017.pdf" "http://gis.csi.it/parchi/schede/IT1160018.pdf" "http://gis.csi.it/parchi/schede/IT1160020.pdf" ...
## $ url2 : chr "http://gis.csi.it/parchi/carte/IT1160026.djvu" "http://gis.csi.it/parchi/carte/IT1160017.djvu" "http://gis.csi.it/parchi/carte/IT1160018.djvu" "http://gis.csi.it/parchi/carte/IT1160020.djvu" ...
## $ fk_ente : chr NA NA NA NA ...
## $ nome_ente : chr NA NA NA NA ...
## $ url3 : chr NA NA NA NA ...
## $ url4 : chr NA NA NA NA ...
## $ tipo_geome: chr "poligono" "poligono" "poligono" "poligono" ...
## $ schema : chr "Natura2000" "Natura2000" "Natura2000" "Natura2000" ...
Neither the questioner nor the answerer say how rgdal was installed. If it is a CRAN binary for Windows or OSX, it may well have a smaller set of drivers than an independent installation of GDAL underlying gdalUtils. Always state your platform, and whether rgdal was installed binary or from source, and always provide the output of the messages displayed as rgdal loads, as well as of sessionInfo() to show the platform on which you are running.
Given the possible difference in sets of drivers, the advice given seems reasonable.

R SpatialPointsDataFrame to SpatialLinesDataFrame

I've imported some GPS points from my Sports watch into R:
library(plotKML)
route <- readGPX("Move_Cycling.gpx")
str(route)
The data looks like this:
List of 5
$ metadata : NULL
$ bounds : NULL
$ waypoints: NULL
$ tracks :List of 1
..$ :List of 1
.. ..$ Move:'data.frame': 677 obs. of 5 variables:
.. .. ..$ lon : num [1:677] -3.8 -3.8 -3.8 -3.8 -3.8 ...
.. .. ..$ lat : num [1:677] 52.1 52.1 52.1 52.1 52.1 ...
.. .. ..$ ele : chr [1:677] "152" "151" "153" "153" ...
.. .. ..$ time : chr [1:677] "2014-06-08T09:17:08.050Z" "2014-06-08T09:17:18.680Z" "2014-06-08T09:17:23.680Z" "2014-06-08T09:17:29.680Z" ...
.. .. ..$ extensions: chr [1:677] "7627.7999992370605141521101800" "7427.6000003814697141511.7000000476837210180.8490009442642210" "9127.523.13003521531.7000000476837210181.799999952316280" "10027.534.96003841534.1999998092651410181.88300029210510" ...
$ routes : NULL
I've managed to transform to get the data points into a SpatialPointsDataFrame and to plot it over Google Earth with:
SPDF <- SpatialPointsDataFrame(coords=route$tracks[[1]]$Move[1:2],
data=route$tracks[[1]]$Move[1:2],
proj4string = CRS("+init=epsg:4326"))
plotKML(SPDF)
What I really want is the cycling track, i.e. a SpatialLinesDataFrame, but I can't work out how to set the ID field correctly to match the SpatialLines object with the data.
This is how far I've got:
tmp <- Line(coords=route$tracks[[1]]$Move[1:2])
tmp2 <- Lines(list(tmp), ID=c("coord"))
tmp3 <- SpatialLines(list(tmp2), proj4string = CRS("+init=epsg:4326"))
# result should be something like,
# but the ID of tmp3 and data don't match at the moment
SPDF <- SpatialLinesDataFrame(tmp3, data)
You can read the GPX file straight into a SpatialLinesDataFrame object with readOGR from the rgdal package. A GPX file can contain tracks, waypoints, etc and these are seen by OGR as layers in the file. So simply:
> track = readOGR("myfile.gpx","tracks")
> plot(track)
should work. You should see lines.
In your last line you've not said what your data is, but it needs to be a data frame with one row per track if you are trying to construct a SpatialLinesDataFrame from some SpatialLines and a data frame, and you can tell it not to bother matching the IDs because you don't actually have any real per-track data you are merging. So:
> SPDF = SpatialLinesDataFrame(tmp3, data.frame(who="me"),match=FALSE)
> plot(SPDF)
But if you use readOGR you don't need to go through all that. It will also read in a bit of per-track metadata from the GPX file.
Happy cycling!
As an update, here's my final solution
library(rgdal)
library(plotKML)
track <- readOGR("Move_Cycling.gpx","tracks")
plotKML(track, colour='red', width=2, labels="Cwm Rhaeadr Trail")

Resources