r heatmap - stat_density2d (ggmap) vs. addHeatmap (shiny leaflet) - r

I made static heatmaps with the library(ggmap) and the stat_density2d() function. Looking to recreate this in a shiny app on a dynamic leaflet map, I found addHeatmap(). However, the resulting images are dissimilar, with the ggmap version seemingly offering the correct result.
GGMAP
LEAFLET
What is causing this difference?
To run both of the below reproducible examples, you can download some data (csv file) I put here.
https://drive.google.com/drive/folders/0B8_GTHBuoKSRR1VIRmhOUTJKYU0?usp=sharing
Note that the leaflet result differs with zoom level, but never matches the ggmap result (e.g. in terms location of maximum heat).
This is the ggmap code.
library(ggmap)
data <- read.csv("DATA.csv", sep=";")
data <- subset(data, !is.na(CrdLatDeg))
xmin <- min(data$CrdLonDeg)
xmax <- max(data$CrdLonDeg)
ymin <- min(data$CrdLatDeg)
ymax <- max(data$CrdLatDeg)
lon <- c(xmin,xmax)
lat <- c(ymin,ymax)
map <- get_map(location = c(lon = mean(lon), lat = mean(lat)), zoom = 17,
maptype = "satellite", source = "google")
ggmap(map) +
labs(x="longitude", y="latitude") +
stat_density2d(data=data, aes(x=CrdLonDeg, y=CrdLatDeg, alpha= ..level.., fill= ..level..), colour=FALSE,
geom="polygon", bins=100) +
scale_fill_gradientn(colours=c(rev(rainbow(100, start=0, end=.7)))) + scale_alpha(range=c(0,.8)) +
guides(alpha=FALSE,fill=FALSE)
This is the leaflet code.
library(leaflet.extras)
data <- read.csv("DATA.csv", sep=";")
data <- subset(data, !is.na(CrdLatDeg))
leaflet(data) %>%
addTiles(group="OSM") %>%
addHeatmap(group="heat", lng=~CrdLonDeg, lat=~CrdLatDeg, max=.6, blur = 60)

The images look different because the algorithms are different.
stat_density2d() extrapolates a probability density function from the discrete data.
Leaflet implementation of heatmaps rely on libraries like simpleheat, heatmap.js or webgl-heatmap. These heatmaps do not rely on probability density. (I'm not fully sure which of these is used by r-leaflet's addHeatmap).
Instead, these heatmaps work by drawing a blurred circle for each point, raising the value of each pixel by an amount directly proportional to the intensity of the point (constant in your case), and inversely proportional to the distance between the point and the circle. Every data point is shown in the heatmap as a circle. You can see this by playing with your mouse cursor in the heatmap.js webpage, or by looking at this lone point in the top-right of your image:
Think of a heatmap like a visualization of the function
f(pixel) = ∑ ( max( 0, radius - distance(pixel, point) ) · intensity(point) )
One can tweak the radius and intensity of heatmaps, but the result will never be the same as a statistical density estimation.

I've found this answer over at GIS, and I've attempted to create a function and applied it to this case. I haven't figured out how to finetune the colour gradient scheme as of yet, but it seems like a good first start otherwise:
library(leaflet)
library(rlang)
addHeatMap <- function(data, lon, lat, intensity, show.legend, ...) {
df <- data.table::as.data.table(data)
df_expanded <- dplyr::slice(df, rep(1:dplyr::n(), times = !! enquo(intensity)))
lon_var <- dplyr::pull(df_expanded, !! enquo(lon))
lat_var <- dplyr::pull(df_expanded, !! enquo(lat))
lon_bw <- MASS::bandwidth.nrd(lon_var)
lat_bw <- MASS::bandwidth.nrd(lat_var)
lon_lat_df <- dplyr::select(df_expanded, !! enquo(lon), !! enquo(lat))
kde <- KernSmooth::bkde2D(lon_lat_df, bandwidth = c(lon_bw, lat_bw))
CL <- contourLines(kde$x1 , kde$x2 , kde$fhat)
LEVS <- as.factor(sapply(CL, `[[`, "level"))
NLEV <- nlevels(LEVS)
pgons <- lapply(1:length(CL), function(i)
sp::Polygons(list(sp::Polygon(cbind(CL[[i]]$x, CL[[i]]$y))), ID = i))
spgons <- sp::SpatialPolygons(pgons)
if (show.legend) {
leaflet::addPolygons(data = spgons, color = heat.colors(NLEV, NULL)[LEVS], stroke = FALSE, ...) %>%
leaflet::addLegend(colors = heat.colors(NLEV, NULL)[LEVS], labels = LEVS)
} else {
leaflet::addPolygons(data = spgons, color = heat.colors(NLEV, NULL)[LEVS], stroke = FALSE, ...)
}
}
mydata <- read.csv("DATA.csv", sep=";")
mydata <- subset(mydata, !is.na(CrdLatDeg))
leaflet() %>%
addTiles(group = "OSM") %>%
addHeatMap(data = mydata, lon = CrdLonDeg, lat = CrdLatDeg, intensity = FsmIdf, show.legend = TRUE)

Both use a different algorithm. You need to tweak the radius and blur arguments of addHeatmap and the h argument of stat_density2d to get somewhat similar results.

Related

Polygons shifted north of raster even with same CRS

I am having trouble. I am unable to identify the issue when plotting a SpatialPixelDataframe and a SpatialPolygonDataframe with the same CRS in tmaps.
The spatialpixels object can be found here saved as RDS, and the polygons shapefile here, zipped.
Here is my attempt with base functions:
library(sf)
library(sp)
ireland <- st_read("Counties.shp")
sp_pred <- readRDS("sppred_range100_sd2.RDS")
#transform polygons into the pixels CRS
ireland_proj <- st_transform(ireland, sp_pred#proj4string)
#turn into sp object
ireland_sp <- as_Spatial(ireland_proj)
#plot with base functions
plot(sp_pred['mean'])
plot(ireland_sp, add = T)
Here is my attempt with tmap
library(tmap)
tm_shape(sp_pred) +
tm_raster("mean", palette = terrain.colors(10)) +
tm_shape(ireland_sp) +
tm_borders("black", lwd = .5) +
tm_legend(show = FALSE)
This is so simple and I can't see where I might have gone wrong, but also I can't see how it can be an error in how tmap works!
As #krenz mentioned you're using different classes here together, however, I'm not fully sure what causes the problem.
Here's a workaround in which your data is first converted to a sf object which is then rasterized using st_rasterize. The result only differs slightly from what you showed. Maybe you have to play around a little bit with the resolution parameters:
library(tmap)
library(sf)
ireland <- st_read("counties/counties.shp")
sp_pred <- readRDS("sppred_range100_sd2.RDS")
sp_pred_proc <- sp_pred %>% st_as_sf()
sp_pred_proc <- st_rasterize(sp_pred_proc["mean"], dx = 5000, dy = 5000)
tm_shape(sp_pred_proc) +
tm_raster("mean", palette = terrain.colors(10)) +
tm_shape(ireland) +
tm_borders("black", lwd = .5) +
tm_legend(show = FALSE)
The left plot was generated with the settings given above, the right one with dx = 6000, dy = 6000.

How can I animate points on a spatial map with gganimate, sf, and ggplot2?

I'm having some trouble with animating some points on a spatial map. For some reason, only about half of the points show up in the animation. In the static plot, I can clearly see all the points.
How can I make sure that the gganimation shows all the points? Does gganimate not 'play nicely' with spatial maps?
Does anyone have more experience with using gganimate and spatial plotting?
I can reproduce the problem with an example dataset:
library(sf)
library(ggplot2)
library(ggspatial)
library(gganimate)
##Reading example data
nc <- st_read(system.file("shape/nc.shp", package="sf"))
##Create new sf=variable of random points
A <- nc %>%
st_sample(size = 30) %>%
st_coordinates() %>%
as.data.frame()
##Create static map
B <- ggplot() +
annotation_spatial(data=nc) +
geom_point(data = A, aes(x=X, y=Y), size = 2, col = "#3a6589")
##Create animation with points showing up one by one
plot_anim <- B +
transition_states(states = Y, state_length = 0, wrap = FALSE) +
enter_recolor(fill = "#f0f5f9") +
shadow_mark(past = TRUE, alpha = 1, fill = "#3a6589")
##Render animation
animate(plot_anim, fps = 40, end_pause = 60)
I suggest to plot your points via ggplot2::geom_sf() - I have found it reliable in animating spatial data.
See the slightly amended code below; what I have done is:
kept the A object in sf format
created a technical variable Y with the second(= Y) coordinate; it is then used as in your original code
removed the ggspatial dependency, and reframed your static map call
removed the fps = 40 (this should have no effect except file size); we need less than 2 MB to upload here
You may find you need to install {transformr} to animate sf objects; it should not be a showstopper.
library(sf)
library(ggplot2)
library(gganimate)
##Reading example data
nc <- st_read(system.file("shape/nc.shp", package="sf"))
##Create new sf=variable of random points
A <- nc %>%
st_sample(size = 30) %>%
st_as_sf() %>%
dplyr::mutate(Y = st_coordinates(.)[,2])
##Create static map
B <- ggplot() +
geom_sf(data = nc) +
geom_sf(data = A, size = 2, col = "#3a6589")
# save static map
ggsave("static_map.png")
##Create animation with points showing up one by one
plot_anim <- B +
transition_states(states = Y, state_length = 0, wrap = FALSE) +
enter_recolor(fill = "#f0f5f9") +
shadow_mark(past = TRUE, alpha = 1, fill = "#3a6589")
##Render animation
animate(plot_anim, end_pause = 60,
height = 200, width = 400) # a higher res img would not upload here :(
# save animated map
anim_save("animated_map.gif")
The static map / 30 random NC points
The dynamic map / small (because 2 MB upload requirements) but otherwise looks legit to me...

Proximity Maps using R

I'm looking to create some proximity maps using R, which show how far areas are from certain points. I can't find any examples in R code, but I've found an output which is the sort of thing I want:
It doesn't necessarily have to have all the labelling/internal boundaries wizardry, but I'd like it to stop at the sea border (thinking of using the rgeos function gintersection - see here).
I've tried doing a density plot as 'heatmaps' (this would be a pretty good solution/alternative) and putting a shapefile over the top (following this suggestion, but they're not lining up and I can't do a gintersection, probably because there's not a coordinate system attached to the density plot.
I used your question to play a little with new libraries...
Get a UK map and define random points
library(raster)
library(sf)
library(ggplot2)
library(dplyr)
library(tidyr)
library(forcats)
library(purrr)
# Get UK map
GBR <- getData(name = "GADM", country = "GBR", level = 1)
GBR_sf <- st_as_sf(GBR)
# Define 3 points on the UK map
pts <- matrix(c(-0.4966766, -2.0772529, -3.8437793,
51.91829, 52.86147, 56.73899), ncol = 2)
# Project in mercator to allow buffer with distances
pts_sf <- st_sfc(st_multipoint(pts), crs = 4326) %>%
st_sf() %>%
st_transform(27700)
ggplot() +
geom_sf(data = GBR_sf) +
geom_sf(data = pts_sf, colour = "red")
Calculate buffer areas
We create a list of multipolygons for each buffer distance. The point dataset must be in projected coordinates (here mercator) as buffer distance is in the scale of the coordinates system.
# Define distances to buffer
dists <- seq(5000, 150000, length.out = 5)
# Create buffer areas with each distances
pts_buf <- purrr::map(dists, ~st_buffer(pts_sf, .)) %>%
do.call("rbind", .) %>%
st_cast() %>%
mutate(
distmax = dists,
dist = glue::glue("<{dists/1000} km"))
# Plot: alpha allows to see overlapping polygons
ggplot() +
geom_sf(data = GBR_sf) +
geom_sf(data = pts_buf, fill = "red",
colour = NA, alpha = 0.1)
Remove overlapping
Buffer areas are overlapping. On the figure above, the more intense red color is due to multiple overlapping layers of transparent red. Let's remove the overlapping. We need to remove from larger areas, the buffer with the lower size. I then need to add again the smallest area to the list.
# Remove part of polygons overlapping smaller buffer
pts_holes <- purrr::map2(tail(1:nrow(pts_buf),-1),
head(1:nrow(pts_buf),-1),
~st_difference(pts_buf[.x,], pts_buf[.y,])) %>%
do.call("rbind", .) %>%
st_cast() %>%
select(-distmax.1, -dist.1)
# Add smallest polygon
pts_holes_tot <- pts_holes %>%
rbind(filter(pts_buf, distmax == min(dists))) %>%
arrange(distmax) %>%
mutate(dist = forcats::fct_reorder(dist, distmax))
# Plot and define color according to dist
ggplot() +
geom_sf(data = GBR_sf) +
geom_sf(data = pts_holes_tot,
aes(fill = dist),
colour = NA) +
scale_fill_brewer(direction = 2)
Remove areas in the sea
If you want to find proximity area on terrestrial parts only, we need to remove buffer areas that are in the sea. Intersection is computed between multipolygons with the same projection. I previously realize an union of the UK map.
# Remove part of polygons in the sea
# Union and projection of UK map
GBR_sf_merc <- st_transform(st_union(GBR_sf), 27700)
pts_holes_uk <- st_intersection(pts_holes_tot,
GBR_sf_merc)
ggplot() +
geom_sf(data = GBR_sf) +
geom_sf(data = pts_holes_uk,
aes(fill = dist),
colour = NA) +
scale_fill_brewer(direction = 2)
And here is the final proximity map using sf, ggplot2 and a few other libraries...
Based on Sébastien's example, a more old-fashioned approach:
library(raster)
GBR <- getData(name = "GADM", country = "GBR", level = 1)
pts <- matrix(c(-0.4966766, -2.0772529, -3.8437793, 51.91829, 52.86147, 56.73899), ncol = 2)
r <- raster(GBR, res=1/12)
d <- distanceFromPoints(r, pts)
m <- mask(d, GBR)
plot(m)

Messed up polygons shape when combining google map with Spatial Polygons

I´m having problems when combining a map from Google Map Api, with a map constructed with geom_polygon from ggplot2. When I plot each map on their own, nothing weird shows up, but when I do combine them, some lines(straight) appear, messing up the boundaries I intend to highlight.
1) The data I´m using to construct the polygos with the black borders comes from this link. The exact url of the file is in my code below. Sometimes it is necessary to unzip the file manually, therefore the code below for downloading the data may not work:
path <- getwd()
fileName <- "R05.zip"
if (!file.exists(fileName)) {
urlFile = "http://www.censo2017.cl/wp-content/uploads/2016/12/R05.zip"
download(urlFile, dest = "./R05.zip", mode ="wb")
}
if (!dir.exists("./R05")) {
unzip("R05.zip")
}
2) Then I load the shapefile with which I´ll construct the polygons.
distritos <- readOGR( dsn= paste(getwd(), "/R05", sep = ""),
layer="Distritos_Censales",
encoding = "UTF-8", stringsAsFactors = FALSE)
3) And select the administrative division I´m interested in
distritos <- distritos[distritos#data$DESC_COMUN=="QUILPUÉ", ]
4) And then the districts (polygons) I´m interested in:
distritos <- distritos[distritos#data$DESC_DISTR=="EL RETIRO" |
distritos#data$DESC_DISTR=="BELLOTO NORTE" |
distritos#data$DESC_DISTR=="VALENCIA" |
distritos#data$DESC_DISTR=="MENA" |
distritos#data$DESC_DISTR=="BELLOTO SUR" |
distritos#data$DESC_DISTR=="ALTO QUILPUÉ" |
distritos#data$DESC_DISTR=="EL SAUCE", ]
5) Construct the base map from the boundaries of distritos. For that,
I use the function to get the center of a map from this StackOverflow question
bbox(distritos)
MapCenter <- function(x1, y1, x2, y2){
center.x <- x1 + ((x2 - x1) / 2)
center.y <- y1 + ((y2 - y1) / 2)
center <- c(center.x, center.y)
center
}
mcdistritos <- MapCenter(bbox(distritos)[1,1], bbox(distritos)[2,1],
bbox(distritos)[1,2], bbox(distritos)[2,2])
basemap <- get_googlemap(mcdistritos, zoom = 13,
maptype = "roadmap",
color = "bw",
style = "feature:administrative|element:labels|visibility:off")
basemap <- ggmap(basemap , extent = "device")
6) prepare the shapefile data to plot it with ggplot2
distritos.fort <- fortify(distritos, region = "DESC_DISTR")
7) Plot together both maps
basemap +
geom_polygon(data = distritos.fort,
aes(x = long, y = lat),
colour = "black",
fill = NA) + coord_map()
I tried by zooming out the base map, in case the polygon boundaries were messed up cause they didn´t fit in the base map, but I got the same result, just a smaller map. Does anyone know how to fix it?
You need to add a group mapping to your aesthetic. e.g.
geom_polygon(data = distritos.fort,
aes(x = long, y = lat, group = group),
colour = "black",
fill = NA) + coord_map()
At the moment you've got one continuous path. The group aesthetic separates your data into different polygons.
I wasn't able to extract your data, so I don't know the exact mapping you require for group. But looking at the other fortify examples & documentation, I believe it is group = group

Add raster to ggmap base map: set alpha (transparency) and fill color to inset_raster() in ggplot2

I want to plot a map with a raster overlaying a GoogleMaps base map in ggplot2. Therefore, I used get_map() and insert_raster() like this:
library(ggplot2)
library(ggmap)
bm <- ggmap(get_map(location = "Bangkok", maptype = "hybrid"))
bm + inset_raster(as.raster(r), xmin = r#extent[1], xmax = r#extent[2],
ymin = r#extent[3], ymax = r#extent[4])
Is there any possibility to set a alpha and change the fill color?
The result looks like this:
Even Faster without fortify:
read the original post below for further information
From this blog entry I found that we can use spatial polygons directly in ggplot::geom_polygon()
r <- raster(system.file("external/test.grd", package="raster"))
# just to make it reproducible with ggmap we have to transform to wgs84
r <- projectRaster(r, crs = CRS("+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs"))
rtp <- rasterToPolygons(r)
bm <- ggmap(get_map(location = bbox(rtp), maptype = "hybrid", zoom = 13))
bm +
geom_polygon(data = rtp,
aes(x = long, y = lat, group = group,
fill = rep(rtp$test, each = 5)),
size = 0,
alpha = 0.5) +
scale_fill_gradientn("RasterValues", colors = topo.colors(255))
How to tackle plotting SPEED if you just need to visualize something
As described below, such plotting might become very slow with large numbers of pixels. Therefore, you might consider to reduce the number of pixels (which in most cases does not really decrease the amount of information in the map) before converting it to polygons. Therefore, raster::aggregate can be used to reduce the number of pixels to a reasonable amount.
The example shows how the number of pixels is decreased by an order of 4 (i.e. 2 * 2, horizontally * vertically). For further information see ?raster::aggregate.
r <- aggregate(r, fact = 2)
# afterwards continue with rasterToPolygons(r)...
Original Post:
After a while, I found a way to solve this problem. Converting the raster to polygons! This idea then basically was implemented after Marc Needham's blog post.
Yet, there is one drawback: ggplot gets really slow with large numbers of polygons, which you will inevitably face. However, you can speed things up by plotting into a png() (or other) device.
Here is a code example:
library(raster)
library(ggplot2)
library(ggmap)
r <- raster(....) # any raster you want to plot
rtp <- rasterToPolygons(r)
rtp#data$id <- 1:nrow(rtp#data) # add id column for join
rtpFort <- fortify(rtp, data = rtp#data)
rtpFortMer <- merge(rtpFort, rtp#data, by.x = 'id', by.y = 'id') # join data
bm <- ggmap(get_map(location = "Shanghai", maptype = "hybrid", zoom = 10))
bm + geom_polygon(data = rtpFortMer,
aes(x = long, y = lat, group = group, fill = layer),
alpha = 0.5,
size = 0) + ## size = 0 to remove the polygon outlines
scale_fill_gradientn(colours = topo.colors(255))
This results in something like this:
just been looking into this myself. The issue i encountered was trying to overlay a ggmap output with a raster was the following error:
Error: geom_raster only works with Cartesian coordinates.
the work around to this issue is to use coord_cartesian() as follows:
library(ggplot2)
library(ggmap)
bm <- ggmap(get_map(location = "Bangkok", maptype = "hybrid"))
bm <- bm + geom_raster(...) # insert your raster here
bm <- bm + coord_cartesian()
plot(bm)
I am not sure where your raster r is coming from. for this to work simply convert your raster r into a data frame and add the data according to the geom_raster() instructions, ensure the coordinates are in lat/long (i.e. same as the map).
To answer your question, through geom_raster() you can manipulate alpha and fill.
Hope this helps.
btw this work around was originally raised at this link:
https://groups.google.com/forum/embed/#!topic/ggplot2/nqzBX22MeAQ

Resources