How to rasterize a single layer of a ggplot? - r

Matplotlib allows to rasterize individual elements of a plot and save it as a mixed pixel/vector graphic (.pdf) (see e.g. this answer). How can the same achieved in R with ggplot2?
The following is a toy problem in which I would like to rasterize only the geom_point layer.
set.seed(1)
x <- rlnorm(10000,4)
y <- 1+rpois(length(x),lambda=x/10+1/x)
z <- sample(letters[1:2],length(x), replace=TRUE)
p <- ggplot(data.frame(x,y,z),aes(x=x,y=y)) +
facet_wrap("z") +
geom_point(size=0.1,alpha=0.1) +
scale_x_log10()+scale_y_log10() +
geom_smooth(method="gam",formula = y ~ s(x, bs = "cs"))
print(p)
ggsave("out.pdf", p)
When saved as .pdf as is, Adobe reader DC needs ~1s to render the figure. Below you can see a .png version:
Of course, it is often possible to avoid the problem by not plotting raw data

Thanks to the ggrastr package by Viktor Petukhov & Evan Biederstedt, it is now possible to rasterize individual layers. However, currently (2018-08-13), only geom_point and geom_tile are supported. and work by Teun van den Brand it is now possible to rasterize any individual ggplot layer by wrapping it in ggrastr::rasterise():
# install.packages('devtools')
# remotes::install_github('VPetukhov/ggrastr')
df %>% ggplot(aes(x=x, y=y)) +
# this layer will be rasterized:
ggrastr::rasterise(geom_point(size=0.1, alpha=0.1)) +
# this one will still be "vector":
geom_smooth()
Previously, only a few geoms were supported:
To use it, you had to replace geom_point by ggrastr::geom_point_rast.
For example:
# install.packages('devtools')
# devtools::install_github('VPetukhov/ggrastr')
library(ggplot2)
set.seed(1)
x <- rlnorm(10000, 4)
y <- 1+rpois(length(x), lambda = x/10+1/x)
z <- sample(letters[1:2], length(x), replace = TRUE)
ggplot(data.frame(x, y, z), aes(x=x, y=y)) +
facet_wrap("z") +
ggrastr::geom_point_rast(size=0.1, alpha=0.1) +
scale_x_log10() + scale_y_log10() +
geom_smooth(method="gam", formula = y ~ s(x, bs = "cs"))
ggsave("out.pdf")
This yields a pdf that contains only the geom_point layer as raster and everything else as vector graphic. Overall the figure looks as the one in the question, but zooming in reveals the difference:
Compare this to an all-raster graphic:

I think you've set yourself up to not have this question answered. You write:
I expect an answer to provide an extension to ggplot2 that allows to export plots with rasterized layers with minimal changes to to existing plotting commands, i.e. as wrapper for geom_... commands or as an additional parameter to these or a ggsave command that expects a list of unevaluated parts of a plot command (every second to be rasterized), not a hacky workaround as provided in the linked question.
This is a major development effort that could easily require several weeks or more of effort by a highly skilled developer. It's unlikely anybody will do this just because of a Stack Overflow question. In lieu of a functioning implementation, I'll describe here how one could implement what you're asking for and why it's rather challenging.
The players
Let's start with the key players we'll be dealing with. At the highest level sits the ggplot2 library. It takes data frames and turns them into figures. ggplot2 itself doesn't know anything about low-level drawing, though. It only deals with lines, polygons, text, etc., which it hands off to the grid library in the form of graphics objects (grobs).
The grid library itself is a fairly high-level library. It also doesn't know much about low-level drawing. It primarily deals with lines, polygons, text, etc., which it hands off to an R graphics device. The device does the actual drawing.
There are many different R graphics devices. Enter ?Devices in an R command line to see an incomplete list. There are vector-graphics devices, such as pdf, postscript, or svg, raster devices such as png, jpeg, or tiff, and interactive devices such as X11 or quartz. Obviously, rasterization as a concept only makes sense for vector-graphics devices, since raster devices raster everything anyways. Importantly, neither ggplot2 nor grid know or care which graphics device you're currently drawing on. They deal with graphical objects that can be drawn on any device.
Ideal high-level interface
The high-level interface should consist of an option rasterize in the layer() function of ggplot2. In this way, one could simply write, e.g., geom_point(rasterize = TRUE) to rasterize the points layer. This would work transparently for all geoms and stats, since they all call layer().
Possible implementations
I see four possible routes of implementation, ordered from most impossible to least.
1. Ideally, the layer() function would simply hand off the rasterize option to the grid library, which would hand it off to the graphics device to tell it which parts of the plot to rasterize. This approach would require major changes in the graphics device API. I don't see this happening. Not in my lifetime, at least.
2. Alternatively, one could write a new grob type that can take any arbitrary grob and rasterize it on demand when the grob is drawn on a graphics device. This approach would not require changes in the graphics device API, but it would require detailed knowledge of the low-level implementation of the grid library. It would also possibly make interactive viewing of such figures very slow.
3. A slightly simpler alternative to 2. would be to rasterize the arbitrary grob only once, on grob construction, and then reuse whenever that grob is drawn. This would be quite a bit faster on interactive graphics devices but the drawing would get distorted if the aspect ratio is changed interactively. Nevertheless, since the primary use of this functionality would be to generate pdf output (I assume), this option might be sufficient.
4. Finally, rasterization could also happen in the layer() function, and that function could simply place a regular raster grob into the grob tree. That solution is similar to the technique described here. Technically, it's not much different from 3. Either way, one needs to write code to rasterize a grob tree and then replace it by a raster grob.
Technical hurdles
To rasterize parts of the grob tree, we'd have to send them to an R raster graphics device to render. However, there isn't one that renders to memory. So, one would have to render to a temporary file (e.g., using png()), and then read the file back in. That's possible but ugly. It also depends on functionality (such as png()) that isn't guaranteed to be available on every R installation.
Second, to render parts of the grob tree separately from the overall rendering, we'll have to open a new graphics device in addition to the one currently open. That's possible but can lead to unexpected bugs. I'm dealing with such bugs all the time, see e.g. here or here for issues related to code using this technique. Whoever implements the rasterization functionality would have to deal with such issues.
Finally, we'll have to get the rasterization code accepted into the ggplot2 library, since we need to replace the layer() function and I don't think there's a way to do that from a separate package. Given how hackish the rasterization solutions are going to be (see previous two paragraphs), that may be a tall order.

Related

creating multiple file types while plotting

I would like to produce a series of plots in both high-resolution and low-resolution versions, or stated differently using two different file types (.png and .eps). I'd like to know the best/least repetetive way to do this. I am using the gplot function in sna, and the plot has a custom legend outside the plot area. I wrote a function something like this:
library(sna)
plotfun <- function(net){
png("test.png",width=800)
p <- gplot(net)
par(xpd=T)
legend(max(p[,1])+1,max(p[,2]),legend=letters[1:10],title="custom legend")
dev.off()
seteps()
postscript(test.eps)
#repeat all the plotting commands, which are much longer in real life
dev.off()
}
#try it with some random data
plotfun(rgraph(10))
This is perfectly functional but seems inefficient and clumsy. The more general version of this question is: if for any reason I want to create a plot (including extra layers like my custom legend), store it as an object, and then plot it later, is there a way to do this? Incidentally, this question didn't seem sna specific to me at first, but in trying to reproduce the problem using a similar function with plot, I couldn't get the legend to appear correctly, so this solution to the outside-the-plot-area legend doesn't seem general.
I would recommend generate graphs only in Postscript/PDF from R and then generate bitmaps (e.g. PNG) from the Postscript/PDF using e.g. ImageMagick with -density parameter (http://www.imagemagick.org/script/command-line-options.php#density) set appropriately to get desired resolution. For example
convert -density 100 -quality 100 picture.pdf picture.png
assuming picture.pdf is 7in-by-7in (R defaults) will give you a 700x700 png picture.
With this approach you will not have to worry that the picture comes out formatted differently depending which R device (pdf() vs png()) is used.

Producing a vector graphics image (i.e. metafile) in R suitable for printing in Word 2007

First a caveat: I posted this question here on SuperUser, but it is clearly the wrong place to ask R questions. I recognize that it is not directly a programming question, but I believe it can be solved by changing how plots are produced (i.e. by coding appropriately). So I hope readers find this appropriate for the forum.
R plots usually consist entirely of vector graphics elements (i.e. points, lines, polygons, text). R permits you to save your figure (or copy-paste) in various formats including various raster formats, as a PDF, or as a Windows meta-file.
I usually save my images as PDFs and print them. This renders the images exactly as I intended them on paper, in the highest quality. I avoid raster formats (e.g. JPG, TIFF) for printing as inevitably the quality is poorer and publishers prefer vector formats.
However, I need to make a large multi-page desktop published document using Microsoft Word 2007, and therefore using PDFs is not an option. When I import my figures from meta-files, or copy and paste directly from R into Word both the screen and print rendering of the image changes slightly (e.g. polygons and their fills become slightly misaligned).
Given that I want to retain high vector quality (and not use raster formats), what can I do to make R vector graphics work with Word? (Of course Sweave and LaTeX would be nice, but again, not a realistic option).
Consider this example:
plot(c(1:100), c(1:100), pch=20)
## Copy and paste to Word 2007 as Windows metafile
## Print
## Quality is poorer (e.g. dot fills misaligned with borders)
pdf("printsPerfectly.pdf")
plot(c(1:100), c(1:100), pch=20)
dev.off()
## Now print PDF
## Quality is as expected
EDIT: Further to suggestions by #John I produced it as an EPS postscript file (see below), inserted it as a picture into Word. Because ultimately it will be printed from a PDF created from Word, I converted it to a PDF using default Word 2007 settings, printed it on my HP Laserjet P1606dn laser printer, and then took aphotograph to illustrate the issue of polygons borders and fills misaligning (image on left, below). I also produced it directly as PDF from R using pdf() and printed the PDF and took a photograph (image on right, below).
It may seem like small potatoes! But when you have gone to a lot of trouble to achieve high quality, it is disappointing to be thwarted at the end. In addition, it is not really obvious here, but the numerals are not as high-quality (left) as in the PDF (right), disregarding differences in focus on the photograph.
The accepted answer to me is not acceptable, since if one goes to the trouble of making a nice vector based figure, the last thing one would like to do is just rasterize it to a bitmap... Unless it's an increadibly complex graph that takes ages to render in vector format, or something like that, but for most graphs that's not the case.
The best solution is to export to Word directly in native Office vector format. I just made a new package, export, that allows one to do exactly that an allows export of either graphs or statistical tables to Word and Powerpoint, see
https://cran.r-project.org/web/packages/export/index.html and for demo see
https://github.com/tomwenseleers/export
For example:
library(devtools)
devtools::install_github("tomwenseleers/export")
library(export)
?graph2ppt
?graph2doc
?table2ppt
?table2doc
## export of ggplot2 plot
library(ggplot2)
qplot(Sepal.Length, Petal.Length, data = iris, color = Species,
size = Petal.Width, alpha = I(0.7))
# export to Word
graph2doc(file="ggplot2_plot.docx", width=7, height=5)
# export to Powerpoint
graph2ppt(file="ggplot2_plot.pptx", width=7, height=5)
You can also export to enhanced metafile using the function
graph2emf(file="ggplot2_plot.emf", width=7, height=5)
but the quality of the native Office format is better.
For final production you can also readily print it to PDF from Powerpoint if need be, and it will stay nicely in vector format then.
Your only option is to use high resolution raster graphics. Once you're over 300 dpi it will be completely indistinguishable from vector printed; it will just make larger files.. Your copy and paste method is coming in at 72 dpi and will look terrible. If you import from a file you can get the resolution in the file and things will be much better. Fortunately Office 2007 is supposed to handle png images, which have the best compression for typical graphs. Let's say you wanted the image 4" wide and 6" high...
png('printsGreat.png', width = 4, height = 6, units = 'in', res = 300)
plot(c(1:100), c(1:100), pch=20)
dev.off()
Also, Office 2007 is supposed to be able to handle EPS files and R postscript files are by default EPS compatible when you print one page.
postscript("printsPerfectly.eps", width = 4, height = 6, horizontal = FALSE, onefile = FALSE)
plot(c(1:100), c(1:100), pch=20)
dev.off()
But if you don't have luck with them go back to the high resolution image.
My preferred solution is to use the windows metafile device for plotting, e.g.:
win.metafile("mygraph.wmf")
print(gg1)
dev.off()
This produces a *.wmf file that can be copy-pasted into the word file.
The devEMF package seems to produce graphics that look nicer than the default wmf when pasted into PowerPoint.
Since I tried to produce png at high res in R and it didn't seem to work on my PC (if I set the resolution higher than, say, 300 dpi, R would produce an error like "cannot start png device"), the way I found was to save the figure using postscript() and then use GSView to convert the ps file into png with 600 dpi resolution. MS Word consumes the png's happily and the quality of print seems to be perfect.
What #Tom Wenseleers said:
The current best answer above to me is not acceptable, since if one
goes to the trouble of making a nice vector based figure, the last
thing one would like to do is just rasterize it to a bitmap... Unless
it's an increadibly complex graph that takes ages to render in vector
format, or something like that, but for most graphs that's not the
case.
For me, there is a new best answer to this question, since graph2ppt and graph2doc tend to move axis labels around (which apparently cannot be fixed; see here: https://github.com/davidgohel/rvg/blob/master/R/body_add_vg.R and here: export::graph2office moves axis labels around).
I think that .svg is the most appropriate vector format for usage with publication graphics. The only drawback is that older versions of e.g. MS Word cannot handle it. IN R, you could use the native graphics::svg - device. However, I'd recommend to use CairoSVG from the Cairo - Package, especially when you are working with non-native fonts (e.g. via the extrafont - package), because in contrast to graphics::svg, Cairo::CairoSVG embeds fonts quite nicely (without relying on GhostScript, if I am right).
If you are working with an older version of MS Word, you could use incscape (a free vector graphic editor) and convert your graph to .wmf, for example (which might be better than printing to .wmf directly, because R rasterizes points when exporting .wmf files).
An example:
## create plot
library (ggplot2)
library (extrafont)
# note: if you want to use other fonts than the standard ones - in this example "ChantillyLH" -
# you must register your fonts via
# font_import () ##run only once (type "y" in the console)
# and
# loadfonts (device = "win") ##run only once.
# Otherwise, the extrafont - package is not needed.
beautiful_plot <-
ggplot (data = iris, mapping = aes (x = Sepal.Length, y = Petal.Length)) +
geom_point () +
theme (text = element_text (size = 18,
family = "ChantillyLH")
)
# export SVG
library (Cairo)
CairoSVG ("My_Path/My_Plot.svg", width = 6, height = 6)
print (beautiful_plot)
dev.off ()
# the resulting SVG-file is in the the "My_Path" - Folder.
In Incscape, it looks like this:
Newer versions of Word can import raster graphics from SVG files. R 3.6.2 has built-in support for creating SVG files with the svg function - no extra packages needed.
Your example then becomes
svg("printsPerfectly.svg", width=4, height=4)
plot(c(1:100), c(1:100), pch=20)
dev.off()
Note that there is a known issue when you try to create PDF files from Word documents with embedded SVG files with thin lines. If you are using thin lines, e.g. with lwd=0.7 somewhere, you need to apply this workaround.

Raster map vs alternative

I recently found this web page Crime in Downtown Houston that I'm interested in reproducing. This is my first learning experience with mapping in R and thus lack the vocabulary and understanding necessary to make appropriate decisions.
At the end of the page David Kahle states:
One last point might be helpful. In making these kinds of plots, one
might tempted to use the map raster file itself as a background. This
method can be used to make map plots much more quickly than the
methods described above. However, the method has one very significant
disadvantage which, if not handled properly, can destroy the entire
purpose of using the map.
In very plain English what is the difference between the raster file
approach and his approach?
Does the RgoogleMaps package have the ability to produce these types
of high quality maps as seen on the page I referenced above that
calls a google map into R?
I ask not because I lack information but the opposite. There's too much and I want to make a good decision(s) about the approach to pursue so I'm not wasting my time on outdated or inefficient techniques.
Feel free to pass along any readings you think would benefit me.
Thank you in advance for your direction.
Basically, you had two options at the time this plot was made:
draw the map as a layer using geom_tile, where each pixel of the image is mapped onto the x,y axes (slow but accurate)
add a background image to the plot, as a purely "cosmetic" annotation. This method is faster, because you can use grid.raster which draws images more efficiently, but the image is not constrained by the axes of the plotting region. In other words, you have to manually adjust the x and y axes limits to make sure that the image corresponds to the actual positions on the plot.
Now, I would suggest you look at the new annotation_raster in ggplot2 v. 0.9.0. It should have the advantage of speed and leaner output files, and still conform to the data space of the plot. I believe that this function, as well as geom_raster and annotation_map did not exist when David made those plots.

Make resizable plots using the grid graphing system in R

Recently I read about the grid graphing system in R. It is very flexible and with its mastery one should be able to make very sophisticated graphs. However I have not found any good place that will allow me to plot a graph that is also re-sizable? The question is as follows: How do you use grid graphing system in R so that the final output is actually resizable?
One way of doing so is not using the grip graphing system directly, but use the lattice interface to it. The lattice package comes installed with R as far as I know, and forms a very flexible interface to the underlying Trellis graphs, which are grid-based graphs. Lattice also allows you to manipulate the grid directly, so in fact for most sophisticated graphs that will be all you need.
If you really are going to work with the grid graphing system itself, you have to use the correct coordinate system for it to be scalable. Either "native", "npc" (Normalized Parent Coordinates) or "snpc" (Square Normalized Parent Coordinates) allow you to rescale a figure, as they give the coordinates relative to the size (or one aspect of it) of the current viewport.
In order to make full use of these, make sure you understand the concept of viewports very well. I have to admit that I still have a lot to learn about it. If you really want to get on with it, I can suggest the book R Graphics from Paul Murrell
Take a closer look at chapter 5 of that book. You can also learn a lot from the R code of the examples, which can also be found on this page
To give you one :
grid.circle(x=seq(0.1, 0.9, length=100),
y=0.5 + 0.4*sin(seq(0, 2*pi, length=100)),
r=abs(0.1*cos(seq(0, 2*pi, length=100))))
Perfectly scaleable. If you look at the help pages of grid.circle, you'll find the default.units="npc" option. That's where in this case the correct coordinate system is set. Compare to
grid.circle(x=seq(0.1, 0.9, length=100),
y=0.5 + 0.4*sin(seq(0, 2*pi, length=100)),
r=abs(0.1*cos(seq(0, 2*pi, length=100))),
default.units="inch")
which is not scaleable.

Plot to specific plot in multiple-plot window?

If I create a multi-plot window with par(mfrow=...), is it possible to send data to a specific plot (i.e. "the one in the lower left corner") or is the plotting always necessarily sequential? Is there a package for R that does something like this?
For those that are interested, this problem arises out of the fact that R is a single-threaded application and is not ideal for real-time visualization. I have multiple real-time data streams coming into R from an outside source that produces the data asynchronously (and therefore the data streams don't always come in the same order). This results in R flipping around the order of the data visualization plots every time it updates.
You could use split.screen():
par(bg = "white") # erase.screen() will appear not to work
# if the background color is transparent
# (as it is by default on most devices).
split.screen(c(2,1)) # split display into two screens
split.screen(c(1,3), screen = 2) # now split the bottom half into 3
screen(1) # prepare screen 1 for output
plot(10:1)
screen(4) # prepare screen 4 for output
plot(10:1)
Have a look at help(layout). This allows you to specify the what, where and in which sizes.
Once plotted, I don't think you re-plot just partially. But you you can use dev.set() et al to switch between different 'plot devices' (ie windows); see help(dev.list).
Note that the suggested answer here is to use split.screen(). It may work, but according to the split.screen help file: "The recommended way to use these functions is to completely draw a plot and all additions (i.e. points and lines) to the base plot, prior to selecting and plotting on another screen. The behavior associated with returning to a screen to add to an existing plot is unpredictable and may result in problems that are not readily visible."
In an answer to my question, there is a more useful solution, using the par(mfg) option:
Change plot panel in multipanel plot in R
Another option is that of implementing a little GUI e.g. with RGtk2 or RTclTk.
I generally do this for graphs that I want to change in realtime and it works great.
For instance, with RGtk2 and cairoDevice you could just do something like (I assume you have a Glade interface)
# Helper function to get a widget from the Glade interface
getWidget <- function(name)
{
return (interface$getWidget(name))
}
interface <- gladeXMLNew("interface.glade", root="mainWindow")
# Our cairo devices (to draw graphics).
# plot1, plot2, and plot3 are GtkDrawingArea widgets
asCairoDevice(getWidget("plot1"))
# dev.cur() will give the device number of the last device we created
# You'll use this to switch device when you draw in different plots
# Storing the device number is important because you may have other
# devices open from other unrelated plots
# (so never assume they'll just start from 1 and be sequential!!!)
plot1.dev <- as.integer(dev.cur())
asCairoDevice(getWidget("plot2"))
plot2.dev <- as.integer(dev.cur())
asCairoDevice(getWidget("plot3"))
plot3.dev <- as.integer(dev.cur())
# To draw in a specific plot you just do
dev.set(plot2.dev)
plot(....)
This has many other advantages, like that of being able to positions the graphs easily where you want (using Glade Interface Designer) and having the possibility of user interaction through specific buttons (e.g. you may have a "pause acquisition" button).

Resources