I am doing a parallel plot with plotly. It works, but get very time consuming when I apply it to huge data, because I am doing as many "trace" as markers.
How can I do it faster?
There is a reproducible example showing how I do it yet :
#Create data
markers=paste("markers",letters,sep="_")
data=data.frame(
mark=rep(markers,3),
map=c(rep(1,26) , rep(2,26) , rep(3,26)),
pos=sample(seq(1:100) , 26*3)
)
# Make the plot with Plotly
library(plotly)
plot_ly(data , x=map , y=pos, mode="markers+lines" , group=mark)
And that is the result
My current plotly parallel plot
Thank you so much for your help !
For each trace, plotly.js must calculate default plot and layout data, so the initial plot will not be very fast. For large data sets, you should use the scattergl plot type instead of scatter, but ideally, you should re-arrange your data so that you don't need to plot as many traces (if this is possible).
Now there's a native parallel coordinates trace parcoords in plotly.js which was built for scalable operations, e.g. uses the GPU for crossfiltering and rendering. Some of the styling is therefore more constrained than with regular lines. Besides the brush filtering, it also has axis drag and drop by grabbing the axis title.
Doc: https://plot.ly/javascript/reference/#parcoords
Examples:
https://plot.ly/javascript/parallel-coordinates-plot/
https://codepen.io/monfera/pen/dNBwOv
As usual, the functionality is accessible via the Python, R etc. API bindings too, e.g. https://plot.ly/python/parallel-coordinates-plot/
It's an older question, just adding this answer for future search hits on this page.
Related
Matplotlib allows to rasterize individual elements of a plot and save it as a mixed pixel/vector graphic (.pdf) (see e.g. this answer). How can the same achieved in R with ggplot2?
The following is a toy problem in which I would like to rasterize only the geom_point layer.
set.seed(1)
x <- rlnorm(10000,4)
y <- 1+rpois(length(x),lambda=x/10+1/x)
z <- sample(letters[1:2],length(x), replace=TRUE)
p <- ggplot(data.frame(x,y,z),aes(x=x,y=y)) +
facet_wrap("z") +
geom_point(size=0.1,alpha=0.1) +
scale_x_log10()+scale_y_log10() +
geom_smooth(method="gam",formula = y ~ s(x, bs = "cs"))
print(p)
ggsave("out.pdf", p)
When saved as .pdf as is, Adobe reader DC needs ~1s to render the figure. Below you can see a .png version:
Of course, it is often possible to avoid the problem by not plotting raw data
Thanks to the ggrastr package by Viktor Petukhov & Evan Biederstedt, it is now possible to rasterize individual layers. However, currently (2018-08-13), only geom_point and geom_tile are supported. and work by Teun van den Brand it is now possible to rasterize any individual ggplot layer by wrapping it in ggrastr::rasterise():
# install.packages('devtools')
# remotes::install_github('VPetukhov/ggrastr')
df %>% ggplot(aes(x=x, y=y)) +
# this layer will be rasterized:
ggrastr::rasterise(geom_point(size=0.1, alpha=0.1)) +
# this one will still be "vector":
geom_smooth()
Previously, only a few geoms were supported:
To use it, you had to replace geom_point by ggrastr::geom_point_rast.
For example:
# install.packages('devtools')
# devtools::install_github('VPetukhov/ggrastr')
library(ggplot2)
set.seed(1)
x <- rlnorm(10000, 4)
y <- 1+rpois(length(x), lambda = x/10+1/x)
z <- sample(letters[1:2], length(x), replace = TRUE)
ggplot(data.frame(x, y, z), aes(x=x, y=y)) +
facet_wrap("z") +
ggrastr::geom_point_rast(size=0.1, alpha=0.1) +
scale_x_log10() + scale_y_log10() +
geom_smooth(method="gam", formula = y ~ s(x, bs = "cs"))
ggsave("out.pdf")
This yields a pdf that contains only the geom_point layer as raster and everything else as vector graphic. Overall the figure looks as the one in the question, but zooming in reveals the difference:
Compare this to an all-raster graphic:
I think you've set yourself up to not have this question answered. You write:
I expect an answer to provide an extension to ggplot2 that allows to export plots with rasterized layers with minimal changes to to existing plotting commands, i.e. as wrapper for geom_... commands or as an additional parameter to these or a ggsave command that expects a list of unevaluated parts of a plot command (every second to be rasterized), not a hacky workaround as provided in the linked question.
This is a major development effort that could easily require several weeks or more of effort by a highly skilled developer. It's unlikely anybody will do this just because of a Stack Overflow question. In lieu of a functioning implementation, I'll describe here how one could implement what you're asking for and why it's rather challenging.
The players
Let's start with the key players we'll be dealing with. At the highest level sits the ggplot2 library. It takes data frames and turns them into figures. ggplot2 itself doesn't know anything about low-level drawing, though. It only deals with lines, polygons, text, etc., which it hands off to the grid library in the form of graphics objects (grobs).
The grid library itself is a fairly high-level library. It also doesn't know much about low-level drawing. It primarily deals with lines, polygons, text, etc., which it hands off to an R graphics device. The device does the actual drawing.
There are many different R graphics devices. Enter ?Devices in an R command line to see an incomplete list. There are vector-graphics devices, such as pdf, postscript, or svg, raster devices such as png, jpeg, or tiff, and interactive devices such as X11 or quartz. Obviously, rasterization as a concept only makes sense for vector-graphics devices, since raster devices raster everything anyways. Importantly, neither ggplot2 nor grid know or care which graphics device you're currently drawing on. They deal with graphical objects that can be drawn on any device.
Ideal high-level interface
The high-level interface should consist of an option rasterize in the layer() function of ggplot2. In this way, one could simply write, e.g., geom_point(rasterize = TRUE) to rasterize the points layer. This would work transparently for all geoms and stats, since they all call layer().
Possible implementations
I see four possible routes of implementation, ordered from most impossible to least.
1. Ideally, the layer() function would simply hand off the rasterize option to the grid library, which would hand it off to the graphics device to tell it which parts of the plot to rasterize. This approach would require major changes in the graphics device API. I don't see this happening. Not in my lifetime, at least.
2. Alternatively, one could write a new grob type that can take any arbitrary grob and rasterize it on demand when the grob is drawn on a graphics device. This approach would not require changes in the graphics device API, but it would require detailed knowledge of the low-level implementation of the grid library. It would also possibly make interactive viewing of such figures very slow.
3. A slightly simpler alternative to 2. would be to rasterize the arbitrary grob only once, on grob construction, and then reuse whenever that grob is drawn. This would be quite a bit faster on interactive graphics devices but the drawing would get distorted if the aspect ratio is changed interactively. Nevertheless, since the primary use of this functionality would be to generate pdf output (I assume), this option might be sufficient.
4. Finally, rasterization could also happen in the layer() function, and that function could simply place a regular raster grob into the grob tree. That solution is similar to the technique described here. Technically, it's not much different from 3. Either way, one needs to write code to rasterize a grob tree and then replace it by a raster grob.
Technical hurdles
To rasterize parts of the grob tree, we'd have to send them to an R raster graphics device to render. However, there isn't one that renders to memory. So, one would have to render to a temporary file (e.g., using png()), and then read the file back in. That's possible but ugly. It also depends on functionality (such as png()) that isn't guaranteed to be available on every R installation.
Second, to render parts of the grob tree separately from the overall rendering, we'll have to open a new graphics device in addition to the one currently open. That's possible but can lead to unexpected bugs. I'm dealing with such bugs all the time, see e.g. here or here for issues related to code using this technique. Whoever implements the rasterization functionality would have to deal with such issues.
Finally, we'll have to get the rasterization code accepted into the ggplot2 library, since we need to replace the layer() function and I don't think there's a way to do that from a separate package. Given how hackish the rasterization solutions are going to be (see previous two paragraphs), that may be a tall order.
I would like to produce a series of plots in both high-resolution and low-resolution versions, or stated differently using two different file types (.png and .eps). I'd like to know the best/least repetetive way to do this. I am using the gplot function in sna, and the plot has a custom legend outside the plot area. I wrote a function something like this:
library(sna)
plotfun <- function(net){
png("test.png",width=800)
p <- gplot(net)
par(xpd=T)
legend(max(p[,1])+1,max(p[,2]),legend=letters[1:10],title="custom legend")
dev.off()
seteps()
postscript(test.eps)
#repeat all the plotting commands, which are much longer in real life
dev.off()
}
#try it with some random data
plotfun(rgraph(10))
This is perfectly functional but seems inefficient and clumsy. The more general version of this question is: if for any reason I want to create a plot (including extra layers like my custom legend), store it as an object, and then plot it later, is there a way to do this? Incidentally, this question didn't seem sna specific to me at first, but in trying to reproduce the problem using a similar function with plot, I couldn't get the legend to appear correctly, so this solution to the outside-the-plot-area legend doesn't seem general.
I would recommend generate graphs only in Postscript/PDF from R and then generate bitmaps (e.g. PNG) from the Postscript/PDF using e.g. ImageMagick with -density parameter (http://www.imagemagick.org/script/command-line-options.php#density) set appropriately to get desired resolution. For example
convert -density 100 -quality 100 picture.pdf picture.png
assuming picture.pdf is 7in-by-7in (R defaults) will give you a 700x700 png picture.
With this approach you will not have to worry that the picture comes out formatted differently depending which R device (pdf() vs png()) is used.
I am trying to plot quite large and dense networks (dput here). All I end up with is a bunch of overlapping dots, which does not really give me a sense of the structure or density of the network:
library(sna)
plot(data, mode = "fruchtermanreingold")
However, I have seen plots which utilizes fading to visualize the degree to which points overlap, e.g.:
How can I implement this "fading" in a plot of a graph?
Here's one way:
library(sna)
library(network)
source("modifieddatafromgist.R")
plot.network(data,
vertex.col="#FF000020",
vertex.border="#FF000020",
edge.col="#FFFFFF")
First, I added a data <- to the gist so it could be sourced.
Second, you need to ensure the proper library calls so the object classes are assigned correctly and the proper plot function will be used.
Third, you should use the extra parameters for the fruchtermanreingold layout (which is the default one for plot.network) to expand the area and increase the # of iterations.
Fourth, you should do a set.seed before the plot so folks can reproduce the output example.
Fifth, I deliberately removed cruft so you can see the point overlap, but you can change the alpha for both edges & vertices (and you should change the edge width, too) to get the result you want.
There's a ton of help in ?plot.network to assist you in configuring these options.
I made a plot in R and I want to repeat all the commands (like plot(), legend() or line()) that were carried out for this plot, with some minor changes. For example I want to set the axes to logarithmic scale and change the title of the plot.
In gnuplot I would use the replot command.
plot ...
set title "The same plot with logarithmic axes"
set logscale
replot
Is something like this possible in R. The only thing that comes to my mind of doing this (besides changing the values manually and re-run the lines of codes) would be setting up a function, that asks for all parameters that might be changed by the user.
Thanks for your help,
Sven
R uses a pen and paper graphics model - once the plot has been drawn on the device that is it. If you want to change some aspect of the plot, you need to replay the graphics function calls that produce the plot with the changes made to the code.
Depending on what you are really doing there are two options:
If this is just for you, write code in a text editor / IDE that knows R and can send chunks of code at a time to R. That way the code to produce the figure is recorded in a separate script which you can paste into/send to R making the changes you need each time to the script.
If you are going to be doing this often, then write yourself a wrapper plotting function that encapsulates the plot code you want but allows you to pass in arguments to alter the aspects you want.
Lattice and ggplot2 are a little different as they are based on grid graphics and create objects that when printed produce a plot on the device. One can manipulate that object to alter what is drawn, and with grid one can push and pop things on to / off a viewport.
If I create a multi-plot window with par(mfrow=...), is it possible to send data to a specific plot (i.e. "the one in the lower left corner") or is the plotting always necessarily sequential? Is there a package for R that does something like this?
For those that are interested, this problem arises out of the fact that R is a single-threaded application and is not ideal for real-time visualization. I have multiple real-time data streams coming into R from an outside source that produces the data asynchronously (and therefore the data streams don't always come in the same order). This results in R flipping around the order of the data visualization plots every time it updates.
You could use split.screen():
par(bg = "white") # erase.screen() will appear not to work
# if the background color is transparent
# (as it is by default on most devices).
split.screen(c(2,1)) # split display into two screens
split.screen(c(1,3), screen = 2) # now split the bottom half into 3
screen(1) # prepare screen 1 for output
plot(10:1)
screen(4) # prepare screen 4 for output
plot(10:1)
Have a look at help(layout). This allows you to specify the what, where and in which sizes.
Once plotted, I don't think you re-plot just partially. But you you can use dev.set() et al to switch between different 'plot devices' (ie windows); see help(dev.list).
Note that the suggested answer here is to use split.screen(). It may work, but according to the split.screen help file: "The recommended way to use these functions is to completely draw a plot and all additions (i.e. points and lines) to the base plot, prior to selecting and plotting on another screen. The behavior associated with returning to a screen to add to an existing plot is unpredictable and may result in problems that are not readily visible."
In an answer to my question, there is a more useful solution, using the par(mfg) option:
Change plot panel in multipanel plot in R
Another option is that of implementing a little GUI e.g. with RGtk2 or RTclTk.
I generally do this for graphs that I want to change in realtime and it works great.
For instance, with RGtk2 and cairoDevice you could just do something like (I assume you have a Glade interface)
# Helper function to get a widget from the Glade interface
getWidget <- function(name)
{
return (interface$getWidget(name))
}
interface <- gladeXMLNew("interface.glade", root="mainWindow")
# Our cairo devices (to draw graphics).
# plot1, plot2, and plot3 are GtkDrawingArea widgets
asCairoDevice(getWidget("plot1"))
# dev.cur() will give the device number of the last device we created
# You'll use this to switch device when you draw in different plots
# Storing the device number is important because you may have other
# devices open from other unrelated plots
# (so never assume they'll just start from 1 and be sequential!!!)
plot1.dev <- as.integer(dev.cur())
asCairoDevice(getWidget("plot2"))
plot2.dev <- as.integer(dev.cur())
asCairoDevice(getWidget("plot3"))
plot3.dev <- as.integer(dev.cur())
# To draw in a specific plot you just do
dev.set(plot2.dev)
plot(....)
This has many other advantages, like that of being able to positions the graphs easily where you want (using Glade Interface Designer) and having the possibility of user interaction through specific buttons (e.g. you may have a "pause acquisition" button).