Jupyter notebooks crashing on multiple plotly plots
Here is my code:
features = ['energy','loudness','speechiness','acousticness', 'instrumentalness', 'liveness', 'valence', 'tempo','duration_ms']
plots = {}
for cols in features:
plots[cols] = px.scatter(genre, x =cols, y = 'danceability' , color = 'Genre', log_x=False,size_max=35)
for key in plots:
plots[key].show()
Is there a way I can clear the memory each time and save it . Then, hopefully close the plots. I tried savefig but that does not work.
Jupyter notebooks keeps on crashing.
I think storing all of your plotly.express figures in your plots dictionary is causing the Jupyter notebook to run out of memory. If you're okay with saving the plots instead of rendering them in your notebook, you can save them locally on each iteration of the loop.
For example:
features = ['energy','loudness','speechiness','acousticness', 'instrumentalness', 'liveness', 'valence', 'tempo','duration_ms']
for cols in features:
fig = px.scatter(genre, x =cols, y = 'danceability' , color = 'Genre', log_x=False,size_max=35)
fig.write_html(f"./plot_{cols}.html")
Related
I'm trying to plot graphs using plotly on EMR Jupyterhub Notebook however the graphs are not being rendered in Pyspark kernel. (Note: Python kernel renders the graph just fine)
Sample code I am trying:
data_canada = px.data.gapminder().query("country == 'Canada'")
fig = px.bar(data_canada, x='year', y='pop')
fig.show()
I am able to plot a graph with %%display sparkmagic however I am not able to figure out if we can get plotly working with %%display sparkmagic -
import random
data = [('Person:%s' % i, i, random.randint(1, 5)) for i in range(1, 50)]
columns = ['Name', 'Age', 'Random']
spark_df = spark.createDataFrame(data, columns)
%%display
spark_df
Has anyone tried this successfully? Please advise.
This is the limitation of sparkmagic. You would have to resort to %%local magic. From sparkmagic docs.
Since all code is run on a remote driver through Livy, all structured data must
be serialized to JSON and parsed by the Sparkmagic library so that it can be
manipulated and visualized on the client side. In practice this means that you
must use Python for client-side data manipulation in %%local mode.
How to display a window with a ggplot figure when executing an R script from the command line (without intermediary save to file)?
Example script test.R
#!/usr/bin/env Rscript
library(ggplot2)
p = ggplot(aes(x = Sepal.Length), data = iris) + geom_histogram(color = 'black', fill = NA)
plot(p)
On the command line run the script with ./test.R.
This dumps the plot to Rplots.pdf - instead I'd like a window just like in an interactive session with the plot, with no file output.
How to specify output device to be the screen? (e.g. on Ubuntu)
You can do this via a call to X11(), which will open a graphics window. Some relevant excerpts from help("X11"):
on Unix-alikes ‘X11’ starts a graphics device driver for the X
Window System (version 11). This can only be done on
machines/accounts that have access to an X server.
Usage:
X11(display = "", width, height, pointsize, gamma, bg, canvas,
fonts, family, xpos, ypos, title, type, antialias)
Arguments:
display: the display on which the graphics window will appear. The
default is to use the value in the user's environment
variable ‘DISPLAY’. This is ignored (with a warning) if an
X11 device is already open on another display.
However, it will close immediately after the R script is done being executed. So, this works to display your plot, but it isn't open for long:
#!/usr/bin/env Rscript
library(ggplot2)
p = ggplot(aes(x = Sepal.Length), data = iris) +
geom_histogram(color = 'black', fill = NA)
X11()
plot(p)
I guess the real questions are
Why are you averse to saving the plot before viewing it? and
If you want to open a plot window but not save the plot, why not just run your commands in an interactive R session? That seems to me more useful if you're not saving results.
I'm trying to execute the below code that display a plot:
using Plots, Measures
pyplot()
data = [rand(100), rand(100)];
histogram(data, layout = 2,
title = ["Dataset A" "Dataset B"], legend = false,
ylabel = "ylabel", margin = 5mm)
I do not get any output once I execute it from the command line!!
Knowing that I tried it in 3 different ways and it works (Jupyter, Juni, Julia session), though I'm confused about the way it works in the Juno, kinldy see my observations below:
With Jupyter it is functioning perfectly:
With Juno, I've to run the file TWICE!! the first time looks to be for compilation, second time for execution, maybe!!
And if I closed the plot, I've to close the Julia session, restart it, then re-execute the file TWICE!! and sometimes nothing is appearing!!
With Julia session, it take time for first time execution, then if I closed the plot and run it again, it is appearing smoothly.
The commands like histogram or plot typically don't display the plots for users, they only generate and return the plots.
What displays the plots is actually the display system in Julia.
When Julia is in interactive use, like in REPL, Jupyter and Juno, the display system will be invoked automatically with commands not ending with ";". That's why you see plots displayed in REPL, Jupyter and Juno.
But when executing the file from the command line, the display system is not automatically activated. So you first have to invoke display yourself like this:
using Plots, Measures
pyplot()
data = [rand(100), rand(100)];
h = histogram(data, layout = 2,
title = ["Dataset A" "Dataset B"], legend = false,
ylabel = "ylabel", margin = 5mm)
display(h)
But even this will not give you the picture, but only text representation of the plot. This is because in command line julia, only a very simple text display system is in place, and it doesn't have "full" support for plots from Plots. To display the plots, you have to write your own display mechanism and push it to Julia display system, which is not hard but a little tedious. I will give an example when I have more time.
BTW, if you just want plots generated from command line, another way is to save it to files, which is more direct than making a display mechanism yourself.
Update
Here is a simple display, which mimics the display system used in Julia REPL. The code here is for Julia 0.7/1.0.
const output = IOBuffer()
using REPL
const out_terminal = REPL.Terminals.TerminalBuffer(output)
const basic_repl = REPL.BasicREPL(out_terminal)
const basic_display = REPL.REPLDisplay(basic_repl)
Base.pushdisplay(basic_display)
Using it before the previous code will show the plot. Please note that you use pyplot() backend for Plots, whose default is to open a new window and display the plot in the window, but when the julia finish execution in the command line, it will close the plot window. To deal with this, we could either change ways for the default display, or use another backend to show the plot, for example, plotly() backend will display the plot through html. The complete code may look like following:
const output = IOBuffer()
using REPL
const out_terminal = REPL.Terminals.TerminalBuffer(output)
const basic_repl = REPL.BasicREPL(out_terminal)
const basic_display = REPL.REPLDisplay(basic_repl)
Base.pushdisplay(basic_display)
using Plots, Measures
plotly()
data = [rand(100), rand(100)];
h = histogram(data, layout = 2,
title = ["Dataset A" "Dataset B"], legend = false,
ylabel = "ylabel", margin = 5mm)
display(h)
Use readline() to get such destination that close the plot window by typing enter.
using GR
y = rand(20,1)
p = plot(y,linewidth=2,title="My Plot")
readline()
I have issue with plotting lines over my existing plot in .Rmd in RStudio. I ran the code within the code chunk in .Rmd (⌘ + return) and the plot gives me a graph within the .Rmd (new feature of RStudio v1.0), however when I ran the second code lines, an error shows up.
plot(density(with$glucose),
ylim = c(0.00, 0.02),
xlab = "Glucose Level",
main = "Figure",
lwd = 2)
lines(density(without$glucose),
col = "red",
lwd = 2)
Error in plot.xy(xy.coords(x, y), type = type, ...) : plot.new has not been called yet
On the other hand, if I copy and paste the codes into the console, I could get the plot I want, in the plot viewer within RStudio.
In addition, when I ran some other codes within the .Rmd (⌘ + return), my plots in the plot viewer in RStudio disappear. This means I have to do copy-paste into the console instead of using the (⌘ + return) shortcut.
Does anyone have the same problem?
This is a known problem, but you can solve it very easy: Press Ctrl+Shift+Enter to run the complete chunk, then everything works fine and you don't have to copy-and-paste all thing to the console.
So do all your plots in one chunk and run this chunk. This will produce you the plot within the RMD file (as you mentioned: new feature of RStudio 1.0)
If you're not a fan of the inline output / notebook mode for R Markdown documents, you can also disable it within the Global Options dialog -- try disabling the option:
Show output inline for all R Markdown document
I'm trying to use Bokeh to have several live plots in a Jupyter notebook. I understand that I cannot use push_notebook() since it will only update the last figure. Is there another way to do it as of today?
push_notebook updates the last cell by defatult, if you don't pass it an argument. But it also accepts a "notebook handle" that show returns when it renders a cell.
# in one cell
p = figure(**opts)
r = p2.circle([1,2,3], [4,5,6])
h = show(p)
# in a different cell
r.glyph.fill_color = "white"
push_notebook(handle=h)
See the Basic Usage example notebook in the GitHub repo.