Animated graphs in ipython notebook - plot

Is there a way of creating animated graphs. For example showing the same graph, with different parameters.
For example is SAGE notebook, one can write:
a = animate([circle((i,i), 1-1/(i+1), hue=i/10) for i in srange(0,2,0.2)],
xmin=0,ymin=0,xmax=2,ymax=2,figsize=[2,2])
a.show()

This has horrible flickering, but at least this creates a plot that animates for me. It is based on Aron's, but Aron's does not work as-is.
import time, sys
from IPython.core.display import clear_output
f, ax = plt.subplots()
n = 30
x = array([i/10.0 for i in range(n)])
y = array([sin(i) for i in x])
for i in range(5,n):
ax.plot(x[:i],y[:i])
time.sleep(0.1)
clear_output()
display(f)
ax.cla() # turn this off if you'd like to "build up" plots
plt.close()

Update: January 2014
Jake Vanderplas has created a Javascript-based package for matplotlib animations available here. Using it is as simple as:
# https://github.com/jakevdp/JSAnimation
from JSAnimation import examples
examples.basic_animation()
See his blog post for a more complete description and examples.
Historical answer (see goger for a correction)
Yes, the Javascript update does not correctly hold the image frame yet, so there is flicker, but you can do something quite simple using this technique:
import time, sys
from IPython.display import clear_output
f, ax = plt.subplots()
for i in range(10):
y = i/10*sin(x)
ax.plot(x,y)
time.sleep(0.5)
clear_output()
display(f)
ax.cla() # turn this off if you'd like to "build up" plots
plt.close()

IPython widgets let you manipulate Python objects in the kernel with GUI objects in the Notebook. You might also like Sage hosted IPython Notebooks. One problem you might have with sharing widgets or interactivity in Notebooks is that if someone else doesn't have IPython, they can't run your work. To solve that, you can use Domino to share Notebooks with widgets that others can run.
Below are three examples of widgets you can build in a Notebook using pandas to filter data, fractals, and a slider for a 3D plot. Learn more and see the code and Notebooks here.
If you want to live-stream data or set up a simulation to run as a loop, you can also stream data into plots in a Notebook. Disclaimer: I work for Plotly.

If you use IPython notebook, v2.0 and above support interactive widgets. You can find a good example notebook here (n.b. you need to download and run from your own machine to see the sliders).
It essentially boils down to importing interact, and then passing it a function, along with ranges for the paramters. e.g., from the second link:
In [8]:
def pltsin(f, a):
plot(x,a*sin(2*pi*x*f))
ylim(-10,10)
In [9]:
interact(pltsin, f=(1,10,0.1), a=(1,10,1));
This will produce a plot with two sliders, for f and a.

If you want 3D scatter plot animations, the Ipyvolume Jupyter widget is very impressive.
http://ipyvolume.readthedocs.io/en/latest/animation.html#

bqplot is a really good option to do this now. its built specifically for animation through python in the notebook
https://github.com/bloomberg/bqplot

On #goger's comment of 'horrible flickering', I found that calling clear_output(wait=True) solved my problem. The flag tells clear_output to wait to render till it has something new to render.

matplotlib has an animation module to do just that. However, examples provided on the site will not run as is in a notebook; you need to make a few tweaks to make it work.
Here is the example of the page below modified to work in a notebook (modifications in bold).
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.animation import FuncAnimation
from matplotlib import rc
from IPython.display import HTML
fig, ax = plt.subplots()
xdata, ydata = [], []
ln, = plt.plot([], [], 'ro', animated=True)
def init():
ax.set_xlim(0, 2*np.pi)
ax.set_ylim(-1, 1)
return ln,
def update(frame):
xdata.append(frame)
ydata.append(np.sin(frame))
ln.set_data(xdata, ydata)
return ln,
ani = FuncAnimation(fig, update, frames=np.linspace(0, 2*np.pi, 128),
init_func=init, blit=True)
rc('animation', html='html5')
ani
# plt.show() # not needed anymore
Note that the animation in the notebook is made via a movie and that you need to have ffmpeg installed and matplotlib configured to use it.

Related

plotly visualization not working in Pyspark kernel on EMR Jupyterhub Notebook

I'm trying to plot graphs using plotly on EMR Jupyterhub Notebook however the graphs are not being rendered in Pyspark kernel. (Note: Python kernel renders the graph just fine)
Sample code I am trying:
data_canada = px.data.gapminder().query("country == 'Canada'")
fig = px.bar(data_canada, x='year', y='pop')
fig.show()
I am able to plot a graph with %%display sparkmagic however I am not able to figure out if we can get plotly working with %%display sparkmagic -
import random
data = [('Person:%s' % i, i, random.randint(1, 5)) for i in range(1, 50)]
columns = ['Name', 'Age', 'Random']
spark_df = spark.createDataFrame(data, columns)
%%display
spark_df
Has anyone tried this successfully? Please advise.
This is the limitation of sparkmagic. You would have to resort to %%local magic. From sparkmagic docs.
Since all code is run on a remote driver through Livy, all structured data must
be serialized to JSON and parsed by the Sparkmagic library so that it can be
manipulated and visualized on the client side. In practice this means that you
must use Python for client-side data manipulation in %%local mode.

Geopandas to_file gives blank prj file

I am trying to use GeoPandas for a (only slightly) more complex project, but at the moment I'm failing to write out a simple shapefile with a single point in it in a projected manner.
The following code results in a shapefile that looks generally good - but the .prj is empty:
import pandas as pd
from geopandas import GeoDataFrame
from shapely.geometry import Point
df=pd.read_csv("richmond.csv")
geometry = [Point(xy) for xy in zip(df.x, df.y)]
crs = {'init': 'epsg:4326'}
geo_df = GeoDataFrame(df, crs=crs, geometry=geometry)
geo_df.to_file("geopan.shp")
The csv is 2 row and 2 columns (header row, then lon and lat in 2nd row):
Am I missing something obvious? I've hunted through stackoverflow, the geopandas docs, etc. All seem to imply to_file() should work just fine.
In the long run, the goal is to create a few functions for my students to use in a lab - one that draws a line along a lat or lon the width / height of the US, another that clips the line to polygons (the states), so that the students can figure out the widest spot in each state as a gentle introduction to working with spatial data. I'm trying to avoid arcpy as it's Python 2, and I thought (and think) I was doing the right thing by teaching them the ways of Python 3. I'd like them to be able to debug their methodologies by being able to open the line in Arc though, hence this test.
So, after playing with this, I've determined that under the current version of Anaconda the problem is with crs = {'init': 'epsg:4326'} on Windows machines. This works fine on Macs, but has not worked on any of my or my students' Windows systems. Changing this line to make use of the proj4 string crs = {'proj': 'latlong', 'ellps': 'WGS84', 'datum': 'WGS84', 'no_defs': True} instead works just fine. More of a workaround than an actual solution, but, it seems to consistently work.
I'm always using from_epsg function from fiona library.
>>> from fiona.crs import from_epsg
>>> from_epsg(4326)
{'init': 'epsg:4326', 'no_defs': True}
I've never had any problems using it. Keep it mind that some local projections are missing, but it shouldn't be a problem in your case.
Another user and I had a similar issue using fiona, and the issue for me was the GDAL_DATA environmental variable not being set correctly. To reiterate my answer there: For reference, I'm using Anaconda, the Spyder IDE, Fiona 1.8.4, and Python 3.6.8, and GDAL 2.3.3.
While Anaconda usually sets the GDAL_DATA variable upon entering the virtual environment, using another IDE like Spyder will not preserve it, and thus causes issues where fiona (and I assume Geopandas) can't export the CRS correctly.
You can test this fix by trying to printing out a EPSG to WKT transformation before & after setting the GDAL_DATA variable explictly.
Without setting GDAL_DATA:
import os
print('GDAL_DATA' in os.environ)
from osgeo import osr
srs = osr.SpatialReference() # Declare a new SpatialReference
srs.ImportFromEPSG(3413) # Import the EPSG code into the new object srs
print(srs.ExportToWkt()) # Print the result before transformation to ESRI WKT (prints nothing)
Results in:
False
With setting GDAL_DATA:
import os
os.environ['GDAL_DATA'] = 'D:\\ProgramData\\Anaconda3\\envs\\cfm\\Library\\share\\gdal'
print('GDAL_DATA' in os.environ)
from osgeo import , osr
srs = osr.SpatialReference() # Declare a new SpatialReference
srs.ImportFromEPSG(3413) # Import the EPSG code into the new object srs
print(srs.ExportToWkt()) # Print the result before transformation to ESRI WKT (prints nothing)
Results in:
True
PROJCS["WGS 84 / NSIDC Sea Ice Polar Stereographic North",GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4326"]],PROJECTION["Polar_Stereographic"],PARAMETER["latitude_of_origin",70],PARAMETER["central_meridian",-45],PARAMETER["scale_factor",1],PARAMETER["false_easting",0],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["X",EAST],AXIS["Y",NORTH],AUTHORITY["EPSG","3413"]]

Bokeh EditTools callback to server

I am trying to understand how to use callbacks for the new Bokeh EditTools (e.g. BoxEditTool or similar).
Specifically, I would like to see on the server side the coordinates of the newly added rectangles, but I am not sure how to do this.
I am running the following server app
def app( curdoc ):
TOOLS = "tap"
p = figure(title="Some Figure", tools=TOOLS)
source = ColumnDataSource( {'xs':[1], 'ys':[1], 'width':[.1],'height':[.1]})
r = p.rect('xs','ys','width','height', source=source)
p.add_tools(BoxEditTool( renderers = [r]))
def cb( attr, old, new ):
print(r.data_source.data)
r.data_source.on_change("selected", cb)
curdoc.add_root(column(p))
I do get printout from the cb when I select different rectangles, but the r.data_source.data does not change
Thanks for the help!
The behaviour you're describing is actually a bug in the current distribution of Bokeh (0.13.0). You can read more in the google groups discussion. To summarize, there was a problem with the synchronization of the data at the server, it has been resolved and merged.
Note that the on_change method for the Rect glyph ColumnDataSource should watch the 'data' attribute and not 'selected'.
Other than that your snippet looks good, but if you want a working example you can look here. This code is under development but at this stage it reads images and allows drawing ROIs, as well as a simple mechanism for serializing and loading them.
Hope this helps!

Using Bokeh to have multiple live plots in a Jupyter notebook

I'm trying to use Bokeh to have several live plots in a Jupyter notebook. I understand that I cannot use push_notebook() since it will only update the last figure. Is there another way to do it as of today?
push_notebook updates the last cell by defatult, if you don't pass it an argument. But it also accepts a "notebook handle" that show returns when it renders a cell.
# in one cell
p = figure(**opts)
r = p2.circle([1,2,3], [4,5,6])
h = show(p)
# in a different cell
r.glyph.fill_color = "white"
push_notebook(handle=h)
See the Basic Usage example notebook in the GitHub repo.

Unable to set bokeh plotting parameters using BOKEH + HOLOVIEW plotting option via Python

Is there any documentation specifying how to pass Bokeh parameters via holoview?
I am reading the tutorials but I think there is something small I have missed.
There is an example online which describes this in Ipython but I am trying to do it via a python WITHOUT Ipython notebook.
http://holoviews.org/Tutorials/Bokeh_Backend.html?highlight=bokeh
When I run this program I get the curves but the color does not change and I also get this error: WARNING:root:Curve01537: Setting non-parameter attribute style={'line_color': 'green'} using a mechanism intended only for parameters
How can we set the parameter?
Code Example here
from pprint import pprint, pformat
import holoviews as hv
import numpy as np
import pathlib, os
import webbrowser
import lasio, las
from holoviews import Store
from holoviews.plotting.bokeh.element import (line_properties, fill_properties, text_properties)
def plot_bokeh(plot):
#Create renderer instance
myrenderer = hv.Store.renderers['bokeh'].instance(fig='html')
out_file_name = "".join(["./OUTPUT/","gyro", "_graph.html"])
with open (out_file_name, 'w') as f:
#Plot static html
f.write (myrenderer.static_html(plot))
f.close()
webbrowser.open_new_tab(pathlib.Path(os.path.abspath(out_file_name)).as_uri())
def holoview_sandbox():
curve_opts = dict(line_color='green')
xs = np.linspace(0, np.pi*4, 100)
data = (xs, np.sin(xs))
holo_plot = hv.Curve(data, label='MY LABEL' , style=curve_opts)
plot_bokeh(holo_plot)
if __name__ == '__main__':
holoview_sandbox()
In HoloViews the options aren't bound to the objects themselves, which has various benefits including being able to plot with different backends. The pure-Python way of setting style options is this:
curve_opts = dict(line_color='green')
xs = np.linspace(0, np.pi*4, 100)
data = (xs, np.sin(xs))
holo_plot = hv.Curve(data, label='MY LABEL')(style=curve_opts)
The Options Tutorial describes how to set options like this, but please let us know if you found some of that unclear.
This syntax works as well
holo_plot.opts(style={'color': 'green'})
When you change the entry 'line_color' to 'color' in the dict() of Philipp's answer, then this works for the matplotlib backend as well.
Details about setting options can also be found here in addition to Philipp's link.

Resources