Is there a way to enable tab completion on Jupyter notebooks or Jupyterlabs attached to an EMR cluster?
Currently, when i start a notebook, there is no autocomplete or tab completion. Even doc strings are not shown.
I use:
emr-5.29.0
Spark 2.4.4
Hive 2.3.6
Hadoop 2.8.5
Hue 4.4.0
Related
I have used Azure DataBricks in my earlier job and it comes with extended support for Notebook widgets to execute notebook manually/ commission a notebook job by selecting some values (Ideally your run state params or variables.)
For information here: https://learn.microsoft.com/en-us/azure/databricks/notebooks/widgets
Is there a similar service or option that I can build while working on SageMaker notebooks?
From Sagemaker Notebook Instances select ‘Open Jupyter’
In Jupyter select the Nbextensions tab
Uncheck the box ‘disable configurations for nbextensions…’
Open Jupyterlab -> Now the UI widgets work in Jupyterlab
Refer - https://forums.fast.ai/t/ipywidgets-not-displaying-on-aws-sagemaker-fastai/85039
I want to use external jar in all instances/nodes of an EMR cluster so that it can be used further in EMR jupyter notebook.
I am currently using the follow
#!/bin/bash aws s3 cp s3://<bucket-name>/<prefix>/jars/TFSReconArtifacts-1.0-WithDependencies.jar /home/hadoop/jars/
But while referring the same in EMR jupyter notebook I am not able to access or find anything in this path.(/home/hadoop/jars)
AIM
My main aim is to use external jar in EMR jupyter notebook which is bigger in size about 300Mb.I have already tried smaller jars and it works by using this command in jupyter notebook, but for bigger jars it is not working :
%%configure
{ "conf": {
"spark.jars":"s3://<bucketname>/jars/TFSReconArtifacts-1.0-WithDependencies.jar"
}}
I shut down all kernels and close jupyterlab in the browser, but when I open it again it starts a session, and a kernel, for all opened notebooks
I'd followed the code and found that in jupyterlab notebook code the start session function calls (POST /api/sessions) on notebook load if there is not any running kernel for the current notebook. and the API creates a session and also starts a kernel.
I can't find any config for stopping this action in jupyter configs or jupyterlab settings (settings -> advanced settings editor)
You can prevent automatically starting kernels when you open a notebook by running Jupyterlab with the jupyter lab --LabServerApp.notebook_starts_kernel=False flag.
To make the configuration permanent, you can add
c.LabServerApp.notebook_starts_kernel = False to $(jupyter --config-dir)/jupyter_lab_config.py. If this file doesn't exist yet, generate it by running jupyter lab --generate-config.
See this issue comment for more details:
https://github.com/jupyterlab/jupyterlab/issues/12019#issuecomment-1145841583
I would like to setup a system such that it not only runs jupyter notebook on start, but it also starts executing a specific notebook on that jupyter server (running all cells in sequence).
Is this possible? I specifically want to be able to access the notebook web interface and inspect/stop/etc the running notebook at any point.
I know nbconvert can execute a notebook, but it seems to run independently of any existing jupyter servers?
Maybe there is some API I can access so that I can write a shell script to run jupyter notebook and then use the API to open and run a notebook?
I am running ipython notebook on my OSX machine and/or my ubuntu 14.04 machine. I am using ipython 3.0.0, and ipython (jupyter) notebooks.
When I start an ipython notebook, under New there is a terminal option, but it's unavailable for me.
I haven't been able to find any documentation on this feature, how to activate it or what it does.
The ipython notebook --help command doesn't mention it and I haven't found anything in the documentation either.
I haven't discovered the special keywords to search google for to get any information either.
What does this feature do? How do I activate it? Is there any documentation on this available?
IPython/Jupyter appears to support browser-based interactive terminal sessions. This is enabled on my machine by installing the terminado package with pip or conda. This fixes the "Terminals Unavailable" message on the drop-down, and lets me start up a (bash) terminal session in a new tab.
See this commit:
IPython on GitHub
Here's the code in Lib/site-packages/IPython/html/notebookapp.py responsible for this item (located the file by searching the source for "Terminals" case-sensitively):
def init_terminals(self):
try:
from .terminal import initialize
initialize(self.web_app)
self.web_app.settings['terminals_available'] = True
except ImportError as e:
log = self.log.debug if sys.platform == 'win32' else self.log.warn
log("Terminals not available (error was %s)", e)
As you can see, there should be a message in the console log specifying what went wrong (you may need to increase log verbosity with ipython notebook --log-level=<level> to see it). In my case, it said:
Terminals not available (error was No module named fcntl)
The html.terminal module that is being imported appears to provide a web-based IPython interactive console.
Support for Windows terminals with terminado dependency was added in Jupyter 5.3.0:
https://github.com/jupyter/notebook/pull/3087
Actually it's jupyter notebook 5.3.0, not jupyter. the two versions is not the same thing.
- jupyter --version
- jupyter notebook --version
I ever suffered from this.