I have used Azure DataBricks in my earlier job and it comes with extended support for Notebook widgets to execute notebook manually/ commission a notebook job by selecting some values (Ideally your run state params or variables.)
For information here: https://learn.microsoft.com/en-us/azure/databricks/notebooks/widgets
Is there a similar service or option that I can build while working on SageMaker notebooks?
From Sagemaker Notebook Instances select ‘Open Jupyter’
In Jupyter select the Nbextensions tab
Uncheck the box ‘disable configurations for nbextensions…’
Open Jupyterlab -> Now the UI widgets work in Jupyterlab
Refer - https://forums.fast.ai/t/ipywidgets-not-displaying-on-aws-sagemaker-fastai/85039
Related
I am currently setting up a Python data-science client for SQL Server Machine Learning Services following this guide: https://learn.microsoft.com/en-us/sql/machine-learning/python/setup-python-client-tools-sql?view=sql-server-ver15
Unfortunately, running Jupyter notebooks for this distribution does not seem to work for me: Typing .\Scripts\jupyter-notebook in the distribution folder, or directly running jupyter-notebook.exe from the Scripts sub-folder does not start Jupyter. In the terminal, the command exits with no ouput.
Afterwards, https://localhost:8889/tree is not reachable as should be the case according to the tutorial above.
Any suggestions? (I already checked https://jupyter-notebook.readthedocs.io/en/stable/troubleshooting.html for solutions). Thank you!
Update: At least .\Scripts\jupyter-console is running, though it is not the same experience.
some cases .\Scripts\jupyter-notebook will not work fine because of the installation issues. As anaconda is having both commercial and enterprise editions, issues will raise.
Instead use the following procedure for cool opening of Jupyter-Notebook in anaconda distribution
Click on Start menu
Type Jupyter
Click on jupter icon which is visible in search results
Jupyter CMD will start running and make it minimized instead closing it. It will open the UI of jupyter notebook in browser.
Jupyter notebook for SQL
Check the documentation link which is available in link which is the configuration of jupyter notebook for SQL.
Document Credit: Rajendra Gupta
https://localhost:8889/tree
I have a Jupyter notebook running within an Amazon SageMaker Studio Lab (https://studiolab.sagemaker.aws/) environment, and I want to use Tensordboard to monitor my model's performance inside the notebook.
I have used the following commands to set up the Tensorboard:
%load_ext tensorboard
# tb_log_dir variable holds the path to the log directory
%tensorboard --logdir tb_log_dir
But nothing shows up in the output of the cell where I execute the commands. See:
The two buttons shown in the picture are not responding, BTW.
How to solve this problem? Any suggestions would be appreciated.
I would try the canonical way to use tensorboard in AWS Sagemaker, it should be supported also by Studio Lab, it is described here. Basically install tensorboard and using the EFS_PATH_LOG_DIR launch tensorboard using the embedded console (you can do the following also from a cell):
pip install tensorboard
tensorboard --logdir <EFS_PATH_LOG_DIR>
Be careful with the EFS_PATH_LOG_DIR, be sure this folder is valida path from the location you are, for example by default you are located in studio-lab-user/sagemaker-studiolab-notebooks/ so the proper command would be !tensorboard --logdir logs/fit.
Then open a browser to:
https://<YOUR URL>/studiolab/default/jupyter/proxy/6006/
Running jupyter notebook and jupyter server give me very similar results and the descriptions are also quite similar.
❯ jupyter notebook -h
The Jupyter HTML Notebook.
This launches a Tornado based HTML Notebook Server that serves up an
HTML5/Javascript Notebook client.
❯ jupyter server -h
The Jupyter Server.
This launches a Tornado-based Jupyter Server.
There are differences like server doesn't load nbextensions but I don't understand why there are two commands that have so much overlap.
For reference, this are the versions of various modules.
❯ jupyter --version
jupyter core : 4.7.1
jupyter-notebook : 6.3.0
qtconsole : 5.0.3
ipython : 7.22.0
ipykernel : 5.3.4
jupyter client : 6.1.12
jupyter lab : 3.0.11
nbconvert : 6.0.7
ipywidgets : 7.6.3
nbformat : 5.1.3
traitlets : 5.0.5
Jupyter Notebook (notebook module) contains both:
the server for notebooks (the backend of the web application that hosts the notebook contents, proxies interaction with kernels, and interacts with the operating system by e.g. opening an internet browser on start; this part is generally written in Python), and
the client (the frontend of the web application, e.g. HTML code, javascript, and some extra REST API points on the server).
However, because there are now multiple clients (frontends) providing different web applications for notebooks:
Jupyter Notebook
JupyterLab
RetroLab
nteract
multiple proprietary clients developed outside of project Jupyter
It made sense to split the server component used by all of these so that e.g. JupyterLab does not have to depend on notebook. This also means that if a fix to the server component is needed, it can be released quickly independent of Jupyter Notebook release cycle (and users of all frontents can benefit immediately).
As a consequence, and to make the break up clean, the old Jupyter Notebook was split into:
jupyter-server - the server which was adapted by JuptyterLab, RetroLab, nteract
nbclassic - the Jupyter Notebook as a jupyter-server extension
This implies changes for users and developers, some already described in "migrate from notebook" docs:
the options specific to server rather than notebook were renamed from c.NotebookApp to c.ServerApp (the options specific to notebook remain c.NotebookApp)
the server-specific configuration is now stored in jupyter_server_config.py rather than jupyter_notebook_config.py (same for .json version)
users should now use jupyter server extension rather than jupyter serverextension (note the extra space!) to list, enable or disable extensions
the server extensions need to place their files in a new location: etc/jupyter/jupyter_server_config.d rather than etc/jupyter/jupyter_notebook_config.d (in practice most extensions that were updated to support jupyter server are now placing files in both locations for backward compatibility with notebook, but this will change in the future)
It is important to note that depending on how you start your jupyter notebook, you will see different servers being used:
jupyter nbclassic (assuming nbclassic is installed) will use the new jupyter-server
jupyter notebook will use the old notebook server
jupyter lab will use new jupyter-server starting with JupyterLab 3.0 unless running on JupyterHub/Binder where it might still be using old notebook server, depending on configuration
This also implies that you may see different extensions when running jupyter notebook vs jupyter nbclassic (depending on whether their developers updated the locations, and whether they decided they want to support the legacy notebook server).
The creation of nbclassic replacement rather than removal of the server code from existing notebook package was meant to ensure backward compatibility, and this is why you still have two copies of the Tornado server (one provided by jupyter notebook and one by jupyter server). To make the situation simpler you could remove notebook and install nblcassic, but given that the transition is in progress you may need to adjust a few things manually. However, this is only a temporary situation, as it is planned that Notebook will be migrated to use jupyter server starting with v7.0.
This might look inconvenient for now, but this step ensures greater maintainability of the core Jupyter infrastructure in the future and will benefit users and system admins greatly later on.
I would like to setup a system such that it not only runs jupyter notebook on start, but it also starts executing a specific notebook on that jupyter server (running all cells in sequence).
Is this possible? I specifically want to be able to access the notebook web interface and inspect/stop/etc the running notebook at any point.
I know nbconvert can execute a notebook, but it seems to run independently of any existing jupyter servers?
Maybe there is some API I can access so that I can write a shell script to run jupyter notebook and then use the API to open and run a notebook?
Is there a way to enable tab completion on Jupyter notebooks or Jupyterlabs attached to an EMR cluster?
Currently, when i start a notebook, there is no autocomplete or tab completion. Even doc strings are not shown.
I use:
emr-5.29.0
Spark 2.4.4
Hive 2.3.6
Hadoop 2.8.5
Hue 4.4.0