Can't create ipyparallel clusters on Jupyter Notebook - jupyter-notebook

I have the following:
ipyparallel (5.0.0)
ipython (4.0.3)
I have enabled ipcluster by typing in the command line:
ipcluster nbextension enable
I'm trying to create a new cluster on the IPython Clusters tab on the Jupyter notebook, but this is what I see:
I was able to do this before. Thanks!

From here:
Instead of editing jupyter_notebook_config.py, edit jupyter_notebook_config.json and look for:
"NotebookApp": {
"server_extensions": [
<some lines>
]
change this to:
"NotebookApp": {
"server_extensions": [
<some lines>,
"ipyparallel.nbextension"
]

I've just stumbled upon the same problem, and the fix mentioned in the accepted answer worked, but let me add some context for the future visitors of this question, just in case.
I have Anaconda 5.0 for Linux, under that I first did:
jupyter notebook --generate-config
pip install ipyparallel
jupyter nbextension install --py ipyparallel --user
jupyter nbextension enable --py ipyparallel --user
jupyter serverextension enable --py ipyparallel --user
Which lead to the situation on the screenshot.
Under ~/.jupyter I have both jupyter_notebook_config.json as well as jupyter_notebook_config.py.
The json file had this inside:
{
"NotebookApp": {
"nbserver_extensions": {
"ipyparallel.nbextension": true
}
}
}
I changed the file by adding a "server_extensions" block as follows:
{
"NotebookApp": {
"nbserver_extensions": {
"ipyparallel.nbextension": true
},
"server_extensions": [
"ipyparallel.nbextension"
]
}
}
After restart, Jupyter reported in the logs:
[W 19:44:14.107 NotebookApp] server_extensions is deprecated, use nbserver_extensions
However, the Clusters tab started working as necessary. Apparently, some recent changes in the configuration logic did not propagate to all of the codebase.

Related

Julia in Google Colab

I am trying to setup Julia with Google Colab. Installation instructions as in https://discourse.julialang.org/t/julia-on-google-colab-free-gpu-accelerated-shareable-notebooks/15319 have been followed. Despite that, I am unable to launch Julia.
I am trying to use Julia with Google Colab. I followed the following steps:
Install CUDA
!wget https://developer.nvidia.com/compute/cuda/9.0/Prod/local_installers/cuda-repo-ubuntu1604-9-0-local_9.0.176-1_amd64-deb
!dpkg -i cuda-repo-ubuntu1604-9-0-local_9.0.176-1_amd64-deb
!apt-key add /var/cuda-repo-9-0-local/7fa2af80.pub
!apt update -q
!apt install cuda gcc-6 g++-6 -y -q
!ln -s /usr/bin/gcc-6 /usr/local/cuda/bin/gcc
!ln -s /usr/bin/g++-6 /usr/local/cuda/bin/g++
Install Julia 1.2.0
!curl -sSL "https://julialang-s3.julialang.org/bin/linux/x64/1.2/julia-1.2.0-linux-x86_64.tar.gz" -o julia.tar.gz
!tar -xzf julia.tar.gz -C /usr --strip-components 1
!rm -rf julia.tar.gz*
!julia -e 'using Pkg; pkg"add IJulia; add CuArrays; add Flux; precompile"'
The above two steps run perfectly fine. I am unable to initiate a Julia session. I tried:
!julia
With this, the Julia start-up screen keeps showing with no command-line.
The easiest option is to use this Colab notebook template.
It supports any Julia version, and also has GPU support.
Turns out that it was just the sequence of steps that was wrong. Very helpful video posted https://www.youtube.com/watch?v=xpZo3L2dYTY. Just to reiterate:
Save the following as .ipynb file, and upload it on Google Colab:
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "Julia on Colab.ipynb",
"version": "0.3.2",
"provenance": []
},
"kernelspec": {
"name": "julia-1.2",
"display_name": "Julia 1.2"
},
"accelerator": "GPU"
},
"cells": [
{
"metadata": {
"id": "oMSuTc3pDlHv",
"colab_type": "code",
"colab": {}
},
"cell_type": "code",
"source": [
""
],
"execution_count": 0,
"outputs": []
}
]
}
Install CUDA in the same notebook using the commands mentioned in the question.
Install Julia 1.2.0 in the same notebook using the commands mentioned above.
Configure the settings as demonstrated in the video and you are all set!
In addition to the answer by user3856486: you can now skip the CUDA installation step (mentioned here). That saves a lot of time, especially since you have to rerun these steps whenever you close the notebook/the runtime disconnects.

Jupyter simple frontend button javascript fails to validate

I am trying Create simple notebook frontend extensions as described here
http://jupyter-notebook.readthedocs.io/en/latest/extending/frontend_extensions.html
This is what I have
~$ cat /home/usr/.local/lib/python2.7/site-packages/my_fancy_module/static/main.js // file my_extension/main.js
define([
'base/js/namespace' ], function(
Jupyter ) {
function load_ipython_extension() {
var handler = function () {
alert('this is an alert from my_extension!');
};
var action = {
icon: 'fa-comment-o', // a font-awesome class used on buttons, etc
help : 'Show an alert',
help_index : 'zz',
handler : handler
};
var prefix = 'my_extension';
var action_name = 'show-alert-usr';
var full_action_name = Jupyter.actions.register(action, action_name, prefix); // returns 'my_extension:show-alert'
Jupyter.toolbar.add_buttons_group([full_action_name]);
}
return {
load_ipython_extension: load_ipython_extension
}; });
When I try to install I get validation errors
$ jupyter nbextension enable /home/usr/.local/lib/python2.7/site-packages/my_fancy_module/static/main.js
--user Enabling notebook extension /home/usr/.local/lib/python2.7/site-packages/my_fancy_module/static/main.js...
- Validating: problems found:
- require? X /home/usr/.local/lib/python2.7/site-packages/my_fancy_module/static/main.js
I am very new to Python notebooks.
Additional information/questions:
When I pip installed my_fance_module it did not copy over the static directory and the .js files inside. I had to manually copy them to ~/usr/.local/ path afterwards.
This is my directory structure
.local/lib/python2.7/site-packages/my_fancy_module
-- __init__.py
|static
--- main.js
Also after installing this main.js and running server and notebook shows only main.js and does not show my notebooks anymore on the browser.
Have you tried?
jupyter nbextension install /home/usr/.local/lib/python2.7/site-packages/my_fancy_module/static/main.js --sys-prefix
before;
jupyter nbextension enable /home/usr/.local/lib/python2.7/site-packages/my_fancy_module/static/main.js
Also what version of Jupyter are you working with? Because the path that you indicated appears to be your systems default Python 2.7 that it ships with. If your developing for Jupyter notebook then I think it would be wise to first install Anaconda then install Jupyter notebook through it conda install notebook then migrate your files to you Anaconda dir. So let me know if that doesn't work out for you.

How to disable auto-quotes and auto-brackets in Jupyter 5.0

I upgraded Jupyter to the latest vesion, 5.0, and it looks like my front-end configuration stopped working.
I don't understand why Jupyter comes with auto closing quotes and brackets by default, which I find pretty annoying. So, at each version I have to change the settings to disable it.
It used to work by creating a file ~/.jupyter/custom/custom.js and adding the next JavaScript code:
require(['notebook/js/codecell'], function (codecell) {
codecell.CodeCell.options_default.cm_config.autoCloseBrackets = false;
})
I've read that since Jupyter 4 this code could be changed by:
IPython.CodeCell.options_default.cm_config.autoCloseBrackets = false;
But it looks like in Jupyter 5, the two previous options stopped working.
The documentation I found regarding the front-end configuration is not helpful (I'll be happy to improve it once I understand it):
http://jupyter-notebook.readthedocs.io/en/latest/frontend_config.html#frontend-config
Can anyone help me understand how to disable auto-brackets and auto-quotes in Jupyter 5 please?
This is the exact version I'm running:
It looks like it can be done by running in a notebook:
from notebook.services.config import ConfigManager
c = ConfigManager()
c.update('notebook', {"CodeCell": {"cm_config": {"autoCloseBrackets": False}}})
This creates a file ~/.jupyter/nbconfig/notebook.json with the content:
{
"CodeCell": {
"cm_config": {
"autoCloseBrackets": false
}
}
}
After executing the Python command, or manually creating the file, restart your Jupyter notebook, and it should stop auto-closing quotes and brackets.
For JupyterLab visitors there is a "User Preferences" panel of the "Notebook" settings editor into which you paste & save:
{
"codeCellConfig": {
"autoClosingBrackets": false
}
}
Open with Ctrl + , or via menu: Settings → Advanced Settings Editor & click "Notebook"
You can simply got to Settings tab and uncheck Auto Close Brackets option to disable auto-complete or check it to enable auto-complete
Here is my 2 jpgs:
[step 1] Go to "Advanced Settings" under "Settings" in menu:
[step 2]
Select the "Notebook" category, in the "User Preference" tab, add text as follows: (just follow the syntax shown in the "System Defaults" tab):
{
"codeCellConfig": {
"autoClosingBrackets": false,
},
}
FYI:
!jupyter --version
jupyter core : 4.7.1
jupyter-notebook : 6.3.0
qtconsole : 5.0.3
ipython : 7.22.0
ipykernel : 5.3.4
jupyter client : 6.1.12
jupyter lab : 3.0.14
nbconvert : 6.0.7
ipywidgets : 7.6.3
nbformat : 5.1.3
traitlets : 5.0.5

How can I change current working directory in Sublime Text 3?

Whenever I open a script (Python or R), I want Sublime automatically changes the current working directory to the path of that file. Is it possible?
I added the setting "working_dir": "$file_path", to Sublime preferences but it doesn't help.
Ok, I solved the problem. Here is what I did:
Install Sublime Text 3 package PackageResourceViewer
Open Command Palette, search for PackageResourceViewer: Open Resource
Browse to Python resource then open Python.sublime-build
Remove the default line "shell_cmd": "python -u \"$file\"",
Add the following 2 lines:
"cmd": ["python", "-u", "$file"],
"working_dir": "$file_path",
Alternately, you can replace the above 2 lines by:
"shell_cmd": "cd $file_path; python -u \"$file\""
Do the same thing for R or other build resources if you want Sublime Text 3 automatically change the current working directory in accordance with the active scripts' path.
You can open the python console using the Show Console command in View menu, and then type
import os
os.chdir('/my/directory')
It changes the working directory for the Sublime Text process and whatever process it spawns.
Another solution: You know the console opens with Alt+". When I use the combination Alt+1 it's directly chosing the current file path. And this way is something you have to do every single time you open sublime text.
I was having the same issue, plus if i wanted to import a module, it would always return module not found.
To fix this: I change the first line of my python.sublime-build file to what i have below
{
"shell_cmd": "cd $file_path; python -u \"$file\"",
"file_regex": "^[ ]*File \"(...*?)\", line ([0-9]*)",
"selector": "source.python",
{
"env": "PYTHONIOENCODING": "utf-8",
],
"variants":
}
{
"name": "Syntax Check",
"shell_cmd": "python -m py_compile \"${file}\"",
}
]
}
Being cuper curious, i wanted to know why this was happening, so after doing more investigations about my os, ran:
import os
for k, v in os.environ.items():
print(k, v)
print(os.getcwd())
which returns a dictionary of my environment variables and their values, including 2 that answered my questions in the build results PWD and OLDPWD:
PWD /Users/<myname>/code
OLDPWD /Users/<myname>/code/code_py
...
[Finished 0.0ms]
...
[shell_cmd: cd /Users/<myname>/code/code_py; python -u "/Users/<myname>/code/code_py/YT-tutorials.py"]
...
AHA!
because i previously added cd $HOME/code my ~/.bash_profile, so every time i would build (command+B) in sublime with python, my shell profile would change directories behind the scenes and move me into my ~/code directory.
Now, after changing the python.sublime-build file as mentioned above changed this build behavior has been fixed -- & i hope this helps you too!

How can I install Qt 5.2.1 from the command line in Cygwin?

$ wget --quiet http://download.qt-project.org/official_releases/qt/5.2/5.2.1/qt-opensource-windows-x86-msvc2012_64_opengl-5.2.1.exe
$
As seen above, I first downloaded the Qt package for Visual Studio in a Cygwin Bash shell.
A sidenote: The Qt library packaged within Cygwin is not useful for me because I need to use the Visual Studio C++ compiler.
First I set the correct permissions on the file
$ chmod 755 qt-opensource-windows-x86-msvc2012_64_opengl-5.2.1.exe
If I start it like this
$ ./qt-opensource-windows-x86-msvc2012_64_opengl-5.2.1.exe
a graphical window (GUI) is shown but that is not what I want as I would later like to have the installation procedure written into a Bash script that I could run in Cygwin.
If I add the option --help, like this
$ ./qt-opensource-windows-x86-msvc2012_64_opengl-5.2.1.exe --help
a new terminal window is opened with the following text
Usage: SDKMaintenanceTool [OPTIONS]
User:
--help Show commandline usage
--version Show current version
--checkupdates Check for updates and return an XML file describing
the available updates
--updater Start in updater mode.
--manage-packages Start in packagemanager mode.
--proxy Set system proxy on Win and Mac.
This option has no effect on Linux.
--verbose Show debug output on the console
--create-offline-repository Offline installer only: Create a local repository inside the
installation directory based on the offline
installer's content.
Developer:
--runoperation [OPERATION] [arguments...] Perform an operation with a list of arguments
--undooperation [OPERATION] [arguments...] Undo an operation with a list of arguments
--script [scriptName] Execute a script
--no-force-installations Enable deselection of forced components
--addRepository [URI] Add a local or remote repo to the list of user defined repos.
--addTempRepository [URI] Add a local or remote repo to the list of temporary available
repos.
--setTempRepository [URI] Set a local or remote repo as tmp repo, it is the only one
used during fetch.
Note: URI must be prefixed with the protocol, i.e. file:///
http:// or ftp://. It can consist of multiple
addresses separated by comma only.
--show-virtual-components Show virtual components in package manager and updater
--binarydatafile [binary_data_file] Use the binary data of another installer or maintenance tool.
--update-installerbase [new_installerbase] Patch a full installer with a new installer base
--dump-binary-data -i [PATH] -o [PATH] Dumps the binary content into specified output path (offline
installer only).
Input path pointing to binary data file, if omitted
the current application is used as input.
I don't know how to continue from here. Do you know how I could install the Qt 5.2.1 library (for Visual Studio) from the Bash shell in Cygwin?
Update: The advantage of writing the build script for a Cygwin environment is that commands like git, wget, and scp are available. This Stackoverflow answer describes how to invoke the MSVC compiler from a Cygwin bash script. Note, that the Qt application I'm building is not going to have any dependency on Cygwin.
I didn't test with Cygwin but I successfully installed Qt5.5 using a script. To do so, you must use the --script line of the normal installer.
.\qt-opensource-windows-x86-msvc2013_64-5.5.1.exe --script .\qt-installer-noninteractive.qs
Here's an example of qt-installer-noninteractive.qs file I used in the command above
function Controller() {
installer.autoRejectMessageBoxes();
installer.installationFinished.connect(function() {
gui.clickButton(buttons.NextButton);
})
}
Controller.prototype.WelcomePageCallback = function() {
gui.clickButton(buttons.NextButton);
}
Controller.prototype.CredentialsPageCallback = function() {
gui.clickButton(buttons.NextButton);
}
Controller.prototype.IntroductionPageCallback = function() {
gui.clickButton(buttons.NextButton);
}
Controller.prototype.TargetDirectoryPageCallback = function() {
gui.currentPageWidget().TargetDirectoryLineEdit.setText("C:/Qt/Qt5.5.1");
gui.clickButton(buttons.NextButton);
}
Controller.prototype.ComponentSelectionPageCallback = function() {
var widget = gui.currentPageWidget();
widget.deselectAll();
widget.selectComponent("qt.55.win64_msvc2013_64");
// widget.selectComponent("qt.55.qt3d");
// widget.selectComponent("qt.55.qtcanvas3d");
// widget.selectComponent("qt.55.qtquick1");
// widget.selectComponent("qt.55.qtscript");
// widget.selectComponent("qt.55.qtwebengine");
// widget.selectComponent("qt.55.qtquickcontrols");
// widget.selectComponent("qt.55.qtlocation");
gui.clickButton(buttons.NextButton);
}
Controller.prototype.LicenseAgreementPageCallback = function() {
gui.currentPageWidget().AcceptLicenseRadioButton.setChecked(true);
gui.clickButton(buttons.NextButton);
}
Controller.prototype.StartMenuDirectoryPageCallback = function() {
gui.clickButton(buttons.NextButton);
}
Controller.prototype.ReadyForInstallationPageCallback = function()
{
gui.clickButton(buttons.NextButton);
}
Controller.prototype.FinishedPageCallback = function() {
var checkBoxForm = gui.currentPageWidget().LaunchQtCreatorCheckBoxForm
if (checkBoxForm && checkBoxForm.launchQtCreatorCheckBox) {
checkBoxForm.launchQtCreatorCheckBox.checked = false;
}
gui.clickButton(buttons.FinishButton);
}
The tricky part was to found the id of the components! I was able to found the right id qt.55.win64_msvc2013_64 by adding the flag --verbose and installing it normally with the UI and stopping at the last page; all the ids that you selected for installation are there.
There is slightly more information in this answer if you need more details.
EDIT (29-11-2017): For installer 3.0.2-online, the "Next" button in the "Welcome" page is disabled for 1 second so you must add a delay
gui.clickButton(buttons.NextButton, 3000);
EDIT (10-11-2019): See Joshua Wade's answer for more traps and pitfalls, like the "User Data Collection" form and "Archive" and "Latest releases" checkboxes.

Resources