Disclaimer: I use jupyter kernel, but the question is also relevant for jupyter notebook.
According to jupyter kernel --help-all, I should be able to change the jupyter kernel JSON connection file by setting a parameter called --KernelManager.connection_file.
If I understand this correctly, that means that the following command:
jupyter kernel --KernelManager.connection_file=connection.json
should start a kernel and give me a connection file called connection.json.
However, this is what I get:
→ jupyter kernel --KernelManager.connection_file='test-this-thing.json'
[KernelApp] Starting kernel 'python3'
[KernelApp] Connection file: /Users/me/Library/Jupyter/runtime/kernel-1e65d0fe-bf8e-1234-8208-463bd4a1234a.json
Now, jupyter doesn't complain that I've passed a wrong argument nor anything, it just doesn't change the connection file.
Am I doing something wrong? How can I correctly change the connection filename?
Essentially, nothing you are doing in the above code is wrong. Previously the kernel overrode whatever you set as the connection file with a hard coded file location.
This has now been fixed as per the following pull requests:
https://github.com/jupyter/jupyter_client/pull/399
Removed the static connection file name declaration on kernelapp initialize method.
https://github.com/jupyter/jupyter_client/pull/432
Set the default connection_file such that it preserves an existing configuration.
I useful workaround to set the connection file is to not call jupyter kernel directly, but rather use the kernel manager module, which is more flexible:
python -m ipykernel_launcher -f ~/kernels/file.json
The above works for current and previous versions of jupyter, so I'd consider it to be more reliable.
Related
I'm trying to replicate the functionality of the code editor on a platform I was previously using called Odoo.sh. The platform would let me create a .ipynb notebook, but in the cells I could reference pre-set variables which required no boilerplate code inside of the notebook. Extremely convenient.
If you're familiar with Odoo, it was like having odoo-bin shell be implicitly run before executing any of the cells inside the notebook. It was wonderful to work with, but Odoo.sh is proprietary, so I'm trying to replicate the same functionality on my local machine.
A minimal example of what I'm going for here would be to have the following python code run before executing any of my .ipynb notebook file's cells.
example_value = False
def example_func():
global example_value
example_value = True
example_func()
So that inside of any notebook's cells I could simply run something like example_value and get an output of True.
In the case of Odoo.sh it almost seemed like there was a special custom kernel set up that was nothing more than a regular Python 3 kernel with some initialization code. This may be exactly what was going on, but I don't know enough about how Jupyter works to know for myself. How do I replicate this functionality?
I figured it out! You need to create a custom kernel, but for this use case you can just reuse the default IPython kernel and just pass some variables into the user namespace.
First, create a Python file for your kernel. Let's use test_kernel.py. Here are the contents:
from ipykernel.ipkernel import IPythonKernel
from ipykernel.kernelapp import IPKernelApp
if __name__ == "__main__":
example_value = False
def example_func():
global example_value
example_value = True
example_func()
IPKernelApp.launch_instance(
kernel_class=IPythonKernel,
user_ns={"example_value": example_value})
See how the arbitrary code from the question is run before launching the kernel instance. Using the user_ns argument, we can pass arbitrary data to the user environment.
To get our kernel up and running we need to make a test directory and then a test/kernel.json file. It will have these contents:
{
"argv": ["python", "-m", "test_kernel", "-f", "{connection_file}"],
"display_name": "Test"
}
Let's install that bad boy. Run jupyter kernelspec install --user test. In that command, test is the name of the directory we created. The --user argument makes Jupyter install the kernel only for the current user. You don't have to use it if you don't want to.
Now we should be good to go! Start things up with jupyter notebook and you will see your new kernel is available to use when using notebooks. And check it out, we can see the variable we passed into the namespace:
Last of all, be sure to note that in order for this to work your test_kernel.py file will need to be somewhere where Python can import it. I'm not an expert on this, but from a bit of Googling I took this to mean that the directory containing the file should either be the current working directory or be in your PYTHONPATH.
I want to view an image in Jupyter notebook. It's a 9.9MB .png file.
from IPython.display import Image
Image(filename='path_to_image/image.png')
I get the below error:
IOPub data rate exceeded.
The notebook server will temporarily stop sending output
to the client in order to avoid crashing it.
A bit surprising and reported elsewhere.
Is this expected and is there a simple solution?
(Error msg suggests changing limit in --NotebookApp.iopub_data_rate_limit.)
Try this:
jupyter notebook --NotebookApp.iopub_data_rate_limit=1.0e10
Or this:
yourTerminal:prompt> jupyter notebook --NotebookApp.iopub_data_rate_limit=1.0e10
I ran into this using networkx and bokeh
This works for me in Windows 7 (taken from here):
To create a jupyter_notebook_config.py file, with all the defaults commented out, you can use the following command line:
$ jupyter notebook --generate-config
Open the file and search for c.NotebookApp.iopub_data_rate_limit
Comment out the line c.NotebookApp.iopub_data_rate_limit = 1000000 and change it to a higher default rate. l used c.NotebookApp.iopub_data_rate_limit = 10000000
This unforgiving default config is popping up in a lot of places. See git issues:
jupyter
IOPub data rate exceeded
It looks like it might get resolved with the 5.1 release
Update:
Jupyter notebook is now on release 5.2.2. This problem should have been resolved. Upgrade using conda or pip.
Removing print statements can also fix the problem.
Apart from loading images, this error also happens when your code is printing continuously at a high rate, which is causing the error "IOPub data rate exceeded". E.g. if you have a print statement in a for loop somewhere that is being called over 1000 times.
By typing 'jupyter notebook --NotebookApp.iopub_data_rate_limit=1.0e10' in Anaconda PowerShell or prompt, the Jupyter notebook will open with the new configuration. Try now to run your query.
Some additional advice for Windows(10) users:
If you are using Anaconda Prompt/PowerShell for the first time, type "Anaconda" in the search field of your Windows task bar and you will see the suggested software.
Make sure to open the Anaconda prompt as administrator.
Always navigate to your user directory or the directory with your Jupyter Notebook files first before running the command. Otherwise you might end up somewhere in your system files and be confused by an unfamiliar file tree.
The correct way to open Jupyter notebook with new data limit from the Anaconda Prompt on my own Windows 10 PC is:
(base) C:\Users\mobarget\Google Drive\Jupyter Notebook>jupyter notebook --NotebookApp.iopub_data_rate_limit=1.0e10
I have the same problem in my Jupyter NB on Win 10 when querying from a MySQL database.
Removing any print statements solved my problem.
For already running docker containers, try editing the file name - ~/.jupyter/jupyter_notebook_config.py
uncomment the line - NotebookApp.iopub_data_rate_limit =
and set high number like 1e10.
Restart the docker, it should fix the problem
I ran into this problem running version 6.3.0. When I tried the top rated solution by Merlin the powershell prompt notified me that iopub_data_rate_limit has moved from NotebookApp to ServerApp. The solution still worked but wanted to mention the variation, especially as internal handling of the config may become deprecated.
Easy workaround is to create a for loop and print. Then there wont be any issue. Printing directly wcc would cause if graph is huge. Hence any of below code will work as workaround.
wcc=list(nx.weakly_connected_components(train_graph))
for i in range(1,10):
print(wcc[i])
for i in wcc):
print(wcc)
Like others pointed out, print statement at a high rate can cause this. Resolve it by printing modulo a number using if statement. Example in python:
k = 10
if (i % k == 0):
print("Something")
Increase k if the warning persists.
Using Visual Studio Code, the Jupyter extension will be able to handle big data. launch from anaconda navigator
In general, trying to print something that is too long will trigger this error. I tried to print a string that was 9221593 characters long (too long), and that triggered the error.
I'm doing a small research on IPython kernel and trying to get debug logs out of it and see how it interacts with a notebook. Now it looks like the documentation and example configs shipped in my distribution is totally outdated.
The questions
Where the ipython kernel log files are located?
How can I enable DEBUG level logging in both jupyter notebook and ipython kernel?
What I've tried
Please read this section before giving links to the official docs
First I created profiles for both IPython and notebook with the following commands:
$ ipython profile create
$ jupyter notebook --generate-config
As expected three files where created:
.jupyter/jupyter_notebook_config.py
.ipython/profile_default/ipython_config.py
.ipython/profile_default/ipython_kernel_config.py
In these files I found similar commented fragments:
# Set the log level by value or name.
# c.Application.log_level = 0
I tried to uncomment it in the jupyter config. Instead of adding more details it totally disabled console output for the jupyter process. I also tried value 50 it has the same result, value DEBUG gave me Python error on start.
I also played with these values in ipython's configs but I wasn't able to find log files location.
In mail list command line option --log-level=DEBUG is mentioned and indeed it works for jupyter. But I really want to persist this setting in a profile and have debug info for the kernel too.
Config options NotebookApp.log_level and IPKernelApp.log_level also change nothing.
I believe that this kind of functionality is still on the wishlist:
https://github.com/ipython/ipython/issues/8570
But you could try something like this:
jupyter notebook --debug > log.file 2>&1
or
ipykernel kernel --debug > log.file 2>&1
You can also try to start ipython kernel without attached frontend with option --debug:
ipython kernel --debug
You can get lot of info about interaction between kernel and the forntend by setting c.Session.debug = True in jupyter_notebook_config.py.
Gaurav Pandey
I used the command
Sys.setenv(http_proxy="http://myusername:pswd#host:port")
and it worked at first.
But on restarting It in the Terminal it showed the following error.-
cannot open: HTTP status was '407 Proxy Authentication Required
How to resolve this issue.Help,,,
You set the environment for the process running R. When the instance of R exits, the process goes away, and the environment is gone.
You cannot set the environment for a parent process.
Put these commands in /etc/profile or ~/.bash_profile instead:
export http_proxy
http_proxy=http://myusername:pswd#host:port
Or all on one line (for bash):
export http_proxy=http://myusername:pswd#host:port
Or run the commands in the shell before starting R.
OS: UNIX Solaries, Oracle Application Server 10g
To run shell script from Oracle Forms, I used the following host('/bin/bash /u01/compile.sh') and it works well
Now, I need to run unix command something like
host('mv form1.fmx FORM1.FMX') but it's not working
I tried to append the command mv form1.fmx FORM1.FMX' to the compile.sh shell script but also it's not working although the rest lines of the shell script is running well
The solution is to just add the full path of the mv command and it worked well, as follow
/bin/mv /u01/oracle/runtime/test/form1.fmx /u01/oracle/runtime/test/FORM1.FMX
In case anyone else encounters the same problem, the cause is that Forms process creates a subprocess to execute host() command, and that subprocess inherits environment variables of the parent process, which are derived from default.env (or other env file as defined in server config). There is a PATH variable defined in that file, but it doesn't contain usual /bin or /usr/bin, so the commands will not execute unless full path is specified.
The solution is to set the correct PATH variable either in the executed script (via export PATH=$PATH:...) or in default.env. I set it in the script, since, knowing Oracle, there's no guarantee that modifying default.env won't break something.