In Jupyter with an ipython kernel, is there a canonical way to execute cells in a non-blocking fashion?
Ideally I'd like to be able to run a cell
%%background
time.sleep(10)
print("hello")
such that I can start editing and running the next cells and in 10 seconds see "hello" appear in the output of the original cell.
I have tried two approaches, but haven't been happy with either.
(1) Create a thread by hand:
def foo():
time.sleep(10)
print("hello")
threading.Thread(target=foo).start()
The problem with this is that "hello" is printed in whatever cell is active in 10 seconds, not necessarily in the cell where the thread was started.
(2) Use a ipywidget.Output widget.
def foo(out):
time.sleep(10)
out.append_stdout("hello")
out = ipywidgets.Output()
display(out)
threading.Thread(target=foo,args=(out,)).start()
This works, but there are problems when I want to update the output (think of monitoring something like memory consumption):
def foo(out):
while True:
time.sleep(1)
out.clear_output()
out.append_stdout(str(datetime.datetime.now()))
out = ipywidgets.Output()
display(out)
threading.Thread(target=foo,args=(out,)).start()
The output now constantly switches between 0 and 1 lines in size, which results in flickering of the entire notebook.
This should be solvable wait=True in the call to clear_output. Alas, for me it results in the output never showing anything.
I could have asked about that issue, which seems to be a bug, specifically, but I wondered whether there is maybe another solution that doesn't require me doing all of this by hand.
I've experienced some issues like this with plotting to an output, it looks like you have followed the examples in the ipywidgets documentation on async output widgets.
The other approach I have found sometimes helpful (particularly if you know the size of the desired output) is to fix the height of your output widget when you create it.
out = ipywidgets.Output(layout=ipywidgets.Layout(height='25px'))
Related
Occasionally, for reasons I can not determine, ess-indent-or-complete sends an input to the R console. In other words, I will be typing something, hit tab to autocomplete (or company autocompletes for me), and the R process will update with a newline (.R file on the left "buffer", R process on the right):
x <- mea█ | >
|
|
... and then I hit tab to complete "mean":
x <- mean█ | >
| > ## note the new line here
|
This can be really annoying for at least two reasons:
It fills up the process buffer with blank lines.
If I am using a browser(), it advances lines which is often undesirable.
I have tried to diagnose this myself to no avail. Restarting emacs fixes it every time, however it seems to start happening at random throughout the day. Once it starts it does not stop until emacs restarts.
I am running emacs 27.1, ess-20210818.843 (though this has been a thing for a long time, so I don't believe it is version-specific), and am in ESS[R] mode.
Calling M-: (ess-command "") from my .R file buffer replicates the undesired behavior once it starts happening in a particular session. I have no insight beyond this.
I think you solved it yourself, with your GH issue suggestion:
https://github.com/emacs-ess/ESS/issues/1198
I've changed things in my own version of ESS, and will commit after a few days of testing it.
Thank you VERY VERY much for persevering and (I think) finding the fault .. a simple typo inside the internal (ess--command-make-restore-function)
How can I define a simple cell magic that just executes the cell as if the %%mymagic wasn't there?
The context is that we are using the wonderful IPython parallel framework. In some places, we also use its defined %%px magic. But sometimes we'd like to run the same notebook without a cluster (local only). In that case, %%px isn't defined and I would have to comment it out. Instead, in that case I'd like to redefine %%px so that:
%%px: would be a no-op.
%%px --local: just runs the cell, no other side-effect.
Alternatively, all %%px (with --local or not) could just run the cell, if that's simpler.
Another approach would be to create an ipyparallel Client that is a fake one, i.e. with 0 nodes (but would still operate correctly, e.g. with regard to %%px --local). But that would be for another question.
Things I've tried:
%alias_magic px time (after all, I don't care if the cell is timed). Unfortunately, %%time doesn't take arguments and chokes on --local.
Define my own "no-op" magic:
if USE_CLIENT:
pass
else:
# temporarily define %%px cell magic
from IPython import get_ipython
def px(line, cell):
"""Do nothing"""
pass
get_ipython().register_magic_function(px, 'cell')
but that succeeds a little too well at doing really nothing (i.e. the cells are not executed).
Look into IPython/core/magics/execution.py to see if there is any hook I could reuse (something that would just execute the cell). I haven't found, but perhaps I haven't looked hard enough.
Any other idea?
I think the relevant command is
self.shell.run_cell(cell)
We can define a magic command that just does not have effect as:
def f(line, cell):
print('==> line:[{}]'.format(line))
print('==> cell:\n # {}'.format('\n # '.join(cell.split())))
print('==================================================================')
res = get_ipython().run_cell(cell)
get_ipython().register_magic_function(f, 'cell', 'cache')
Here is an example run:
In your case, try:
if USE_CLIENT:
pass
else:
# temporarily define %%px cell magic
from IPython import get_ipython
def px(line, cell):
res = get_ipython().run_cell(cell)
get_ipython().register_magic_function(px,'cell','px')
I'm doing long-running experiments in an IPython notebook running on a server, where the typical work cycle is: launch experiment, go to lunch, come back, check progress, check Facebook, check email, check Facebook again, turn off computer, come back, check Facebook, check progress, ...
The problem is that when I close the browser window where the notebook is running, the print/logging outputs are no longer saved in the notebook.
For instance, in my simple experiment:
import time
start_time = time.time()
for i in xrange(5):
print '%s seconds have passed' % (time.time()-start_time)
time.sleep(2)
print 'Done!'
If I run, close the tab, and come back 10 seconds later, I just see whatever the output was when the notebook was last saved. What I expect to see is:
0.000111818313599 seconds have passed
2.00515794754 seconds have passed
4.01105999947 seconds have passed
6.0162498951 seconds have passed
8.01735782623 seconds have passed
Done!
Presumably this will be fixed at some point in the future, but in the mean time is there some easy hack to make it continue printing to notebook output after closing the browser? Bonus points if it works for inline images.
Well, found an ok solution. Solution is in this file:
https://github.com/QUVA-Lab/artemis/blob/master/artemis/fileman/persistent_print.py
With example use:
https://github.com/QUVA-Lab/artemis/blob/master/artemis/fileman/test_persistent_print.py
The demo now looks like:
import time
from general.persistent_print import capture_print, reprint
capture_print()
start_time = time.time()
for i in xrange(5):
print '%s seconds have passed' % (time.time()-start_time)
time.sleep(2)
print 'Done!'
And if I run
reprint()
In the next cell, it will redisplay all the print statements made since capture_print was called. Obviously it would be better if this were unnecessary, but it works for now.
I am trying to pause my code for a little while, time for me to observe the plots.
I tried:
print('A')
something = readline("Press Enter")
print('B')
print('C')
, then there is no pause, the line print('B') is fed to readline and get stored into something and therefore only A and C got printed on the screen. Note that if I add an empty line between Something = readline("Press Enter") and print("B"), then print("B") get printed on the screen but still the console doesn't allow the user to press enter before continuing.
And I tried:
print('A')
Sys.sleep(3)
print('B')
print('C')
The program waits 3 seconds before starting and then run "normally" without doing any pause between print('A') and print('B').
What do I missunderstand?
Here is my R version: R 3.1.1 GUI 1.65 Snow Leopard build (6784)
The problem with readline is that if you paste your script into an R console, or execute it from eg Rstudio, the redline function is read and then the next line of the script is read in as the console entry, which in your case sets the value of something to print('B).
An easy way to get around this is to stick your entire code in a function, then call the function to run it. So, in your case:
myscript = function(){
print('A')
something = readline(prompt = "Press Enter")
print('B')
print('C')
}
myscript()
The output of this for me (in Rstudio, with R version 3.1.1):
[1] "A"
Press Enter
[1] "B"
[1] "C"
This has always felt like a bit of a hack to me, but it's essentially what the readline documentation recommends in its example.
I've never used sleep in my code, so I can't help you there.
Edit to clarify based on comments: This will only work if myscript() is the very last line of your script, or if it is manually entered into the console after running the script to generate the function. Otherwise, you will run into the same problem as before- the next line of code will be automatically entered.
Sometimes my Ipython notebooks crash because I left a print statement in a big loop or in a recursive function. The kernel shows busy and the stop button is usually unresponsive. Eventually Chrome asks me if I want to kill the page or wait.
Is there a way to limit the number of output lines in a given cell? Or any other way to avoid this problem?
You can suppress output using this command:
‘;’ at the end of a line
Perhaps create a condition in your loop to suppress output past a certain threshold.
For anyone else stumbling across:
If you want to see some of the output rather than suppress the output entirely, there is an extension called limit-output.
You'll have to follow the installation instructions for the extensions at the first link. Then I ran the following code to update the maximum number of characters output by each cell:
from notebook.services.config import ConfigManager
cm = ConfigManager().update('notebook', {'limit_output': 10})
Note: you'll need to run the block of code, then restart your notebook server entirely (not just the kernel) for the change to take effect.
Results on jupyter v4.0.6 running a Python 2.7.12 kernel
for i in range(0,100):
print i
0
1
2
3
4
limit_output extension: Maximum message size exceeded