I am trying to integrate interactive ipywidgets with a loop in my code that also performs other tasks (in this case, acquiring data from some hardware attached from the computer and updating live plots).
In the past, I could do this by using IPython.kernel.do_one_iteration() in my while loop: this would trigger a sync of the ipywidget changes and I would be able to retrieve them from the python widget objects. A minimal example is here:
import ipywidgets as widgets
from time import sleep
import IPython
do_one_iteration = IPython.get_ipython().kernel.do_one_iteration
w = widgets.ToggleButton()
display(w)
i=0
while True:
do_one_iteration()
print(i, w.value, end="\r")
w.decription = str(i)
sleep(0.5)
i+=1
Here, the for loop prints out the ticker integer along with the state of the widget. (In the real code, I would also acquire data, update plots, and change plot / acquisition settings dependent on the interaction with the user via the widgets.)
With ipykernel 5.3.2 and ipython 7.16.1, this worked fine: if the widget changed, calling do_one_iteration() synced the widget states to the kernel and I could retrieve it from my while loop.
After an upgrade (to 6.4.1 and 7.29.0), this no longer works. It seems that do_one_iteration() is now a coroutine: I get a warning coroutine 'Kernel.do_one_iteration' was never awaited if I use the above code.
With some help of a friend, we found a way to do this with threading an asyncio:
%gui asyncio
import asyncio
import ipywidgets as widgets
button = widgets.ToggleButton()
display(button)
text = widgets.Text()
display(text)
text.value= str(button.value)
stop_button = widgets.ToggleButton()
stop_button.description = "Stop"
display(stop_button)
async def f():
i=0
while True:
i += 1
text.value = str(i) + " " + str(button.value)
await asyncio.sleep(0.2)
if stop_button.value == True:
return
asyncio.create_task(f());
And this works (also adding a stop button, and changing to text output widget instead of printing). But to throw a spanner in the works, I need to use a library that itself uses a QT gui event loop. Some more puzzling suggests that this should be the code to make this work:
%gui qt5
import asyncio
import ipywidgets as widgets
import qasync
button = widgets.ToggleButton()
display(button)
text = widgets.Text()
display(text)
text.value= str(button.value)
stop_button = widgets.ToggleButton()
stop_button.description = "Stop"
display(stop_button)
async def f():
while True:
i += 1
text.value = str(i) + " " + str(button.value)
await asyncio.sleep(0.2)
if stop_button.value == True:
return
from qtpy import QtWidgets
APP = QtWidgets.QApplication.instance()
loop = qasync.QEventLoop(APP)
asyncio.set_event_loop(loop)
asyncio.create_task(f());
But with this code, the updates do not propagate, and I get the following error on the terminal running my notebook server:
[IPKernelApp] ERROR | Error in message handler
Traceback (most recent call last):
File "/Users/gsteele/anaconda3/envs/myenv2/lib/python3.9/site-packages/ipykernel/kernelbase.py", line 457, in dispatch_queue
await self.process_one()
File "/Users/gsteele/anaconda3/envs/myenv2/lib/python3.9/site-packages/ipykernel/kernelbase.py", line 440, in process_one
t, dispatch, args = await self.msg_queue.get()
RuntimeError: Task <Task pending name='Task-2'
coro=<Kernel.dispatch_queue() running at
/Users/gsteele/anaconda3/envs/myenv2/lib/python3.9/site-packages/ipykernel/kernelbase.py:457>
cb=[IOLoop.add_future.<locals>.<lambda>() at /Users/gsteele/anaconda3/envs/myenv2/lib/python3.9/site-packages/tornado/ioloop.py:688]>
got Future <Future pending> attached to a different loop
It seems that somehow my ipywidgets events are propagating to the wrong event loop.
And now my question is: does anybody know what is going on here?
It's hard for me to identify if this is a "bug", and if so, in which software package do things go wrong? ipykernel? Or tornado? Or ipywidgets? Or asyncio? Or maybe I'm missing something?
Any thoughts highly welcome, thanks!
Found at least a partial solution: using the nest_asyncio package allows me to now use do_one_iteration(), just by adding the following to the first code block:
import nest_asyncio
nest_asyncio.apply()
and then using await do_one_iteration() instead of calling it directly.
(see https://github.com/ipython/ipykernel/issues/825)
For my purposes, this solves my issue since I don't need asynchronous interaction with my GUI. The problem of the %gui qt5 interaction with the event loop in the asynchronous versions of the code is still a mystery though...
Related
Let's say I have a ui file created in Qt Designer that I want to load dynamically to then manipulate the widgets, such as:
example.py:
from PyQt5 import QtWidgets, uic
class MyWidget(QtWidgets.QWidget):
def __init__(self, parent=None):
super(MyWidget, self).__init__(parent)
uic.loadUi('example.ui', self)
# No code completion here for self.myPushButton:
self.myPushButton.clicked.connect(self.handleButtonClick)
self.show()
Is there a standard / convenient way of enabling code completion for the widgets loaded this way in PyCharm (2017.1.4)?
At the moment I am using this (written in the constructor after the ui file is loaded):
self.myPushButton = self.myPushButton # type: QtWidgets.QPushButton
# Code completion for myPushButton works at this point
I also thought of this, but it does not seem to do the trick:
assert isinstance(self.myPushButton, QtWidgets.QPushButton)
# PyCharm does not even recognise myPushButton as an attribute of self at this point
Finally, I also thought of using python stubs, such as:
example.pyi:
class MyWidget(QtWidgets.QWidget):
def __init__(self):
self.myPushButton: QtWidgets.QPushButton = ...
However, myPushButton is properly recognised in code outside example.py but not in code inside example.py itself, which is kind of the opposite of what I wanted.
I am also considering taking my first approach but with all those lines put in a private method that will never get called, such as:
example.py:
from PyQt5 import QtWidgets, uic
class MyWidget(QtWidgets.QWidget):
def __init__(self, parent=None):
super(MyWidget, self).__init__(parent)
uic.loadUi('example.ui', self)
# Code completion now works here for self.myPushButton:
self.myPushButton.clicked.connect(self.handleButtonClick)
self.show()
def __my_private_method_never_called():
self.myPushButton = self.myPushButton # type: QtWidgets.QPushButton
# Or even this (it should have the same effect if this
# function is never called, plus it is less verbose):
self.myPushButton = QtWidgets.QPushButton()
# If I want to make sure that this is never called
# could raise an error at some point:
raise YouShouldNotHaveCalledThisError()
This seems to work fine, and it also allows me to group all my type hinting code together, isolated from the rest. I could even make some script to write all those lines for me by parsing the ui files. I am just wondering if people reading my code would find this approach very unorthodox, even if I comment clearly why am I writing a technically useless private function.
If anybody is interested, I made the script I mentioned to parse the .ui files and generate stub code ready to be copied to my class:
ui_stub_generator.py:
from __future__ import print_function
import os
import sys
import xml.etree.ElementTree
def generate_stubs(file):
root = xml.etree.ElementTree.parse(file).getroot()
print('Stub for file: ' + os.path.basename(file))
print()
print(' def __stubs(self):')
print(' """ This just enables code completion. It should never be called """')
for widget in root.findall('.//widget'):
name = widget.get('name')
if len(name) > 3 and name[:2] == 'ui' and name[2].isupper():
cls = widget.get('class')
print(' self.{} = QtWidgets.{}()'.format(
name, cls
))
print(' raise AssertionError("This should never be called")')
print()
def main():
for file in sys.argv[1:]:
generate_stubs(file)
if __name__ == '__main__':
main()
This only parses widgets whose names start with 'ui' followed by an uppercase letter, such as 'uiMyWidget', which is the naming convention that I typically follow in the Qt Designer. By doing this, the widgets with names automatically generated by the Qt Designer are ignored (if I cared about these, I would have given them a proper name). It should be straightforward to update this for any other naming conventions, or other type of objects, such as actions.
For convenience, I have set this up as an external tool in PyCharm as well; see screenshot here (change the paths as appropriate). That way, I only have to right-click my ui file in the project window, then External Tools -> Stub Generator for Qt UI Files, and I get the following output in the Run window ready to be copied:
C:\ProgramData\Anaconda3\python.exe D:\MyProject\bin\ui_stub_generator.py D:\MyProject\my_ui_file.ui
Stub for file: my_ui_file.ui
def __stubs(self):
""" This just enables code completion. It should never be called """
self.uiNameLabel = QtWidgets.QLabel()
self.uiOpenButton = QtWidgets.QPushButton()
self.uiSplitter = QtWidgets.QSplitter()
self.uiMyCombo = QtWidgets.QComboBox()
self.uiDeleteButton = QtWidgets.QPushButton()
raise AssertionError("This should never be called")
Process finished with exit code 0
In my pytest script, I need to customize the pytest-HTML report for capturing the stdout at the same time writing into the console as I have user input in my automated test.
test_TripTick.py
import os
import sys
import pytest
from Process import RunProcess
from recordtype import recordtype
from pip._vendor.distlib.compat import raw_input
#pytest.fixture(scope="module")
def Process(request):
# print('\nProcess setup - module fixture')
fileDirectory = os.path.abspath(os.path.dirname(__file__))
configFilePath = os.path.join(fileDirectory, 'ATR220_Config.json')
process = RunProcess.RunProcess()
process.SetConfigVariables(configFilePath)
process.GetComPort(["list=PID_0180"])
def fin():
sys.exit()
request.addfinalizer(fin)
return process
def test_WipeOutReader(Process):
assert Process.WipeOutTheReader() == True
def test_LoadingKeysIntoMemoryMap(Process):
assert Process.LoadingKeysIntoMemoryMap() == True
def test_LoadingFW(Process): # only use bar
assert Process.LoadingFW() == True
def test_LoadingSBL(Process):
assert Process.LoadingSBL() == True
def test_CCIDReadForPaymentCards(Process):
assert Process.CCIDReadWrite('Payment Card') == True
Currently, if I run the following command from the windows command line, I get output on the console, but no captured output on the HTML report.
pytest C:\Projects\TripTickAT\test_TripTick.py -s --html=Report.html --verbose
Also, I would like to know the programmatic way of customizing the HTML report where I can update the file name, ordering test based on time of the execution and capturing the std-out.
I have tried additional flags to the pytest command. --capture sys and
-rP for Passed tests. --capture sys and -rF for failed tests
And I can see the console log as well in the html document after I click show All details button as shown in the output. I have captured only partial screen for the purpose of showing to you. But you can scroll down the output to see all the console logs. The grey area in the image below is the console output. I am not sure of the command line level flag that works regardless of failed or passed tests. But here is a temporary solution. This will print logs on command line console as well as html logs console
`
I'm developing a visualization using
% bokeh serve --show myapp.py
The problem is that when I change myapp.py I have to kill the above command and restart it. Is there a better workflow for this kind of development?
Thanks!
Not yet. As of Bokeh 0.11.1 (and soon to be 0.12 too), this is a planned, but still open, feature request There are only a few folks working a huge pile of work for Bokeh. New contributors can help accelerate new features and fixes. If you're able to help, please reach out to us on the project gitter channel.
I was unable to understand enough of the bokeh internals to make this work nicer, but here is a hacky script that does what I want anyway.
# bokeh_watcher.py
#
# Watches specific files in directory and restarts bokeh server upon change.
#
# % python bokeh_watcher filename.py
#
# Note that you stil have to navigate your browser to localhost:5006/filename
# to see your Bokeh visualization and you might have to refresh the browser.
import sys
import time
import logging
from watchdog.observers import Observer
from watchdog.events import RegexMatchingEventHandler
from bokeh.command.bootstrap import main
import multiprocessing
import os
JOBS = []
FILE = []
def spawn_bokeh(args):
main(args)
class BokehHandler(RegexMatchingEventHandler):
'''
kills and restarts bokeh server upon filechange.
'''
def on_modified(self, event):
super(BokehHandler, self).on_modified(event)
what = 'directory' if event.is_directory else 'file'
logging.info("Modified %s: %s"% (what, event.src_path))
p=JOBS.pop()
p.terminate()
time.sleep(1) # time to die
logging.info('terminated')
logging.info('initiating restart')
p = multiprocessing.Process(target=spawn_bokeh,
args=(self.args,))
p.start()
JOBS.append(p)
if __name__ == "__main__":
here = os.path.realpath(__file__)
fullpathname=os.path.dirname(here)+os.sep+sys.argv[1]
# local logger
logging.basicConfig(level=logging.INFO,
format='%(asctime)s - %(message)s',
datefmt='%Y-%m-%d %H:%M:%S')
filemod_handler = BokehHandler(['.*%s'%(sys.argv[1])])
filemod_handler.args = ['','serve',fullpathname, '--log-level','info']
# fire up bokeh server
p = multiprocessing.Process(target=spawn_bokeh,args=(filemod_handler.args,))
p.start()
# store object in global for later
JOBS.append(p)
observer = Observer()
observer.schedule(filemod_handler, '.', recursive=False)
observer.start()
try:
while True:
time.sleep(3)
except KeyboardInterrupt:
observer.stop()
observer.join()
I am working on a program that will allow someone to enter details in order to write a CV. I am using the Tkinter module (as extra practice) but am already stuck on the menu!
At the moment I have three different options the user can choose: Write CV, Review CV and Exit. I have created a button for each option and when the user presses the button it'll open, however the menu window remains open (there is a different subroutine for each option).
I understand that you need to do something like window.destroy(), however I'm not sure how to give a button two commands without doing something too fiddly like create more subroutines etc.?
The other option I think I'd prefer is is I could clear the menu screen?
Here is the programming I have at the moment:
def Main_Menu():
import tkinter
main_menu = tkinter.Tk()
main_menu.title("CV Writer")
main_menu.geometry("300x300")
main_menu.wm_iconbitmap('cv_icon.ico')
title = tkinter.Label(main_menu, text = "Main Menu", font=("Helvetica",25))
title.pack()
gap = tkinter.Label(main_menu, text = "")
gap.pack()
write_cv = tkinter.Button(main_menu, text = "1) Write CV", font=("Helvetica"), command=Write_CV)
write_cv.pack()
review_cv = tkinter.Button(main_menu, text = "2) Review CV", font=("Helvetica"), command=Review_CV)
review_cv.pack()
leave = tkinter.Button(main_menu, text = "3) Exit", font=("Helvetica"), command=Exit)
leave.pack()
main_menu.mainloop()
def Write_CV():
import tkinter
write_cv = tkinter.Tk()
write_cv.geometry("300x300")
write_cv.title("Write CV")
def Review_CV():
import tkinter
review_cv = tkinter.Tk()
review_cv.geometry("300x300")
review_cv.title("Review CV")
def Exit():
import tkinter
leave = tkinter.Tk()
leave.geometry("300x300")
leave.title("Exit")
Main_Menu()
Running the program should help make this question make more sense!
I am so sorry for the wordy question, but any kind of help would be appreciated! Please bear in mind I am only a GCSE student so simple language would also be so nice! Thank you!
I don't know why are you importing tkinter under each method, it's completely useless. Simply import it once at the beginning of your file with a syntax like this:
import tkinter as tk
So that you can refer to the widgets simply with the duo tk:
btn = tk.Button(None, text='I can simply refer to a widget with tk')
Apart from this, the structure of your program is really bad. In my opinion, you should not instantiate Tk inside your function Main_Menu, because it will only be visible inside it. If you want to refer to the master or root or whatever you want to call the instance of Tk, you can't, because it's a local instance, as I said above.
I usually instantiate Tk in the main function of my program, or in the following if __name__ == '__main__': construct:
if __name__ == '__main__':
master = tk.Tk() # note I am using "tk"
# create your objects or call your functions here
master.mainloop()
Your are creating an instance of Tkin each of your function, that is really a bad practice, never do that. You should only create one instance of Tk for each Tkinter application.
You should use the object-oriented paradigm or make all your widgets global to structure your application.
Except these details, you can simply call master.destroy() when you want to destroy your main window and all its children widgets, where master is the Tk instance.
In general, you have a lot of errors and bad practices. My advice is:
Read a tutorial on Python first and then on Tkinter, before
proceeding.
I am enjoying developing inside the ipython notebook, but I am having a problem when I want to write a main() function that reads the command line args (with OptionParser, for example). I want to be able to export the code to a .py file and run it from the command line, but I haven't found a way to have a main() that runs both in the notebook with predefined arguments or from the command line with python and command line args. What is the secret?
In case that is not clear, I would like to do something like this:
if __name__ == '__main__':
# if in the notebook
vals = {'debug':True, 'tag_file': 't.tags'}
options = Object()
for k,v in vals.items():
options.setattr(k,v)
args = 'fname1.txt'
# if running as a command line python script
from optparse import OptionParser
parser = OptionParser()
parser.add_option('-d','--debug',action='store_true',dest='debug')
parser.add_option('-t','--tags',action='store',dest='tag_file')
options,args = parser.parse_args()
You cannot determine that you are in an IPython notebook or a qtconsole, or a simple IPython shell, for the simple reason the 3 can be connected to the same kernel at the same time.
It would be like asking, what color is the current key the user is typing. You could get it by looking the plugged usb devices and look for images on the internet and guess the keyboard color, but nothing guarantees you it will be accurate, nor that it won't change, and user can have multiple keyboard plugged, or even painted keyboard.
It is really the same with the notebook, Even if you determine you are in ZMQKernel, are you speeking to qtconsole or webserver ? Again, you found that you were talking to the webserver, are you talking to JS or Emacs ? And so on and so forth.
The only thing you can do, you can ask the user.
What is reliable, is test wether you are in IPython or not.
If you really but reeaaalllyy want a way, as until now, the notebook is the only thing that can display Javascript. And javascript can execute code in pyton. So you might be able to create something that display JS that send back info to the kernel. And using thread and timer you can say that you were not in a notebook (but you will have a race condition).
Don't worry about the distinction. Just set default values, and unless they are overridden from the command line, use those.
if __name__ == '__main__':
parser = OptionParser()
parser.add_option('-d', '--debug', action='store_true', dest='debug',
default=True)
parser.add_option('-t','--tags',action='store',dest='tag_file',
default='t.tags')
options, args = parser.parse_args()
if not args:
args = ['fname1.txt']