I can set the unknowns and residuals in components' solve_nonlinear function. Can I also set the values of the params? Why or why not?
Edit
Here is my attempt at a "pure python" reader/writer component. My problem is that I can't read/write parameters from the top level.
$ cat test.py
from openmdao.api import Component, Group, Problem
class reader():
def __init__(self):
self.file_to_read = 'test.in'
self.file_data = 0
def execute(self):
dat = open(self.file_to_read, 'r')
self.file_data = dat.read()
class writer():
def __init__(self):
self.file_to_write = 'test.out'
self.data = -99
def execute(self):
dat = open(self.file_to_write, 'w')
dat.write(str(self.data))
class ReadWriteComp(Component):
def __init__(self):
super(ReadWriteComp, self).__init__()
self.reader = reader()
self.writer = writer()
self.reader.execute()
def solve_nonlinear(self, params, unknowns, resids):
self.writer.data = self.reader.file_data
self.writer.execute()
root = Group()
root.add('testio', ReadWriteComp())
prob = Problem(root)
prob.setup()
prob['testio.writer.file_to_write'] = 'newname' # "Variable 'testio.writer.file_to_write' not found."
prob.run()
$ cat test.in
8
Params are incoming values to a component. They are externally provided information. You can't/shouldn't change them because of this externality.
said another way:
If you have incoming connections, then your param's value is defined by the output (source) of that upstream component. Changing your param would be like changing the output of that upstream component.
I can't make a comment since my rep is too low, but this is more of a comment to the read/write components and not an answer to the question. I'd suggest to make the read/write and wrapper classes pure python and then only use openmdao at the top level with a component executing all three, perhaps with a group wrapped around if you need to parallelize cases.
Related
I implement a label that displays the current time in several of my PyQt5 applications. Here is a MRE:
import sys
import logging
from PyQt5 import QtWidgets, QtCore, QtGui
__log__ = logging.getLogger()
class App(QtWidgets.QApplication):
def __init__(self, sys_argv):
super(App, self).__init__(sys_argv)
self.main_view = MainView()
self.main_view.show()
class MainView(QtWidgets.QMainWindow):
def __init__(self):
super().__init__()
self.setObjectName("MreUi")
self.resize(300, 100)
self.setWindowTitle('MreUi')
self.label = QtWidgets.QLabel()
self.setCentralWidget(self.label)
config_clock(self.label)
self.start_clocks()
def start_clocks(self):
timer = QtCore.QTimer(self)
timer.timeout.connect(self.show_time)
timer.start(1000)
def show_time(self):
current_time = QtCore.QTime.currentTime()
clock_label_time = current_time.toString('hh:mm:ss')
self.label.setText(clock_label_time)
def config_clock(label):
label.setAlignment(QtCore.Qt.AlignCenter)
font = QtGui.QFont('Arial', 24, QtGui.QFont.Bold)
label.setFont(font)
if __name__ == '__main__':
logging.basicConfig()
app = App(sys.argv)
try:
sys.exit(app.exec_())
except Exception as e:
__log__.error('%s', e)
As I implemented a similar clock in several of my PyQt apps, I thought it would be nice to implement it as a component / encapsulate it. First I thought of doing this by calling a config_clock function from any QWidget, and have that function do ~ALL of the work implementing the clock for the specified label. This would avoid having to repeat myself in multiple applications from writing/calling start_clocks and show_time instance methods of MainView. but as I started to code that ...
# from inside my QWidget:
config_clock(self.label)
# this function would live outisde the class, thus reusable by diff Qt apps:
def config_clock(label):
# start the clock
# set default font, etc for label
# instantiate QtCore.QTimer
# # but that's when I realized I've always passed self to QtCore.QTimer and that maybe encapsulating this isn't as trivial as I thought.
Should I create some kind of ClockLabel() object of my own that gets passed a QtWidget's label and can also be an instance attribute of each QtWidget that might need it? That smells kind of clunky to me. But surely there must be a way to make a 'reusable component' in PyQt, I just don't know how...
I also am not certain if the MainView(QtWidgets.QMainWindow) could rightly be referred to as the 'parent class' if I were to pass it as a parameter to a ClockLabel() class I write or a config_clock function whose signature could look like:
def config_clock(label, parent_qtwidget):
# also feels clunky and not sure if parent would be the right term
Thanks
With QtWidgets it is normal to specialize widgets by inheritance. Here is an example of how you might rearrange your code to produce a reusable widget:
class ClockLabel(QtWidgets.QLabel):
def __init__(self, parent=None):
super().__init__(parent)
self.setAlignment(QtCore.Qt.AlignCenter)
font = QtGui.QFont('Arial', 24, QtGui.QFont.Bold)
self.setFont(font)
self._timer = QtCore.QTimer(self)
self._timer.timeout.connect(self._show_time)
def start(self):
self._timer.start(1000)
def stop(self):
self._timer.stop()
def _show_time(self):
current_time = QtCore.QTime.currentTime()
clock_label_time = current_time.toString('hh:mm:ss')
self.setText(clock_label_time)
If I need to load multiple tables, can I create a python function like this or is it not considered good practice?
def bqLoad(task, table_id, fileList):
load = GoogleCloudStorageToBigQueryOperator(
task_id = task,
bucket = "bucket_name",
destination_project_dataset_table="{}.{}.{}".format("project_id","dataset_id",table_id),
source_format="PARQUET",
source_objects = fileList,
create_disposition="CREATE_IF_NEEDED",
write_disposition="WRITE_APPEND",
dag=dag,
)
return load
That approach is fine, and considered DRY (the principle aimed at reducing repetition of software, which you can read more about here or in this book which is very good)
Furthermore, the Airflow documentation outlines a variety of best practices that you can have a read through if you're interested.
Personally I prefer to define a Custom Operator by extending any other Operator, could be something like this:
class BigQueryLoadOperator(GoogleCloudStorageToBigQueryOperator):
"""
Custom Operator..
"""
#apply_defaults
def __init__(self,
table_id,
fileList,
*args, **kwargs) -> None:
super(BigQueryLoadOperator, self).__init__(*args, **kwargs)
self.bucket = "bucket_name",
self.destination_project_dataset_table = "{}.{}.{}".format(
"project_id", "dataset_id", table_id),
self.source_format = "PARQUET",
self.source_objects = fileList,
self.create_disposition = "CREATE_IF_NEEDED",
self.write_disposition = "WRITE_APPEND"
def execute(self, context):
# Implement here the expected behavior
# Take advantage from attributes and methods
# from inherited operator
pass
Consume it from your DAG with something like this:
load_to_bq_task = BigQueryLoadOperator(
task_id='some_unique_id',
table_id='table_unique_id',
file_list=['file1','file2']
)
Check the provided link for more details and examples. Good luck!
I define two networks in my model.
def Net_One():
conv2d
conv2d
...
def Net_Two():
Net_One(input)
conv2d
fc
So my question is: when I train Net_Two, using back propagation optimization, will pytorch automatically train Net_One or not? Why?
Firstly model class name's convention should be CapWords: NetOne,NetTwo, but this doesn't hurt anything, just a convention.
As for your questions, It depends on the processing of NetTwo().
if the final loss of NetTwo has nothing to do with NetOne(), then the back propagation will not flow through NetOne(), and therefore will not update NetOne()'s parameters.
Otherwise the back propagation will compute the NetOne()'s gradients and update its weights.
For code samples:
# NetTwo's loss has nothing to do with NetOne:
def NetOne():
def __init__(self):
super(NetOne, self).__init__()
...
def forward(inputs):
...
def NetTwo():
def __init__(self):
super(NetTwo, self).__init__()
...
def forward(inputs):
# in this processing, temp is never used by NetTwo's layers..
...
temp = NetOne(inputs)
inputs = conv2d(inputs)
...
In the upper codes, temp is never used, so NetOne won't be updated when NetTwo is updating.
But if temp is used by NetTwo.forward(), it'll be updated, like following:
def NetTwo():
def __init__(self):
super(NetTwo, self).__init__()
...
def forward(inputs):
# in this processing, temp is used by NetTwo's layers.
...
temp = NetOne(inputs)
inputs = conv2d(temp)
...
Did this answer your question?
I have encountered a theoretical question. I'm using pyqt5, but this is probably are very generalistic and framework independent question.
I have a QMainwindow sitting around waiting for the user to do stuff. The user can show / hide dialogues (subclasses of QDockwidgets) as he chooses using the QMenu and the associated shortcuts (it's a checkable QAction for each individual dialogue).
I have been struggling with showing / hiding the dialogues efficiently. Currently, I'm just initiating them all at start up, hiding those that I don't want to show up in the beginning. This makes triggering the dialogues easy, since I can just dialogue.show() /dialogue.hide() depending on the dialogues current visibility.
But I cannot believe that this is best practice and very efficient.
I have tried (I currently do not have my pyqt environment set up on this computer, so I had to strip down my actual code without being able to test if this runs):
from PyQt5.QtCore import *
from PyQt5.QtGui import *
from PyQt5.QtWidgets import *
class InfoPanel(QDockWidget):
def __init__(self, title='Tool Box'):
QDockWidget.__init__(self, title)
self.setFeatures(QDockWidget.DockWidgetFloatable | QDockWidget.DockWidgetMovable | QDockWidget.DockWidgetClosable)
self.setAllowedAreas(Qt.LeftDockWidgetArea | Qt.RightDockWidgetArea)
frame = QFrame()
layout = QGridLayout()
self.canvas = QGraphicsView()
self.canvas.setBackgroundBrush(QtGui.QBrush(QtGui.QColor(40, 40, 40)))
layout.addWidget(self.canvas)
frame.setLayout(layout)
self.setWidget(frame)
class MainWindow(QMainWindow):
def __init__(self):
super().__init__()
self.showpanelAct = QAction("&Show Panel", self, enabled=True,checkable=True, shortcut="F10")
self.showpanelAct.triggered.connect(lambda: self.showPanel(0))
self.viewMenu = QMenu("&View", self)
self.viewMenu.addAction(self.showpanelAct)
self.setDockOptions(QMainWindow.AnimatedDocks)
def showPanel(self,i:int = 0): # this is not so smart - should construct and deconstuct to save memory!?
if i == 0: #infopanel
dialogueExists = True
try: self.infoPanel
#except NameError: #does not catch the error
except:
dialogueExists = False
if dialogueExists:
print('destroy')
self.infoPanel.destroy()
else:
print('create')
self.infoPanel = InfoPanel() #init
self.infoPanel.show()
if __name__ == '__main__':
import sys
app = QApplication(sys.argv)
window = MainWindow()
window.show()
app.exec_()
Which works the first time, but after that, it only seems to trigger the destruction of the dialogue (which, surprisingly, does not crash anything it just keeps on going).
Why is that and is there a standard way to approach the showing hiding of dialogues?
I took the exposed MCVE of OP and tried to make it running in my cygwin64 on Windows 10.
At first I had to apply little fixes. (OP stated that he was not able to test it at the time of publishing.)
First, I inserted a “hut” at first line for convenient start in bash:
#!/usr/bin/python3
Second, the self.viewMenu didn't appear. Hence, I inserted a line after
self.viewMenu = QMenu("&View", self)
self.viewMenu.addAction(self.showpanelAct)
to add the viewMenu to main menu bar:
self.menuBar().addMenu(self.viewMenu)
which fixed it.
Third, when clicking the menu item I got:
Traceback (most recent call last):
File "./testQDockPanelShowHide.py", line 27, in <lambda>
self.showpanelAct.triggered.connect(lambda: self.showPanel(0))
File "./testQDockPanelShowHide.py", line 45, in showPanel
self.infoPanel = InfoPanel() #init
File "./testQDockPanelShowHide.py", line 17, in __init__
self.canvas.setBackgroundBrush(QtGui.QBrush(QtGui.QColor(40, 40, 40)))
NameError: name 'QtGui' is not defined
Aborted (core dumped)
I must admit that my Python knowledge is very limited. (I'm the guy who writes the Python bindings in C++ for the colleagues. So, my colleagues are the actual experts. At most, I play a little bit in Python when I test whether new implemented bindings do what's expected.) However, I modified
self.canvas.setBackgroundBrush(QtGui.QBrush(QtGui.QColor(40, 40, 40)))
to:
self.canvas.setBackgroundBrush(QBrush(QColor(40, 40, 40)))
which fixed this issue.
After this, I got the behavior described by OP and did a closer look where I (and OP) suspected the error:
def showPanel(self,i:int = 0): # this is not so smart - should construct and deconstuct to save memory!?
if i == 0: #infopanel
dialogueExists = True
try: self.infoPanel
#except NameError: #does not catch the error
except:
dialogueExists = False
if dialogueExists:
print('destroy')
self.infoPanel.destroy()
else:
print('create')
self.infoPanel = InfoPanel() #init
self.infoPanel.show()
I strongly believe that try: self.infoPanel doesn't do what OP thinks it would.
It tries to access self.infoPanel which isn't existing until the first call of this method. (Please, be aware, the member variable self.infoPanel isn't existing.) So, the except: branch is executed and sets dialogueExists = False which a few lines later causes self.infoPanel = InfoPanel() #init. Now, the member variable self.infoPanel is existing, and the try: self.infoPanel will never fail again until destruction of this MainWindow.
Out of curiosity, I had a look at QWidget.destroy() (to be sure not to tell something wrong):
QWidget.destroy (self, bool destroyWindow = True, bool destroySubWindows = True)
Frees up window system resources. Destroys the widget window if destroyWindow is true.
destroy() calls itself recursively for all the child widgets, passing destroySubWindows for the destroyWindow parameter. To have more control over destruction of subwidgets, destroy subwidgets selectively first.
This function is usually called from the QWidget destructor.
It definitely doesn't destroy the member variable self.infoPanel.
After having understood this, a fix was easy and obvious:
def showPanel(self,i:int = 0): # this is not so smart - should construct and deconstuct to save memory!?
if i == 0: #infopanel
try: self.infoPanel
#except NameError: #does not catch the error
except:
print('create')
self.infoPanel = InfoPanel() #init
if self.infoPanel.isVisible():
self.infoPanel.hide()
else:
self.infoPanel.show()
Btw. I replaced destroy() by hide() which makes a re-creation of the InfoPanel() obsolete.
I tested this by toggling the menu item multiple times – it works as expected now (at least, it looks like).
The complete sample finally:
#!/usr/bin/python3
from PyQt5.QtCore import *
from PyQt5.QtGui import *
from PyQt5.QtWidgets import *
class InfoPanel(QDockWidget):
def __init__(self, title='Tool Box'):
QDockWidget.__init__(self, title)
self.setFeatures(QDockWidget.DockWidgetFloatable | QDockWidget.DockWidgetMovable | QDockWidget.DockWidgetClosable)
self.setAllowedAreas(Qt.LeftDockWidgetArea | Qt.RightDockWidgetArea)
frame = QFrame()
layout = QGridLayout()
self.canvas = QGraphicsView()
# self.canvas.setBackgroundBrush(QtGui.QBrush(QtGui.QColor(40, 40, 40)))
self.canvas.setBackgroundBrush(QBrush(QColor(40, 40, 40)))
layout.addWidget(self.canvas)
frame.setLayout(layout)
self.setWidget(frame)
class MainWindow(QMainWindow):
def __init__(self):
super().__init__()
self.showpanelAct = QAction("&Show Panel", self, enabled=True,checkable=True, shortcut="F10")
self.showpanelAct.triggered.connect(lambda: self.showPanel(0))
self.viewMenu = QMenu("&View", self)
self.viewMenu.addAction(self.showpanelAct)
self.menuBar().addMenu(self.viewMenu)
self.setDockOptions(QMainWindow.AnimatedDocks)
def showPanel(self,i:int = 0): # this is not so smart - should construct and deconstuct to save memory!?
if i == 0: #infopanel
try: self.infoPanel
#except NameError: #does not catch the error
except:
print('create')
self.infoPanel = InfoPanel() #init
if self.infoPanel.isVisible():
self.infoPanel.hide()
else:
self.infoPanel.show()
if __name__ == '__main__':
import sys
app = QApplication(sys.argv)
window = MainWindow()
window.show()
app.exec_()
After taking a break from coding the solution to my problem was very obvious. Going back to my original code (which did not produce the expected output of creating / destroying the dialogue self.infoPanel on demand):
dialogueExists = True
try: self.infoPanel
#except NameError: #does not catch the error
except:
dialogueExists = False
if dialogueExists:
print('destroy')
self.infoPanel.destroy()
else:
print('create')
self.infoPanel = InfoPanel() #init
self.infoPanel.show()
My main problem was that I confused two separate things. Qt destroyed the widget contained in the object self.infoPanel when I called self.infoPanel.destroy(). But that doesn't mean the object self.infoPanel does not exist (that's exactly what I use try: ... for, to see if the object exists). The simple and obvious way to create and destroy dialogues on demand obviously involves deleting the object from the environment (del self.infoPanel).
The working code is:
dialogueExists = True
try:
self.infoPanel.destroy() #not sure this is needed, but I guess it doesn't hurt
del self.infoPanel #this is the deletion of the actual object
except:
dialogueExists = False
if not dialogueExists :
self.infoPanel = InfoPanel()
Cheers and many thanks for the helpful advice on deciding whether to show / hide dialogues or to create / destroy them!
I was wondering if I can make an output interactively run a piece of code. So if for example I had a class (parts in pseudo-code):
import numpy as np
class test(object):
def __init__():
self.a = np.random.randn(10)
print ## Interactive Output: Click me to view data array##
def show():
print a
So when I create a class instance it should output some interactive link (maybe in html) or something like that and when I click it, the show() method should be called. However, I have no idea how to achieve that.
You could use the widgets shipped with the notebook (for jupyter they are an independent package).
Something like this could do what you want (Python 3):
from IPython.html import widgets
from IPython.display import display
import numpy as np
class Test(object):
def __init__(self, arraylen):
self.a = np.random.randn(arraylen)
self.button = widgets.Button(description = 'Show')
self.button.on_click(self.show)
display(self.button)
def show(self, ev = None):
display(self.a)
self.button.disabled = True
test = Test(10)
You create a button widget when you initialise the class widgets.Button(description = 'Show')
Attach an event to it button.on_click(self.show)
And display the button display(self.button)
In the show method I included a way to disable the button functionality once the array is showed self.button.disabled = True. You can comment this line if you want to show more times the array.