Application Rapidly Opens and Closes Multiple Consoles Before Opening (PyQt5, Cx_Freeze & Py2Exe) - console

I'm attempting to freeze my Python application and have been running into the same problem when using both Cx_Freeze and Py2Exe. Once I've built/frozen the code I start up the executable and rapidly about a half dozen consoles flash (open and close rapidly in succession) on the screen until my GUI Window (created using PyQt5) opens. Once the GUI Window is open, everything appears to be operating fine.
Note: Dll Files
There seems to be a common error around DLL files and I've already included a folder called platforms with the qwindows.dll file, as well as the libEGL.dll file directly in the same folder as the executable. I don't believe this is relevant however as I am able to actually see my initial Widget.
Here is my setup.py file for Cx_Freeze:
import sys
from cx_Freeze import setup, Executable
base = 'Win32GUI'
executables = [
Executable('__main__.py', base=base)
]
# Dependencies are automatically detected, but it might need fine tuning.
buildOptions = {"packages": [], "excludes": []}
#serial, requests, idna
setup(name = "Test",
version = "0.1",
description = "Manufacturing Testing Software",
options = dict(build_exe = buildOptions),
executables = executables)
Here is my setup.py file for Py2Exe:
from setuptools import setup
import os
import py2exe
includes = ["sip",
"PyQt5",
"PyQt5.QtCore",
"PyQt5.QtGui",
"PyQt5.QtWidgets",
"PyQt5.QtWebKit",
"PyQt5.QtWebKitWidgets",
"PyQt5.QtWebKitWidgets",
"PyQt5.QtNetwork",
"PyQt5.QtPrintSupport"]
datafiles = [("platforms", [r"C:\Users\allan\AppData\Local\Continuum\Anaconda2\Library\plugins\platforms\qwindows.dll"]),
("", [r"c:\windows\syswow64\MSVCP100.dll",
r"c:\windows\syswow64\MSVCR100.dll",
r"C:\Python36-32\Lib\site-packages\PyQt5\Qt\bin\libEGL.dll"])]
setup(
name='Test',
version='1',
windows=['__main__.py'],
data_files = datafiles,
options={
"py2exe":{
"includes": includes,
}
}
)

The problem was that I invoked certain os.system commands before the main event look so prior to actually opening up a pyqt window, several shells were opening and closing in rapid succession.

Related

How to use a File Selector with hvplot in a Docker Panel Serve Environment?

Similar to this question about this error message I get a
Uncaught (in promise) Error: Error rendering Bokeh model: could not find #53e7b147-e581-431b-90e1-1fe7ade39263 HTML tag
at u (bokeh.min.js?v=3c61e952b808bb7e346ce828a565a5f23aaf7708d034fa9d0906403813355d45bb4e8d8b0b23a93f032c76831d4f0221846f28699c7f5147caa62e0d31668314:585:113)
at n._resolve_root_elements (bokeh.min.js?v=3c61e952b808bb7e346ce828a565a5f23aaf7708d034fa9d0906403813355d45bb4e8d8b0b23a93f032c76831d4f0221846f28699c7f5147caa62e0d31668314:585:621)
at w (bokeh.min.js?v=3c61e952b808bb7e346ce828a565a5f23aaf7708d034fa9d0906403813355d45bb4e8d8b0b23a93f032c76831d4f0221846f28699c7f5147caa62e0d31668314:163:515)
at t.embed_items (bokeh.min.js?v=3c61e952b808bb7e346ce828a565a5f23aaf7708d034fa9d0906403813355d45bb4e8d8b0b23a93f032c76831d4f0221846f28699c7f5147caa62e0d31668314:163:1455)
when running this app
import panel as pn
import glob
file_list = glob.glob( r'/usr/src/app/apps/*.nc' )
def get_comps():
select_file = pnw.Select(name="File", options=file_list).servable(target="sidebar")
#pn.depends(select_file.param.value)
def selected_file(select_file):
return pn.pane.Str(select_file, height=100, sizing_mode='stretch_width')
return [ select_file , selected_file ]
bootstrap = pn.template.BootstrapTemplate(title='Glob Demo')
sfe = get_comps()
bootstrap.sidebar.append(sfe[0])
bootstrap.main.append(sfe[1])
bootstrap.servable()
even though the FileSelector widget sees the files and displays as expected in another example app that is ran with the same panel serve command inside a Docker container that the Glob Demo App is ran with
docker run -p 8066:8000 -v /PATH_TO_APPS/apps:/usr/src/app/apps yp2
I would love to mount a volume when running my Docker Container that I can select a file from and load with Xarray and plot with HvPlot but I am stuck at even selecting a file. I already have the app I want to serve thanks to #Marc and this question.
Panel Version : 0.13.1
OS : Ubuntu
Docker Version : 20.10.17, build 100c701

Vaadin Flow 14, Jetty embedded and static files

I'm trying to create app based on Jetty 9.4.20 (embedded) and Vaadin Flow 14.0.12.
It based on very nice project vaadin14-embedded-jetty.
I want to package app with one main-jar and all dependency libs must be in folder 'libs' near main-jar.
I remove maven-assembly-plugin, instead use maven-dependency-plugin and maven-jar-plugin. In maven-dependency-plugin i add section <execution>get-dependencies</execution> where i unpack directories META-INF/resources/,META-INF/services/ from Vaadin Flow libs to the result JAR.
In this case app work fine. But if i comment section <execution>get-dependencies</execution> then result package didn't contain that directories and app didn't work.
It just cannot give some static files from Vaadin Flow libs.
This error occurs only if i launch packaged app with ...
$ java -jar vaadin14-embedded-jetty-1.0-SNAPSHOT.jar
... but from Intellij Idea it launch correctly.
There was an opinion that is Jetty staring with wrong ClassLoader and cannot maintain requests to static files in Jar-libs.
The META-INF/services/ files MUST be maintained from the Jetty libs.
That's important for Jetty to use java.util.ServiceLoader.
If you are merging contents of JAR files into a single JAR file, that's called a "uber jar".
There are many techniques to do this, but if you are using maven-assembly-plugin or maven-dependency-plugin to build this "uber jar" then you will not be merging critical files that have the same name across multiple JAR files.
Consider using maven-shade-plugin and it's associated Resource Transformers to properly merge these files.
http://maven.apache.org/plugins/maven-shade-plugin/
http://maven.apache.org/plugins/maven-shade-plugin/examples/resource-transformers.html
The ServicesResourceTransformer is the one that merges META-INF/services/ files, use it.
As for static content, that works fine, but you have to setup your Base Resource properly.
Looking at your source, you do the following ...
final URI webRootUri = ManualJetty.class.getResource("/webapp/").toURI();
final WebAppContext context = new WebAppContext();
context.setBaseResource(Resource.newResource(webRootUri));
That won't work reliably in 100% of cases (as you have noticed when running in the IDE vs command line).
The Class.getResource(String) is only reliable if you lookup a file (not a directory).
Consider that the Jetty Project Embedded Cookbook recipes have techniques for this.
See:
WebAppContextFromClasspath.java
ResourceHandlerFromClasspath.java
DefaultServletFileServer.java
DefaultServletMultipleBases.java
XmlEnhancedServer.java
MultipartMimeUploadExample.java
Example:
// Figure out what path to serve content from
ClassLoader cl = ManualJetty.class.getClassLoader();
// We look for a file, as ClassLoader.getResource() is not
// designed to look for directories (we resolve the directory later)
URL f = cl.getResource("webapp/index.html");
if (f == null)
{
throw new RuntimeException("Unable to find resource directory");
}
// Resolve file to directory
URI webRootUri = f.toURI().resolve("./").normalize();
System.err.println("WebRoot is " + webRootUri);
WebAppContext context = new WebAppContext();
context.setBaseResource(Resource.newResource(webRootUri));

How can I read a config file from airflow packaged DAG?

Airflow packaged DAGs seem like a great building block for a sane production airflow deployment.
I have a DAG with dynamic subDAGs, driven by a config file, something like:
config.yaml:
imports:
- project_foo
- project_bar`
which yields subdag tasks like imports.project_{foo|bar}.step{1|2|3}.
I've normally read in the config file using python's open function, a la config = open(os.path.join(os.path.split(__file__)[0], 'config.yaml')
Unfortunately, when using packaged DAGs, this results in an error:
Broken DAG: [/home/airflow/dags/workflows.zip] [Errno 20] Not a directory: '/home/airflow/dags/workflows.zip/config.yaml'
Any thoughts / best practices to recommend here?
It's a bit of a kludge, but I eventually just fell back on reading zip file contents via ZipFile.
import yaml
from zipfile import ZipFile
import logging
import re
def get_config(yaml_filename):
"""Parses and returns the given YAML config file.
For packaged DAGs, gracefully handles unzipping.
"""
zip, post_zip = re.search(r'(.*\.zip)?(.*)', yaml_filename).groups()
if zip:
contents = ZipFile(zip).read(post_zip.lstrip('/'))
else:
contents = open(post_zip).read()
result = yaml.safe_load(contents)
logging.info('Parsed config: %s', result)
return result
which works as you'd expect from the main dag.py:
get_config(os.path.join(path.split(__file__)[0], 'config.yaml'))

In a Jupyter Notebook in Watson Studio , how to refer to a file uploaded to the "assets"?

in Watson Studio I am writing code in a Jupyter Notebook to use a Watson Visual Recognition custom model.
It works ok with external images.
I haven't been able yet to refer to an image I have uploaded to the Assets of my project.
The url of the asset gets to a full page not the image only:
https://dataplatform.ibm.com/projects/2f4b89d9-b93a-4c98-a327-9b863a467b7c/data-assets/ed16c385-e09e-4bcb-bfab-67ee864538e4/?context=data
Thank you
In Watson the asset files are saved in Cloud Object Store, or COS. You have to download the image from COS to the notebook server file system and then you can refer to the file in your notebook as a regular local file.
I use the cos api to get the files. https://github.com/IBM/ibm-cos-sdk-python.
https://console.bluemix.net/docs/services/cloud-object-storage/libraries/python.html#using-python
First, find out what your credentials are by
highlighting a notebook cell,
click Data menu,
select Files,
"Insert to Code"
credential.
Then you can download the file to your local disk storage using the API. For example, to download a file from COS:
# The following code contains the credentials for a file in your IBM Cloud Object Storage.
# You might want to remove those credentials before you share your notebook.
credentials_1 = {
'IBM_API_KEY_ID': '**************************************',
'IAM_SERVICE_ID': 'iam-ServiceId-**************************',
'ENDPOINT': 'https://s3-api.us-geo.objectstorage.service.networklayer.com',
'IBM_AUTH_ENDPOINT': 'https://iam.ng.bluemix.net/oidc/token',
'BUCKET': '********************************',
'FILE': 'file.xlsx'
}
from ibm_botocore.client import Config
import ibm_boto3
def download_file_cos(credentials, local_file_name, key):
cos = ibm_boto3.client(service_name='s3',
ibm_api_key_id=credentials['IBM_API_KEY_ID'],
ibm_service_instance_id=credentials['IAM_SERVICE_ID'],
ibm_auth_endpoint=credentials['IBM_AUTH_ENDPOINT'],
config=Config(signature_version='oauth'),
endpoint_url=credentials['ENDPOINT'])
try:
res=cos.download_file(Bucket=credentials['BUCKET'], Key=key, Filename=local_file_name)
except Exception as e:
print(Exception, e)
else:
print("Dowloaded:", key, 'from IBM COS to local:', local_file_name)
In a Notebook cell list directory contents:
%%script bash
ls -l
# to list all .png files in COS you can use a function like this:
def list_objects(credentials):
cos = ibm_boto3.client(service_name='s3',
ibm_api_key_id=credentials['IBM_API_KEY_ID'],
ibm_service_instance_id=credentials['IAM_SERVICE_ID'],
ibm_auth_endpoint=credentials['IBM_AUTH_ENDPOINT'],
config=Config(signature_version='oauth'),
endpoint_url=credentials['ENDPOINT'])
return cos.list_objects(Bucket=credentials['BUCKET'])
response = list_objects(credentials_1)
for c in response['Contents']:
if c['Key'].endswith('.png'):
print(c['Key'], "last modified:", c['LastModified'])

How can I install Qt 5.2.1 from the command line in Cygwin?

$ wget --quiet http://download.qt-project.org/official_releases/qt/5.2/5.2.1/qt-opensource-windows-x86-msvc2012_64_opengl-5.2.1.exe
$
As seen above, I first downloaded the Qt package for Visual Studio in a Cygwin Bash shell.
A sidenote: The Qt library packaged within Cygwin is not useful for me because I need to use the Visual Studio C++ compiler.
First I set the correct permissions on the file
$ chmod 755 qt-opensource-windows-x86-msvc2012_64_opengl-5.2.1.exe
If I start it like this
$ ./qt-opensource-windows-x86-msvc2012_64_opengl-5.2.1.exe
a graphical window (GUI) is shown but that is not what I want as I would later like to have the installation procedure written into a Bash script that I could run in Cygwin.
If I add the option --help, like this
$ ./qt-opensource-windows-x86-msvc2012_64_opengl-5.2.1.exe --help
a new terminal window is opened with the following text
Usage: SDKMaintenanceTool [OPTIONS]
User:
--help Show commandline usage
--version Show current version
--checkupdates Check for updates and return an XML file describing
the available updates
--updater Start in updater mode.
--manage-packages Start in packagemanager mode.
--proxy Set system proxy on Win and Mac.
This option has no effect on Linux.
--verbose Show debug output on the console
--create-offline-repository Offline installer only: Create a local repository inside the
installation directory based on the offline
installer's content.
Developer:
--runoperation [OPERATION] [arguments...] Perform an operation with a list of arguments
--undooperation [OPERATION] [arguments...] Undo an operation with a list of arguments
--script [scriptName] Execute a script
--no-force-installations Enable deselection of forced components
--addRepository [URI] Add a local or remote repo to the list of user defined repos.
--addTempRepository [URI] Add a local or remote repo to the list of temporary available
repos.
--setTempRepository [URI] Set a local or remote repo as tmp repo, it is the only one
used during fetch.
Note: URI must be prefixed with the protocol, i.e. file:///
http:// or ftp://. It can consist of multiple
addresses separated by comma only.
--show-virtual-components Show virtual components in package manager and updater
--binarydatafile [binary_data_file] Use the binary data of another installer or maintenance tool.
--update-installerbase [new_installerbase] Patch a full installer with a new installer base
--dump-binary-data -i [PATH] -o [PATH] Dumps the binary content into specified output path (offline
installer only).
Input path pointing to binary data file, if omitted
the current application is used as input.
I don't know how to continue from here. Do you know how I could install the Qt 5.2.1 library (for Visual Studio) from the Bash shell in Cygwin?
Update: The advantage of writing the build script for a Cygwin environment is that commands like git, wget, and scp are available. This Stackoverflow answer describes how to invoke the MSVC compiler from a Cygwin bash script. Note, that the Qt application I'm building is not going to have any dependency on Cygwin.
I didn't test with Cygwin but I successfully installed Qt5.5 using a script. To do so, you must use the --script line of the normal installer.
.\qt-opensource-windows-x86-msvc2013_64-5.5.1.exe --script .\qt-installer-noninteractive.qs
Here's an example of qt-installer-noninteractive.qs file I used in the command above
function Controller() {
installer.autoRejectMessageBoxes();
installer.installationFinished.connect(function() {
gui.clickButton(buttons.NextButton);
})
}
Controller.prototype.WelcomePageCallback = function() {
gui.clickButton(buttons.NextButton);
}
Controller.prototype.CredentialsPageCallback = function() {
gui.clickButton(buttons.NextButton);
}
Controller.prototype.IntroductionPageCallback = function() {
gui.clickButton(buttons.NextButton);
}
Controller.prototype.TargetDirectoryPageCallback = function() {
gui.currentPageWidget().TargetDirectoryLineEdit.setText("C:/Qt/Qt5.5.1");
gui.clickButton(buttons.NextButton);
}
Controller.prototype.ComponentSelectionPageCallback = function() {
var widget = gui.currentPageWidget();
widget.deselectAll();
widget.selectComponent("qt.55.win64_msvc2013_64");
// widget.selectComponent("qt.55.qt3d");
// widget.selectComponent("qt.55.qtcanvas3d");
// widget.selectComponent("qt.55.qtquick1");
// widget.selectComponent("qt.55.qtscript");
// widget.selectComponent("qt.55.qtwebengine");
// widget.selectComponent("qt.55.qtquickcontrols");
// widget.selectComponent("qt.55.qtlocation");
gui.clickButton(buttons.NextButton);
}
Controller.prototype.LicenseAgreementPageCallback = function() {
gui.currentPageWidget().AcceptLicenseRadioButton.setChecked(true);
gui.clickButton(buttons.NextButton);
}
Controller.prototype.StartMenuDirectoryPageCallback = function() {
gui.clickButton(buttons.NextButton);
}
Controller.prototype.ReadyForInstallationPageCallback = function()
{
gui.clickButton(buttons.NextButton);
}
Controller.prototype.FinishedPageCallback = function() {
var checkBoxForm = gui.currentPageWidget().LaunchQtCreatorCheckBoxForm
if (checkBoxForm && checkBoxForm.launchQtCreatorCheckBox) {
checkBoxForm.launchQtCreatorCheckBox.checked = false;
}
gui.clickButton(buttons.FinishButton);
}
The tricky part was to found the id of the components! I was able to found the right id qt.55.win64_msvc2013_64 by adding the flag --verbose and installing it normally with the UI and stopping at the last page; all the ids that you selected for installation are there.
There is slightly more information in this answer if you need more details.
EDIT (29-11-2017): For installer 3.0.2-online, the "Next" button in the "Welcome" page is disabled for 1 second so you must add a delay
gui.clickButton(buttons.NextButton, 3000);
EDIT (10-11-2019): See Joshua Wade's answer for more traps and pitfalls, like the "User Data Collection" form and "Archive" and "Latest releases" checkboxes.

Resources