I have a problem which I have reduced to a small program. The program tries to write a Pandas dataframe. This works find in PyCharm, but when I produce an exe using PyInstaller it errors at runtime:
.\pyinst_excel.exe
Traceback (most recent call last):
File "pyinst_excel.py", line 31, in <module>
File "pyinst_excel.py", line 25, in add_generated_files_row
File "site-packages\pandas\io\excel\_openpyxl.py", line 18, in __init__
ModuleNotFoundError: No module named 'openpyxl'
[11408] Failed to execute script pyinst_excel
Now I have tried explicitly importing openpyxl and that didn't work and I have also used --hidden-import:
call venv\Scripts\activate
pyinstaller --onefile --hidden-import _openpyxl --hidden-import openpyxl --clean pyinst_excel.py
The generated spec file looks like this:
# -*- mode: python ; coding: utf-8 -*-
block_cipher = None
a = Analysis(['pyinst_excel.py'],
pathex=['D:\\Development\\OEM-SR-Rules'],
binaries=[],
datas=[],
hiddenimports=['_openpyxl', 'openpyxl'],
hookspath=[],
runtime_hooks=[],
excludes=[],
win_no_prefer_redirects=False,
win_private_assemblies=False,
cipher=block_cipher,
noarchive=False)
pyz = PYZ(a.pure, a.zipped_data,
cipher=block_cipher)
exe = EXE(pyz,
a.scripts,
a.binaries,
a.zipfiles,
a.datas,
[],
name='pyinst_excel',
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=True,
upx_exclude=[],
runtime_tmpdir=None,
console=True )
However this doesn't fix the problem. From the build/warnings file I see:
...
missing module named openpyxl - imported by pandas.io.excel._openpyxl (delayed, conditional), D:\Development\OEM-SR-Rules\pyinst_excel.py (top-level)
...
missing module named 'openpyxl.styles' - imported by pandas.io.excel._openpyxl (delayed)
missing module named 'openpyxl.style' - imported by pandas.io.excel._openpyxl (delayed)
Program code:
import argparse
from operator import attrgetter
import os
import sys
import json
import pandas as pd
import numpy as np
from pathlib import Path
import glob
import re
import time
from sys import exit
from shutil import copyfile
REPORT_DATE_FMT = '%d-%m-%Y %H.%M'
def add_generated_files_row(generated_excel, sequence: int, date_generated):
# gen_files_df = pd.read_excel(generated_files_excel, index_col=0)
gen_files_df = pd.DataFrame(columns=['Sequence', 'Date Generated'])
new_row = {'Sequence': sequence,
'Date Generated': date_generated}
gen_files_df = gen_files_df.append(new_row, ignore_index=True, sort=False)
writer = pd.ExcelWriter(generated_excel)
gen_files_df.to_excel(writer)
generated_files_excel = Path('D:\temp\test.xlsx')
date_generated = time.strftime(f"{REPORT_DATE_FMT}")
add_generated_files_row(generated_files_excel, 1, date_generated)
I am using Python 3.8 and PyInstaller 3.6.
I am out of ideas. Any help would be much appreciated.
Thanks,
Clive
The problem turned out to be caused PyInstaller being run from a different environment.
To solve this I had to do:
venv\Scripts\activate.bat
(venv) D:\Development\OEM-SR-Rules>pip install PyInstaller
Related
First, set up a virtual environment with anaconda, then install modules such as selenium,pyqt5, pyinstaller, and webdriver_manager in the virtual environment,
After creating the .spec file by setting pyinstaller.exe -w -F (project), the .spec file was modified as follows.
# -*- mode: python ; coding: utf-8 -*-
block_cipher = None
a = Analysis(
['GUI.py'],
pathex=[],
binaries=[],
datas=[('gui.ui','.'),('BHA_icon.png','.'),('process.py','.')],
hiddenimports=["webdriver_manager.chrome","selenium"],
hookspath=[],
hooksconfig={},
runtime_hooks=[],
excludes=[],
win_no_prefer_redirects=False,
win_private_assemblies=False,
cipher=block_cipher,
noarchive=False,
)
pyz = PYZ(a.pure, a.zipped_data, cipher=block_cipher)
exe = EXE(
pyz,
a.scripts,
a.binaries,
a.zipfiles,
a.datas,
[],
name='Baekjoon Hub Automation',
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=True,
icon=r'BHA_icon.ico',
upx_exclude=[],
runtime_tmpdir=None,
console=False,
disable_windowed_traceback=False,
argv_emulation=False,
target_arch=None,
codesign_identity=None,
entitlements_file=None,
)
After that, I built it as an exe file, and the program ran well with the console window below on my computer, but when I ran the exe file on another computer, the following error occurred.
enter image description here
enter image description here
and my console window : [13676:20232:0201/200946.234:ERROR:network_change_notifier_win.cc(224)] WSALookupServiceBegin failed with: 0
DevTools listening on ws://127.0.0.1:7831/devtools/browser/4cce63d7-9b48-420e-a020-ff714694c193
[16408:19512:0201/200946.366:ERROR:network_change_notifier_win.cc(224)] WSALookupServiceBegin failed with: 0
[13676:6804:0201/200947.548:ERROR:cert_issuer_source_aia.cc(34)] Error parsing cert retrieved from AIA (as DER):
ERROR: Couldn't read tbsCertificate as SEQUENCE
ERROR: Failed parsing Certificate
[13676:20232:0201/200950.547:ERROR:device_event_log_impl.cc(215)] [20:09:50.547] Bluetooth: bluetooth_adapter_winrt.cc:1205 Getting Radio failed. Chrome will be unable to change the power state by itself.
[13676:20232:0201/200950.565:ERROR:device_event_log_impl.cc(215)] [20:09:50.565] Bluetooth: bluetooth_adapter_winrt.cc:1283 OnPoweredRadioAdded(), Number of Powered Radios: 1
[13676:20232:0201/200950.566:ERROR:device_event_log_impl.cc(215)] [20:09:50.566] Bluetooth: bluetooth_adapter_winrt.cc:1298 OnPoweredRadiosEnumerated(), Number of Powered Radios: 1
[7568:21300:0201/201146.489:ERROR:gpu_init.cc(523)] Passthrough is not supported, GL is disabled, ANGLE is
[13676:20232:0201/202032.150:ERROR:device_event_log_impl.cc(215)] [20:20:32.148] USB: usb_service_win.cc:104 SetupDiGetDeviceProperty({{A45C254E-DF1C-4EFD-8020-67D146A850E0}, 6}) failed: 요소가 없습니다. (0x490)
[13676:20232:0201/202032.174:ERROR:device_event_log_impl.cc(215)] [20:20:32.173] USB: usb_service_win.cc:307 SetupDiEnumDeviceInterfaces: 사용 가능한 데이터가 없습니다. (0x103)
driver setting code
import time
from selenium.webdriver.common.by import By
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
def set_chrome_driver():
options = webdriver.ChromeOptions()
options.add_experimental_option("detach", True)
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=options)
return driver
#####
def start_setting_BJ(id, pw, i, driver):
if i!=True:
url = 'https://www.acmicpc.net/login'
driver.get(url)
selector = "#submit_button"
time.sleep(1)
driver.find_element(By.NAME, 'login_user_id').send_keys(id)
driver.find_element(By.NAME, 'login_password').send_keys(pw)
elem = driver.find_element(By.CSS_SELECTOR, selector)
elem.click()
time.sleep(2)
login_check= driver.find_element(By.XPATH, "/html/body/div[2]/div[1]/div[1]/div/ul/li[3]/a") # 로그인 여부 확인용 "로그인" 텍스트 인식
if login_check.text=="로그인":
return -1
else:
driver.get('https://www.acmicpc.net/user/' + id)
Peoblem_list = '/html/body/div[2]/div[2]/div[3]/div[2]/div/div[2]/div[2]/div[2]/div'
time.sleep(1)
elem = driver.find_element(By.XPATH, Peoblem_list)
all = elem.text.split()
return all
I have created an executable file using --onefile option with pyinstaller (pyside6 app) and it works fine on the windows computer on which it is created. When I run this file on another windows machine, if get the following error:
qt.qpa.plugin: Could not load the Qt platform plugin "windows" in "" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.
Available platform plugins are: direct2d, minimal, offscreen, windows.
I have tried putting the platform folder with .dll files next to the executable but that doesn't solve this problem.
Further, none of the suggestions mentioned in PyInstaller generated exe file error : qt.qpa.plugin: could not load the qt platform plugin "windows" in "" even though it was found and the thread therein have helped.
Could you please let me know a possible way to fix this issue?
Below is an MWE having the two python source files and the pyinstaller spec file.
test_app.py
# -*- coding: utf-8 -*-
################################################################################
## Form generated from reading UI file 'test_app.ui'
##
## Created by: Qt User Interface Compiler version 6.3.1
##
## WARNING! All changes made in this file will be lost when recompiling UI file!
################################################################################
from PySide6.QtCore import (QCoreApplication, QDate, QDateTime, QLocale,
QMetaObject, QObject, QPoint, QRect,
QSize, QTime, QUrl, Qt)
from PySide6.QtGui import (QBrush, QColor, QConicalGradient, QCursor,
QFont, QFontDatabase, QGradient, QIcon,
QImage, QKeySequence, QLinearGradient, QPainter,
QPalette, QPixmap, QRadialGradient, QTransform)
from PySide6.QtWidgets import (QApplication, QLabel, QLineEdit, QMainWindow,
QPushButton, QSizePolicy, QStatusBar, QWidget)
class Ui_MainWindow(object):
def setupUi(self, MainWindow):
if not MainWindow.objectName():
MainWindow.setObjectName(u"MainWindow")
MainWindow.resize(300, 150)
self.centralwidget = QWidget(MainWindow)
self.centralwidget.setObjectName(u"centralwidget")
self.label = QLabel(self.centralwidget)
self.label.setObjectName(u"label")
self.label.setGeometry(QRect(10, 39, 41, 21))
self.enteredName = QLineEdit(self.centralwidget)
self.enteredName.setObjectName(u"enteredName")
self.enteredName.setGeometry(QRect(70, 40, 113, 20))
self.showHi = QLineEdit(self.centralwidget)
self.showHi.setObjectName(u"showHi")
self.showHi.setGeometry(QRect(80, 100, 113, 20))
self.run = QPushButton(self.centralwidget)
self.run.setObjectName(u"run")
self.run.setGeometry(QRect(210, 40, 56, 17))
MainWindow.setCentralWidget(self.centralwidget)
self.statusbar = QStatusBar(MainWindow)
self.statusbar.setObjectName(u"statusbar")
MainWindow.setStatusBar(self.statusbar)
self.retranslateUi(MainWindow)
QMetaObject.connectSlotsByName(MainWindow)
# setupUi
def retranslateUi(self, MainWindow):
MainWindow.setWindowTitle(QCoreApplication.translate("MainWindow", u"MainWindow", None))
self.label.setText(QCoreApplication.translate("MainWindow", u"Name", None))
self.run.setText(QCoreApplication.translate("MainWindow", u"Run", None))
# retranslateUi
The main file (test_MWE.py)
import sys
from PySide6.QtWidgets import QApplication, QMainWindow, QPushButton, QLineEdit, QFileDialog
from test_app import Ui_MainWindow
def showMessage():
window.ui.showHi.setText("HI "+window.ui.enteredName.text())
class MainWindow(QMainWindow):
def __init__(self):
super(MainWindow, self).__init__()
self.ui = Ui_MainWindow()
self.ui.setupUi(self)
if __name__ == "__main__":
app = QApplication(sys.argv)
window = MainWindow()
window.show()
window.ui.run.clicked.connect(showMessage)
sys.exit(app.exec())
The test_MWE.spec file
# -*- mode: python ; coding: utf-8 -*-
block_cipher = None
a = Analysis(
['test_MWE.py'],
pathex=[],
binaries=[],
datas=[],
hiddenimports=[],
hookspath=[],
hooksconfig={},
runtime_hooks=[],
excludes=[],
win_no_prefer_redirects=False,
win_private_assemblies=False,
cipher=block_cipher,
noarchive=False,
)
pyz = PYZ(a.pure, a.zipped_data, cipher=block_cipher)
exe = EXE(
pyz,
a.scripts,
a.binaries,
a.zipfiles,
a.datas,
[],
name='test_MWE',
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=True,
upx_exclude=[],
runtime_tmpdir=None,
console=True,
disable_windowed_traceback=False,
argv_emulation=False,
target_arch=None,
codesign_identity=None,
entitlements_file=None,
)
Recap, I am compiling the executable with --onefile option on windows 11. The resulting file is working as expected. On running the executable file on Windows 7, I am getting the error given above.
Qt6 was never intended to work on versions older than Windows 10, as explained in the early blog post about hosts and targets, and as can be seen in the official documentation about the supported platforms.
If you need your program to also work on Windows <10, you must use Qt5.
I'm trying to import classes in external jar with PySpark, I'm running the spark-shell with --jars and the path to the jar that contains the classes I want to use.
However, when I import a class inside my code, the namespace is not found:
from io.warp10.spark import WarpScriptFilterFunction
The error:
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Traceback (most recent call last):
File "warp10-test.py", line 1, in <module>
from io.warp10.spark import WarpScriptFilterFunction
ImportError: No module named warp10.spark
You have to use a WarpScript™ UDF if you want to run warpscript on Spark.
Here is an example:
from pyspark.sql import SparkSession
from pyspark.sql import SQLContext
from pyspark.sql.types import StringType
from pyspark.sql.types import ArrayType
spark = SparkSession.builder.appName("WarpScript Spark Test").getOrCreate()
sc = spark.sparkContext
sqlContext = SQLContext(sc)
sqlContext.registerJavaFunction("foo", "io.warp10.spark.WarpScriptUDF3", ArrayType(StringType()))
print sqlContext.sql("SELECT foo('SNAPSHOT \"Easy!\"', 3.14, 'pi')").collect()
For more information, see: https://www.warp10.io/content/05_Ecosystem/04_Data_Science/06_Spark/02_WarpScript_PySpark
I'm trying to convert R-dataframe to Python Pandas DataFrame.
I use the following code:
from rpy2.robjects import pandas2ri
pandas2ri.activate()
r_dataframe = r_function(my_dataframe['Numbers'])
print(r_dataframe)
python_dataframe = pandas2ri.ri2py(r_dataframe)
The above code works well in Jupyter Notebook (Anaconda). But if I run this code through a my_program.py file through the terminal, I get an error:
:~$ python3 my_program.py
Traceback (most recent call last):
File "my_program.py", line 223, in <module>
python_dataframe = pandas2ri.ri2py(r_dataframe)
AttributeError: module 'rpy2.robjects.pandas2ri' has no attribute 'ri2py'
Line of code: print(r_dataframe) shows right result in the terminal.
If I try to use code print(dir(pandas2ri)) in Jupyter Notebook I get ('ri2py'):
['DataFrame', 'FactorVector', 'FloatSexpVector', 'INTSXP', 'ISOdatetime', 'IntSexpVector', 'IntVector', 'ListSexpVector', 'ListVector', 'OrderedDict', 'POSIXct', 'PandasDataFrame', 'PandasIndex', 'PandasSeries', 'SexpVector', 'StrSexpVector', 'StrVector', 'Vector', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'activate', 'as_vector', 'conversion', 'converter', 'datetime', 'deactivate', 'dt_O_type', 'dt_datetime64ns_type', 'get_timezone', 'numpy', 'numpy2ri', 'original_converter', 'os', 'pandas', 'py2ri', 'py2ri_categoryseries', 'py2ri_pandasdataframe', 'py2ri_pandasindex', 'py2ri_pandasseries', 'py2ro', 'pytz', 'recarray', 'ri2py', 'ri2py_dataframe', 'ri2py_floatvector', 'ri2py_intvector', 'ri2py_listvector', 'ri2py_vector', 'ri2ro', 'rinterface', 'ro', 'warnings']
And if I try to use the same code print(dir(pandas2ri)) in Terminal I get ('rpy2py'):
['DataFrame', 'FactorVector', 'FloatSexpVector', 'ISOdatetime', 'IntSexpVector', 'IntVector', 'ListSexpVector', 'OrderedDict', 'POSIXct', 'PandasDataFrame', 'PandasIndex', 'PandasSeries', 'Sexp', 'SexpVector', 'StrSexpVector', 'StrVector', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'activate', 'as_vector', 'conversion', 'converter', 'datetime', 'deactivate', 'default_timezone', 'dt_O_type', 'get_timezone', 'is_datetime64_any_dtype', 'numpy', 'numpy2ri', 'original_converter', 'pandas', 'py2rpy', 'py2rpy_categoryseries', 'py2rpy_pandasdataframe', 'py2rpy_pandasindex', 'py2rpy_pandasseries', 'pytz', 'ri2py_vector', 'rinterface', 'rpy2py', 'rpy2py_dataframe', 'rpy2py_floatvector', 'rpy2py_intvector', 'rpy2py_listvector', 'tzlocal', 'warnings']
It turns out the developers have changed the name of the functions.
Since no one bothered to write down the way to do it with newer versions of rpy2:
Conversion is done using a localconverter block which automatically converts from pandas dataframe to r dataframe and back.
import pandas as pd
import rpy2.robjects as ro
from rpy2.robjects.packages import importr
from rpy2.robjects import pandas2ri
from rpy2.robjects.conversion import localconverter
pd_df = pd.DataFrame({'int_values': [1,2,3],
'str_values': ['abc', 'def', 'ghi']})
base = importr('base')
with localconverter(ro.default_converter + pandas2ri.converter):
df_summary = base.summary(pd_df)
You are likely using documentation/code written for a different version of rpy2 than what you have installed.
If using the latest release, consider checking the documentation for it:
https://rpy2.github.io/doc/v3.0.x/html/generated_rst/pandas.html
For anyone having issues with localconverter, here is an alternative way to convert a df of type rpy2.robjects.vectors.ListVector to a pandas dataframe. This solution drops columns names
pd.Dataframe(np.array(df).reshape((nrows, ncols)))
I created a Qt resource file with all my ui files inside of it, I compiled that with pyrcc4 command-line in a python file, and then I loaded the ui files using loadUi. Here an example:
#!/usr/bin/env python
#-*- coding:utf-8 -*-
import os
import sys
from PyQt4.QtCore import Qt, QFile
from PyQt4.uic import loadUi
from PyQt4.QtGui import QDialog
from xarphus.gui import ui_rc
# I import the compiled qt resource file named ui_rc
BASE_PATH = os.path.dirname(os.path.abspath(__file__))
#UI_PATH = os.path.join(BASE_PATH, 'gui', 'create_user.ui')
UI_PATH = QFile(":/ui_file/create_user.ui")
# I want to load those compiled ui files,
# so I just create QFile.
class CreateUser_Window(QDialog):
def __init__(self, parent):
QDialog.__init__(self, parent)
# I open the created QFile
UI_PATH.open(QFile.ReadOnly)
# I read the QFile and load the ui file
self.ui_create_user = loadUi(UI_PATH, self)
# After then I close it
UI_PATH.close()
Well its works fine, but I have a problem. When I open the GUI-window once, everything works fine. After closing the window I try to open the same GUI-window again, I get ja long traceback.
Traceback (most recent call last): File "D:\Dan\Python\xarphus\xarphus\frm_mdi.py", line 359, in
create_update_form self.update_form = Update_Window(self) File
"D:\Dan\Python\xarphus\xarphus\frm_update.py", line 135, in init
self.ui_update = loadUi(UI_PATH, self) File
"C:\Python27\lib\site-packages\PyQt4\uic__init__.py", line 238, in
loadUi return DynamicUILoader(package).loadUi(uifile, baseinstance,
resource_suffix) File
"C:\Python27\lib\site-packages\PyQt4\uic\Loader\loader.py", line 71,
in loadUi return self.parse(filename, resource_suffix, basedir) File
"C:\Python27\lib\site-packages\PyQt4\uic\uiparser.py", line 984, in
parse document = parse(filename) File
"C:\Python27\lib\xml\etree\ElementTree.py", line 1182, in parse
tree.parse(source, parser) File
"C:\Python27\lib\xml\etree\ElementTree.py", line 657, in parse
self._root = parser.close() File
"C:\Python27\lib\xml\etree\ElementTree.py", line 1654, in close
self._raiseerror(v) File "C:\Python27\lib\xml\etree\ElementTree.py",
line 1506, in _raiseerror raise err xml.etree.ElementTree.ParseError:
no element found: line 1, column 0
Can everyone help me?
Maybe I have a solution, but I don't know if that is a perfectly pythonic.
Well, we know all python projects have an __ init __-file. We need it for initializing Python packages, right? Well I thought: Why not use this file? What did I do? I define in the __ init __ -file a function like so:
#!/usr/bin/env python
#-*- coding:utf-8 -*-
from PyQt4.uic import loadUi
from PyQt4.QtCore import Qt, QFile
def ui_load_about(self):
uiFile = QFile(":/ui_file/about.ui")
uiFile.open(QFile.ReadOnly)
self.ui_about = loadUi(uiFile)
uiFile.close()
return self.ui_about
Now in my "About_Window"-class I do this:
#!/usr/bin/env python
#-*- coding:utf-8 -*-
import os
import sys
from PyQt4.QtCore import Qt, QFile
from PyQt4.uic import loadUi
from PyQt4.QtGui import QDialog
import __init__ as ui_file
class About_Window(QDialog):
def __init__(self, parent):
QDialog.__init__(self, parent)
self.ui_about = ui_file.ui_load_about(self)
You see I importe the package-file (__ init __-file) as ui_file and then I call the function and save the return of the function in the variable self.ui_about.
In my case I open the About_Window from a MainWindow(QMainWindow), and it looks like so:
def create_about_form(self):
self.ui_about = About_Window(self)
# Now when I try to show (show()-method) a window I get two windows
# The reason is: I open and load the ui files from compiled
# qt resorce file that was define in __init__-module.
# There is a function that opens the resource file, reads
# the ui file an closes and returns the ui file back
# That's the reason why I have commented out this method
#self.ui_about.show()
You see I commented out the show()-method. It works without this method. I only define the About_Window()-class. Well I know that isn't maybe the best solution, but it works. I can open the window again and again without traceback.
If you have a better solution or idea let me know :-)