Script compiled with pyinstaller is missing a .dll file, when the file is manually copied in the program's folder it just dies - pyinstaller

I have a python script which is basically a graphic interface (pysimpleguy) to a mysql database.
I am working in python 3.8; my dependencies are:
PySimpleGUI 4.55.1
sqlalchemy 1.3.20
pymysql 1.0.2
pandas 1.1.3
regex 2020.10.15
pillow 8.0.1
The code works and I'd like to compile it to .exe to distribute it to users in my organization.
I tried to compile it with:
pyinstaller -D .\db_interface_v3.6.1_release.py --debug=imports
However, pyinstaller throws some errors when compiling:
201667 INFO: Building COLLECT COLLECT-00.toc
Traceback (most recent call last):
File "c:\users\spit\anaconda3\lib\runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "c:\users\spit\anaconda3\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\Users\Spit\anaconda3\Scripts\pyinstaller.exe\__main__.py", line 7, in <module>
File "c:\users\spit\anaconda3\lib\site-packages\PyInstaller\__main__.py", line 124, in run
run_build(pyi_config, spec_file, **vars(args))
File "c:\users\spit\anaconda3\lib\site-packages\PyInstaller\__main__.py", line 58, in run_build
PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
File "c:\users\spit\anaconda3\lib\site-packages\PyInstaller\building\build_main.py", line 782, in main
build(specfile, kw.get('distpath'), kw.get('workpath'), kw.get('clean_build'))
File "c:\users\spit\anaconda3\lib\site-packages\PyInstaller\building\build_main.py", line 714, in build
exec(code, spec_namespace)
File "C:\Users\Spit\Desktop\DIPEx db parser\db_interface_v3.6.1_release.spec", line 37, in <module>
coll = COLLECT(exe,
File "c:\users\spit\anaconda3\lib\site-packages\PyInstaller\building\api.py", line 818, in __init__
self.__postinit__()
File "c:\users\spit\anaconda3\lib\site-packages\PyInstaller\building\datastruct.py", line 155, in __postinit__
self.assemble()
File "c:\users\spit\anaconda3\lib\site-packages\PyInstaller\building\api.py", line 866, in assemble
shutil.copy(fnm, tofnm)
File "c:\users\spit\anaconda3\lib\shutil.py", line 415, in copy
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "c:\users\spit\anaconda3\lib\shutil.py", line 261, in copyfile
with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\Spit\\Desktop\\DIPEx db parser\\dist\\db_interface_v3.6.1_release\\share\\jupyter\\lab\\staging\\node_modules\\.cache\\terser-webpack-p
lugin\\content-v2\\sha512\\2e\\ba\\cfce62ec1f408830c0335f2b46219d58ee5b068473e7328690e542d2f92f2058865c600d845a2e404e282645529eb0322aa4429a84e189eb6b58c1b97c1a'
If I try to run the compiled exe, I get an error regarding a specific .dll:
INTEL MKL ERROR: Impossibile trovare il modulo specificato. mkl_intel_thread.dll.
Intel MKL FATAL ERROR: Cannot load mkl_intel_thread.dll.
If I take this missing .dll from my Anaconda environment and copy it into the program's folder, when I try to run the .exe again it just dies without further messages:
import 'numpy.ma' # <pyimod03_importers.FrozenImporter object at 0x000001F6A455BEE0>
PS C:\Users\Spit\Desktop\DIPEx db parser\dist\db_interface_v3.6.1_release>
Any idea on how to sort it out?
Thanks!

Sorted out. As a future reference if someone stumbles upon this question, the error is caused by Windows' PATH_MAX limitation, preventing pyinstaller to find all the necessary files.
In order to disable said limitation: https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=cmd
Kudos to https://github.com/bwoodsend

Related

importlib.metadata.PackageNotFoundError: regex while building an .exe with pyinstaller

I am trying to build an executable with pyinstaller and I am getting the following errors:
Traceback (most recent call last):
File "transformers/utils/versions.py", line 105, in require_version
File "importlib/metadata.py", line 551, in version
File "importlib/metadata.py", line 524, in distribution
File "importlib/metadata.py", line 187, in from_name
importlib.metadata.PackageNotFoundError: regex
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "chatbot2.py", line 8, in <module>
File "PyInstaller/loader/pyimod03_importers.py", line 546, in exec_module
File "transformers/__init__.py", line 43, in <module>
File "PyInstaller/loader/pyimod03_importers.py", line 546, in exec_module
File "transformers/dependency_versions_check.py", line 41, in <module>
File "transformers/utils/versions.py", line 120, in require_version_core
File "transformers/utils/versions.py", line 107, in require_version
importlib.metadata.PackageNotFoundError: The 'regex!=2019.12.17' distribution was not found and is required by this application.
Try: pip install transformers -U or pip install -e '.[dev]' if you're working with git master
[30391] Failed to execute script 'chatbot2' due to unhandled exception!
I am using PyInstaller version 5.0.dev0 and Python version 3.9.1.
I tried pip3.9 install transformers, pip3.9 install regex, I tried adding hooks hook-regex.py and hook-transformers.py.
I would really appreciate any suggestions! Thank you!

problem launching Jupyter notebook or Jupyter lab anaconda

I have a problem launching the Jupyter notebook on a remote Windows computer. running Jupyter from conda prompt returns WinError 1326 as below:
Traceback (most recent call last):
File "D:\Anaconda3\lib\site-packages\traitlets\traitlets.py", line 535, in get
value = obj._trait_values[self.name]
KeyError: 'runtime_dir'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "D:\Anaconda3\Scripts\jupyter-notebook-script.py", line 10, in <module>
sys.exit(main())
File "D:\Anaconda3\lib\site-packages\jupyter_core\application.py", line 254, in launch_instance
return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
File "D:\Anaconda3\lib\site-packages\traitlets\config\application.py", line 844, in launch_instance
app.initialize(argv)
File "D:\Anaconda3\lib\site-packages\traitlets\config\application.py", line 87, in inner
return method(app, *args, **kwargs)
File "D:\Anaconda3\lib\site-packages\notebook\notebookapp.py", line 2127, in initialize
self.init_configurables()
File "D:\Anaconda3\lib\site-packages\notebook\notebookapp.py", line 1650, in init_configurables
connection_dir=self.runtime_dir,
File "D:\Anaconda3\lib\site-packages\traitlets\traitlets.py", line 575, in __get__
return self.get(obj, cls)
File "D:\Anaconda3\lib\site-packages\traitlets\traitlets.py", line 538, in get
default = obj.trait_defaults(self.name)
File "D:\Anaconda3\lib\site-packages\traitlets\traitlets.py", line 1578, in trait_defaults
return self._get_trait_default_generator(names[0])(self)
File "D:\Anaconda3\lib\site-packages\jupyter_core\application.py", line 85, in _runtime_dir_default
ensure_dir_exists(rd, mode=0o700)
File "D:\Anaconda3\lib\site-packages\jupyter_core\utils\__init__.py", line 11, in ensure_dir_exists
os.makedirs(path, mode=mode)
File "D:\Anaconda3\lib\os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
File "D:\Anaconda3\lib\os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
File "D:\Anaconda3\lib\os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
[Previous line repeated 3 more times]
File "D:\Anaconda3\lib\os.py", line 223, in makedirs
mkdir(name, mode)
OSError: [WinError 1326] Username or password is incorrect: 'MY_SERVER_ADDRESS'
I tried changing some values in config file but no luck! please help me fix this problem
This might help:->
steps include:->
1-open cmd
2-paste the path of your Jupyter folder
3-write "Jupyter notebook" and press enter you will redirect to Jupyter
4-still not working check your python path is configured or not and then go to step1,2,3:->
upvote if it's helpful
or
you can made another folder jupyter folder and then configure python
and install jupyter notebook and use above steps.

Is the working directory of the dagster main process different of the scheduler processes

I'm having an issue with the loading of a file from dagster code (setup, not pipelines). Say I have the following project structure:
pipelines
-app/
--environments
----schedules.yaml
--repository.py
--repository.yaml
When I run dagit while inside the project folder($cd project && dagit -y app/repository.yaml), this folder becomes the working dir and inside the repository.py I could load a file knowing the root is project
# repository.py
with open('app/evironments/schedules.yaml', 'r'):
# do something with the file
However, if I set up a schedule the pipelines in the project do not run. Checking the cron logs it seems the open line throws a file not found exception. I was wondering if this happens because the working directory is different when executing the cron.
For context, I'm loading a config file with parameters of cron_schedules for each pipeline. Also, here's the tail of the stacktrace in my case:
File "/home/user/.local/share/virtualenvs/pipelines-mfP13m0c/lib/python3.8/site-packages/dagster/core/definitions/handle.py", line 190, in from_yaml
return LoaderEntrypoint.from_file_target(
File "/home/user/.local/share/virtualenvs/pipelines-mfP13m0c/lib/python3.8/site-packages/dagster/core/definitions/handle.py", line 161, in from_file_target
module = import_module_from_path(module_name, os.path.abspath(python_file))
File "/home/user/.local/share/virtualenvs/pipelines-mfP13m0c/lib/python3.8/site-packages/dagster/seven/__init__.py", line 75, in import_module_from_path
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 783, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/user/pipelines/app/repository.py", line 28, in <module>
schedule_builder = ScheduleBuilder(settings.CRON_PRESET, settings.ENV_DICT)
File "/home/user/pipelines/app/schedules.py", line 12, in __init__
self.cron_schedules = self._load_schedules_yaml()
File "/home/user/pipelines/app/schedules.py", line 16, in _load_schedules_yaml
with open(path) as f:
FileNotFoundError: [Errno 2] No such file or directory: 'app/environments/schedules.yaml'
You could open the file using the absolute path of the file so that it opens correctly.
from dagster.utils import file_relative_path
with open(file_relative_path(__file__, './environments/schedules.yaml'), 'r'):
# do something with the file
All file_relative_path is simply doing the following, so you can call the os.path methods directly if you prefer:
def file_relative_path(dunderfile, relative_path):
os.path.join(os.path.dirname(dunderfile), relative_path)

iprof and iprof_totals profiling error

I get this error after trying :
openmdao iprof x.py
or
openmdao iprof_totals x.py
on my terminal. Any idea why it could be? Do we have a simple sample code where the iprof works smoothly.
Traceback (most recent call last):
File "/home/user/miniconda3/bin/openmdao", line 11, in
sys.exit(openmdao_cmd())
File "/home/user/miniconda3/lib/python3.6/site-packages/openmdao/utils/om.py", line 403, in openmdao_cmd
options.executor(options)
File "/home/user/miniconda3/lib/python3.6/site-packages/openmdao/devtools/iprofile.py", line 373, in _iprof_totals_exec
_iprof_py_file(options)
File "/home/user/miniconda3/lib/python3.6/site-packages/openmdao/devtools/iprofile.py", line 429, in _iprof_py_file
_finalize_profile()
File "/home/user/miniconda3/lib/python3.6/site-packages/openmdao/devtools/iprofile.py", line 183, in _finalize_profile
qfile, qclass, qname = find_qualified_name(filename, int(line), cache, full=False)
File "/home/user/miniconda3/lib/python3.6/site-packages/openmdao/devtools/iprof_utils.py", line 73, in find_qualified_name
with open(filename, 'Ur') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'packages/openmdao/jacobians/jacobian.py'
I looked into this and there is currently a bug when using 'openmdao iprof' and 'openmdao iprof_totals' on a version of OpenMDAO that was not installed from an OpenMDAO repository using 'pip install -e'. I put a story in our bug tracker to fix it.

Plone buildout error non-existant file

I have a Plone 4.3.3 site that is throwing an error on buildout:
Traceback (most recent call last):
File "/usr/local/Proforest4.3/buildout-cache/eggs/zc.buildout-2.2.5-py2.7.egg/zc/buildout/buildout.py", line 1946, in main
getattr(buildout, command)(args)
File "/usr/local/Proforest4.3/buildout-cache/eggs/zc.buildout-2.2.5-py2.7.egg/zc/buildout/buildout.py", line 626, in install
installed_files = self[part]._call(recipe.install)
File "/usr/local/Proforest4.3/buildout-cache/eggs/zc.buildout-2.2.5-py2.7.egg/zc/buildout/buildout.py", line 1370, in _call
return f()
File "/data/usr/local/Proforest4.3/buildout-cache/eggs/plone.recipe.precompiler-0.6-py2.7.egg/plone/recipe/precompiler/__init__.py", line 29, in install
return self._run()
File "/data/usr/local/Proforest4.3/buildout-cache/eggs/plone.recipe.precompiler-0.6-py2.7.egg/plone/recipe/precompiler/__init__.py", line 35, in _run
self._compile_eggs()
File "/data/usr/local/Proforest4.3/buildout-cache/eggs/plone.recipe.precompiler-0.6-py2.7.egg/plone/recipe/precompiler/__init__.py", line 67, in _compile_eggs
py_compile.compile(fn, None, None, True)
File "/usr/lib64/python2.7/py_compile.py", line 123, in compile
with open(cfile, 'wb') as fc:
IOError: [Errno 13] Permission denied: '/data/usr/local/Proforest4.3/test/src/proforest.content/proforest/content/behaviours/accordion.pyc'
accordion.py does exist, but not the .pyc version
Permissions on accordion.py seem correct (owned by plone_buildout etc.)
How do I resolve?
You user is trying to create that file, but filesystem permissions are wrong.
Check that the Plone effective-user is able to write in buildout folders.

Resources