trying to convert .py into .exe, and when i go to the dist folder to run the .exe generated i get a black/blank cmd screen and nothing happens - pyinstaller
any thoughts? per the warning log, it looks like i am missing a ton of modules (~200) but i am not sure what is relevant or if these are even causing the issue. below is a list of all the missing modules.
Types if import:
top-level: imported at the top-level - look at these first
conditional: imported within an if-statement
delayed: imported within a function
optional: imported within a try-except-statement
IMPORTANT: Do NOT post this list to the issue-tracker. Use it as a basis for
tracking down the missing module yourself. Thanks!
missing module named pyimod02_importers - imported by C:\Users\jwebb007\AppData\Local\Programs\Python\Python39\Lib\site-packages\PyInstaller\hooks\rthooks\pyi_rth_pkgutil.py (top-level), C:\Users\jwebb007\AppData\Local\Programs\Python\Python39\Lib\site-packages\PyInstaller\hooks\rthooks\pyi_rth_pkgres.py (top-level)
missing module named StringIO - imported by PyInstaller.lib.modulegraph._compat (conditional), PyInstaller.lib.modulegraph.zipio (conditional), setuptools._vendor.six (conditional), six (conditional), pkg_resources._vendor.six (conditional)
missing module named org - imported by pickle (optional)
missing module named _posixsubprocess - imported by subprocess (optional), multiprocessing.util (delayed)
missing module named grp - imported by subprocess (optional), shutil (optional), tarfile (optional), pathlib (delayed, optional), distutils.archive_util (optional)
missing module named pwd - imported by posixpath (delayed, conditional), subprocess (optional), shutil (optional), tarfile (optional), pathlib (delayed, conditional, optional), http.server (delayed, optional), webbrowser (delayed), netrc (delayed, conditional), getpass (delayed), distutils.util (delayed, conditional, optional), distutils.archive_util (optional)
missing module named posix - imported by os (conditional, optional), shutil (conditional)
missing module named resource - imported by posix (top-level), test.support (delayed, conditional, optional)
missing module named termios - imported by tty (top-level), getpass (optional)
missing module named _frozen_importlib_external - imported by importlib._bootstrap (delayed), importlib (optional), importlib.abc (optional), zipimport (top-level)
excluded module named _frozen_importlib - imported by importlib (optional), importlib.abc (optional), zipimport (top-level), PyInstaller.loader.pyimod01_archive (delayed)
missing module named pep517 - imported by importlib.metadata (delayed)
missing module named readline - imported by cmd (delayed, conditional, optional), code (delayed, conditional, optional), pdb (delayed, optional)
missing module named __builtin__ - imported by pkg_resources._vendor.pyparsing (conditional), setuptools._vendor.pyparsing (conditional)
missing module named ordereddict - imported by pkg_resources._vendor.pyparsing (optional), setuptools._vendor.pyparsing (optional)
missing module named 'org.python' - imported by copy (optional), xml.sax (delayed, conditional), setuptools.sandbox (conditional)
missing module named _manylinux - imported by pkg_resources._vendor.packaging.tags (delayed, optional), setuptools._vendor.packaging.tags (delayed, optional)
missing module named 'pkg_resources.extern.pyparsing' - imported by pkg_resources._vendor.packaging.markers (top-level), pkg_resources._vendor.packaging.requirements (top-level)
missing module named 'win32com.shell' - imported by pkg_resources._vendor.appdirs (conditional, optional)
missing module named 'com.sun' - imported by pkg_resources._vendor.appdirs (delayed, conditional, optional)
missing module named com - imported by pkg_resources._vendor.appdirs (delayed)
missing module named win32api - imported by distutils.msvccompiler (optional), pkg_resources._vendor.appdirs (delayed, conditional, optional)
missing module named win32com - imported by pkg_resources._vendor.appdirs (delayed)
missing module named _winreg - imported by platform (delayed, optional), pkg_resources._vendor.appdirs (delayed, conditional)
missing module named pkg_resources.extern.packaging - imported by pkg_resources.extern (top-level), pkg_resources (top-level)
missing module named pkg_resources.extern.appdirs - imported by pkg_resources.extern (top-level), pkg_resources (top-level)
missing module named 'pkg_resources.extern.six.moves' - imported by pkg_resources (top-level), pkg_resources._vendor.packaging.requirements (top-level)
missing module named pkg_resources.extern.six - imported by pkg_resources.extern (top-level), pkg_resources (top-level)
missing module named 'java.lang' - imported by platform (delayed, optional), xml.sax._exceptions (conditional)
missing module named vms_lib - imported by platform (delayed, optional)
missing module named java - imported by platform (delayed)
missing module named _posixshmem - imported by multiprocessing.resource_tracker (conditional), multiprocessing.shared_memory (conditional)
missing module named multiprocessing.BufferTooShort - imported by multiprocessing (top-level), multiprocessing.connection (top-level)
missing module named multiprocessing.AuthenticationError - imported by multiprocessing (top-level), multiprocessing.connection (top-level)
missing module named multiprocessing.get_context - imported by multiprocessing (top-level), multiprocessing.pool (top-level), multiprocessing.managers (top-level), multiprocessing.sharedctypes (top-level)
missing module named multiprocessing.TimeoutError - imported by multiprocessing (top-level), multiprocessing.pool (top-level)
missing module named multiprocessing.set_start_method - imported by multiprocessing (top-level), multiprocessing.spawn (top-level)
missing module named multiprocessing.get_start_method - imported by multiprocessing (top-level), multiprocessing.spawn (top-level)
missing module named _scproxy - imported by urllib.request (conditional)
missing module named numarray - imported by numpy.distutils.system_info (delayed, conditional, optional)
missing module named Numeric - imported by numpy.distutils.system_info (delayed, conditional, optional)
missing module named win32con - imported by distutils.msvccompiler (optional)
missing module named _curses - imported by curses (top-level), curses.has_key (top-level)
missing module named 'nose.plugins' - imported by numpy.testing._private.noseclasses (top-level), numpy.testing._private.nosetester (delayed)
missing module named 'nose.util' - imported by numpy.testing._private.noseclasses (top-level)
missing module named psutil - imported by numpy.testing._private.utils (delayed, optional)
missing module named _dummy_thread - imported by cffi.lock (conditional, optional), numpy.core.arrayprint (optional)
missing module named numpy.core.result_type - imported by numpy.core (delayed), numpy.testing._private.utils (delayed)
missing module named numpy.core.float_ - imported by numpy.core (delayed), numpy.testing._private.utils (delayed)
missing module named numpy.core.number - imported by numpy.core (delayed), numpy.testing._private.utils (delayed)
missing module named numpy.core.object_ - imported by numpy.core (top-level), numpy.linalg.linalg (top-level), numpy.testing._private.utils (delayed)
missing module named numpy.core.all - imported by numpy.core (top-level), numpy.linalg.linalg (top-level), numpy.testing._private.utils (delayed)
missing module named numpy.core.bool_ - imported by numpy.core (delayed), numpy.testing._private.utils (delayed)
missing module named numpy.core.inf - imported by numpy.core (delayed), numpy.testing._private.utils (delayed)
missing module named numpy.core.array2string - imported by numpy.core (delayed), numpy.testing._private.utils (delayed)
missing module named numpy.core.signbit - imported by numpy.core (delayed), numpy.testing._private.utils (delayed)
missing module named numpy.core.isscalar - imported by numpy.core (delayed), numpy.testing._private.utils (delayed), numpy.lib.polynomial (top-level)
missing module named win32pdh - imported by numpy.testing._private.utils (delayed, conditional)
missing module named numpy.core.isinf - imported by numpy.core (delayed), numpy.testing._private.utils (delayed)
missing module named numpy.core.errstate - imported by numpy.core (top-level), numpy.linalg.linalg (top-level), numpy.testing._private.utils (delayed)
missing module named numpy.core.isfinite - imported by numpy.core (top-level), numpy.linalg.linalg (top-level), numpy.testing._private.utils (delayed)
missing module named numpy.core.isnan - imported by numpy.core (top-level), numpy.linalg.linalg (top-level), numpy.testing._private.utils (delayed)
missing module named numpy.core.array - imported by numpy.core (top-level), numpy.linalg.linalg (top-level), numpy.testing._private.utils (top-level), numpy.lib.polynomial (top-level)
missing module named numpy.core.isnat - imported by numpy.core (top-level), numpy.testing._private.utils (top-level)
missing module named numpy.core.ndarray - imported by numpy.core (top-level), numpy.testing._private.utils (top-level), numpy.lib.utils (top-level)
missing module named numpy.core.array_repr - imported by numpy.core (top-level), numpy.testing._private.utils (top-level)
missing module named numpy.core.arange - imported by numpy.core (top-level), numpy.testing._private.utils (top-level), numpy.fft.helper (top-level)
missing module named numpy.core.empty - imported by numpy.core (top-level), numpy.linalg.linalg (top-level), numpy.testing._private.utils (top-level), numpy.fft.helper (top-level)
missing module named numpy.core.float32 - imported by numpy.core (top-level), numpy.testing._private.utils (top-level)
missing module named numpy.core.intp - imported by numpy.core (top-level), numpy.linalg.linalg (top-level), numpy.testing._private.utils (top-level)
missing module named asyncio.DefaultEventLoopPolicy - imported by asyncio (delayed, conditional), asyncio.events (delayed, conditional)
missing module named numpy.core.linspace - imported by numpy.core (top-level), numpy.lib.index_tricks (top-level)
missing module named numpy.core.iinfo - imported by numpy.core (top-level), numpy.lib.twodim_base (top-level)
missing module named numpy.core.transpose - imported by numpy.core (top-level), numpy.lib.function_base (top-level)
missing module named numpy.core.asarray - imported by numpy.core (top-level), numpy.linalg.linalg (top-level), numpy.lib.utils (top-level), numpy.fft._pocketfft (top-level), numpy.fft.helper (top-level)
missing module named numpy.core.integer - imported by numpy.core (top-level), numpy.fft.helper (top-level)
missing module named numpy.core.sqrt - imported by numpy.core (top-level), numpy.linalg.linalg (top-level), numpy.fft._pocketfft (top-level)
missing module named numpy.core.conjugate - imported by numpy.core (top-level), numpy.fft._pocketfft (top-level)
missing module named numpy.core.swapaxes - imported by numpy.core (top-level), numpy.linalg.linalg (top-level), numpy.fft._pocketfft (top-level)
missing module named numpy.core.zeros - imported by numpy.core (top-level), numpy.linalg.linalg (top-level), numpy.fft._pocketfft (top-level)
missing module named numpy.core.sort - imported by numpy.core (top-level), numpy.linalg.linalg (top-level)
missing module named numpy.core.argsort - imported by numpy.core (top-level), numpy.linalg.linalg (top-level)
missing module named numpy.core.sign - imported by numpy.core (top-level), numpy.linalg.linalg (top-level)
missing module named numpy.core.count_nonzero - imported by numpy.core (top-level), numpy.linalg.linalg (top-level)
missing module named numpy.core.divide - imported by numpy.core (top-level), numpy.linalg.linalg (top-level)
missing module named numpy.core.matmul - imported by numpy.core (top-level), numpy.linalg.linalg (top-level)
missing module named numpy.core.asanyarray - imported by numpy.core (top-level), numpy.linalg.linalg (top-level)
missing module named numpy.core.atleast_2d - imported by numpy.core (top-level), numpy.linalg.linalg (top-level)
missing module named numpy.core.product - imported by numpy.core (top-level), numpy.linalg.linalg (top-level)
missing module named numpy.core.amax - imported by numpy.core (top-level), numpy.linalg.linalg (top-level)
missing module named numpy.core.amin - imported by numpy.core (top-level), numpy.linalg.linalg (top-level)
missing module named numpy.core.moveaxis - imported by numpy.core (top-level), numpy.linalg.linalg (top-level)
missing module named numpy.core.geterrobj - imported by numpy.core (top-level), numpy.linalg.linalg (top-level)
missing module named numpy.core.finfo - imported by numpy.core (top-level), numpy.linalg.linalg (top-level), numpy.lib.polynomial (top-level)
missing module named numpy.core.sum - imported by numpy.core (top-level), numpy.linalg.linalg (top-level)
missing module named numpy.core.fastCopyAndTranspose - imported by numpy.core (top-level), numpy.linalg.linalg (top-level)
missing module named numpy.core.multiply - imported by numpy.core (top-level), numpy.linalg.linalg (top-level)
missing module named numpy.core.add - imported by numpy.core (top-level), numpy.linalg.linalg (top-level)
missing module named numpy.core.dot - imported by numpy.core (top-level), numpy.linalg.linalg (top-level), numpy.lib.polynomial (top-level)
missing module named numpy.core.Inf - imported by numpy.core (top-level), numpy.linalg.linalg (top-level)
missing module named numpy.core.newaxis - imported by numpy.core (top-level), numpy.linalg.linalg (top-level)
missing module named numpy.core.complexfloating - imported by numpy.core (top-level), numpy.linalg.linalg (top-level)
missing module named numpy.core.inexact - imported by numpy.core (top-level), numpy.linalg.linalg (top-level)
missing module named numpy.core.cdouble - imported by numpy.core (top-level), numpy.linalg.linalg (top-level)
missing module named numpy.core.csingle - imported by numpy.core (top-level), numpy.linalg.linalg (top-level)
missing module named numpy.core.double - imported by numpy.core (top-level), numpy.linalg.linalg (top-level)
missing module named numpy.core.single - imported by numpy.core (top-level), numpy.linalg.linalg (top-level)
missing module named numpy.core.intc - imported by numpy.core (top-level), numpy.linalg.linalg (top-level)
missing module named numpy.core.empty_like - imported by numpy.core (top-level), numpy.linalg.linalg (top-level)
missing module named numpy.core.ufunc - imported by numpy.core (top-level), numpy.lib.utils (top-level)
missing module named numpy.core.ones - imported by numpy.core (top-level), numpy.lib.polynomial (top-level)
missing module named numpy.core.hstack - imported by numpy.core (top-level), numpy.lib.polynomial (top-level)
missing module named numpy.core.atleast_1d - imported by numpy.core (top-level), numpy.lib.polynomial (top-level)
missing module named numpy.core.atleast_3d - imported by numpy.core (top-level), numpy.lib.shape_base (top-level)
missing module named numpy.core.vstack - imported by numpy.core (top-level), numpy.lib.shape_base (top-level)
missing module named pickle5 - imported by numpy.compat.py3k (optional)
missing module named numpy.eye - imported by numpy (delayed), numpy.core.numeric (delayed)
missing module named numpy.recarray - imported by numpy (top-level), numpy.ma.mrecords (top-level)
missing module named numpy.expand_dims - imported by numpy (top-level), numpy.ma.core (top-level)
missing module named numpy.array - imported by numpy (top-level), numpy.ma.core (top-level), numpy.ma.extras (top-level), numpy.ma.mrecords (top-level)
missing module named numpy.iscomplexobj - imported by numpy (top-level), numpy.ma.core (top-level)
missing module named numpy.amin - imported by numpy (top-level), numpy.ma.core (top-level)
missing module named numpy.amax - imported by numpy (top-level), numpy.ma.core (top-level)
missing module named numpy.float64 - imported by numpy (top-level), numpy.array_api._typing (top-level)
missing module named numpy.float32 - imported by numpy (top-level), numpy.array_api._typing (top-level)
missing module named numpy.uint64 - imported by numpy (top-level), numpy.array_api._typing (top-level)
missing module named numpy.uint32 - imported by numpy (top-level), numpy.array_api._typing (top-level)
missing module named numpy.uint16 - imported by numpy (top-level), numpy.array_api._typing (top-level)
missing module named numpy.uint8 - imported by numpy (top-level), numpy.array_api._typing (top-level)
missing module named numpy.int64 - imported by numpy (top-level), numpy.array_api._typing (top-level)
missing module named numpy.int32 - imported by numpy (top-level), numpy.array_api._typing (top-level)
missing module named numpy.int16 - imported by numpy (top-level), numpy.array_api._typing (top-level)
missing module named numpy.int8 - imported by numpy (top-level), numpy.array_api._typing (top-level)
missing module named _ufunc - imported by numpy.typing (conditional)
missing module named numpy.bytes_ - imported by numpy (top-level), numpy.typing._array_like (top-level)
missing module named numpy.str_ - imported by numpy (top-level), numpy.typing._array_like (top-level)
missing module named numpy.void - imported by numpy (top-level), numpy.typing._array_like (top-level)
missing module named numpy.object_ - imported by numpy (top-level), numpy.typing._array_like (top-level)
missing module named numpy.datetime64 - imported by numpy (top-level), numpy.typing._array_like (top-level)
missing module named numpy.timedelta64 - imported by numpy (top-level), numpy.typing._array_like (top-level)
missing module named numpy.number - imported by numpy (top-level), numpy.typing._array_like (top-level)
missing module named numpy.complexfloating - imported by numpy (top-level), numpy.typing._array_like (top-level)
missing module named numpy.floating - imported by numpy (top-level), numpy.typing._array_like (top-level)
missing module named numpy.integer - imported by numpy (top-level), numpy.typing._array_like (top-level), numpy.ctypeslib (top-level)
missing module named numpy.unsignedinteger - imported by numpy (top-level), numpy.typing._array_like (top-level)
missing module named numpy.bool_ - imported by numpy (top-level), numpy.typing._array_like (top-level), numpy.ma.core (top-level), numpy.ma.mrecords (top-level)
missing module named numpy.generic - imported by numpy (top-level), numpy.typing._array_like (top-level)
missing module named numpy.dtype - imported by numpy (top-level), numpy.typing._array_like (top-level), numpy.array_api._typing (top-level), numpy.ma.mrecords (top-level), numpy.ctypeslib (top-level)
missing module named numpy.ndarray - imported by numpy (top-level), numpy.typing._array_like (top-level), numpy.ma.core (top-level), numpy.ma.extras (top-level), numpy.ma.mrecords (top-level), numpy.ctypeslib (top-level), pandas.compat.numpy.function (top-level)
missing module named numpy.ufunc - imported by numpy (top-level), numpy.typing (top-level)
missing module named numpy.histogramdd - imported by numpy (delayed), numpy.lib.twodim_base (delayed)
missing module named xlwt - imported by pandas.io.excel._xlwt (delayed, conditional)
missing module named xlsxwriter - imported by pandas.io.excel._xlsxwriter (delayed)
missing module named defusedxml - imported by openpyxl.xml (delayed, optional)
missing module named 'defusedxml.ElementTree' - imported by openpyxl.xml.functions (conditional)
missing module named htmlentitydefs - imported by lxml.html.soupparser (optional)
missing module named BeautifulSoup - imported by lxml.html.soupparser (optional)
missing module named bs4 - imported by pandas.io.html (delayed), lxml.html.soupparser (optional)
missing module named urlparse - imported by lxml.html (optional), lxml.ElementInclude (optional), lxml.html.html5parser (optional)
missing module named urllib2 - imported by lxml.ElementInclude (optional), lxml.html.html5parser (optional)
missing module named 'html5lib.treebuilders' - imported by lxml.html.html5parser (top-level)
missing module named html5lib - imported by lxml.html._html5builder (top-level), lxml.html.html5parser (top-level)
missing module named PIL - imported by PyInstaller.building.icon (delayed, optional), PyInstaller.building.splash (optional), openpyxl.drawing.image (optional)
missing module named tests - imported by openpyxl.reader.excel (optional)
missing module named 'odf.config' - imported by pandas.io.excel._odswriter (delayed)
missing module named 'odf.style' - imported by pandas.io.excel._odswriter (delayed)
missing module named 'odf.text' - imported by pandas.io.excel._odfreader (delayed), pandas.io.excel._odswriter (delayed)
missing module named 'odf.table' - imported by pandas.io.excel._odfreader (delayed), pandas.io.excel._odswriter (delayed)
missing module named 'odf.opendocument' - imported by pandas.io.excel._odfreader (delayed), pandas.io.excel._odswriter (delayed)
missing module named xlrd - imported by pandas.io.excel._xlrd (delayed), pandas.io.excel._base (delayed, conditional)
missing module named 'odf.element' - imported by pandas.io.excel._odfreader (delayed)
missing module named 'odf.namespaces' - imported by pandas.io.excel._odfreader (delayed)
missing module named odf - imported by pandas.io.excel._odfreader (delayed)
missing module named botocore - imported by pandas.io.common (delayed, conditional, optional)
missing module named 'matplotlib.pyplot' - imported by pandas.io.formats.style (optional), pandas._testing._io (delayed), pandas._testing.asserters (delayed)
missing module named matplotlib - imported by pandas.io.formats.style (optional)
missing module named markupsafe - imported by pandas.io.formats.style_render (top-level)
missing module named 'IPython.core' - imported by pandas.io.formats.printing (delayed, conditional)
missing module named IPython - imported by pandas.io.formats.printing (delayed)
missing module named pyarrow - imported by pandas.core.arrays.masked (delayed), pandas.core.arrays.boolean (delayed, conditional), pandas.core.arrays.numeric (delayed, conditional), pandas.core.arrays._arrow_utils (top-level), pandas.core.arrays.interval (delayed), pandas.core.arrays.period (delayed), pandas.io.feather_format (delayed), pandas.core.arrays.string_ (delayed, conditional), pandas.core.arrays.string_arrow (conditional), pandas.core.dtypes.dtypes (delayed, conditional), pandas.compat.pyarrow (optional)
missing module named six.moves.range - imported by six.moves (top-level), dateutil.rrule (top-level)
runtime module named six.moves - imported by dateutil.tz.tz (top-level), dateutil.tz._factories (top-level), dateutil.tz.win (top-level), dateutil.rrule (top-level)
missing module named dateutil.tz.tzfile - imported by dateutil.tz (top-level), dateutil.zoneinfo (top-level)
missing module named numba - imported by pandas.core.util.numba_ (delayed, conditional), pandas.core.window.numba_ (delayed, conditional), pandas.core.window.online (delayed, conditional), pandas.core._numba.executor (delayed, conditional), pandas.core.groupby.numba_ (delayed, conditional), pandas.core._numba.kernels.mean_ (top-level), pandas.core._numba.kernels.shared (top-level), pandas.core._numba.kernels.min_max_ (top-level), pandas.core._numba.kernels.sum_ (top-level), pandas.core._numba.kernels.var_ (top-level)
missing module named numexpr - imported by pandas.core.computation.expressions (conditional), pandas.core.computation.engines (delayed)
missing module named 'pyarrow.parquet' - imported by pandas.io.parquet (delayed)
missing module named Foundation - imported by pandas.io.clipboard (delayed, conditional, optional)
missing module named AppKit - imported by pandas.io.clipboard (delayed, conditional, optional)
missing module named PyQt4 - imported by pandas.io.clipboard (delayed, conditional, optional)
missing module named PyQt5 - imported by pandas.io.clipboard (delayed, conditional, optional)
missing module named qtpy - imported by pandas.io.clipboard (delayed, conditional, optional)
missing module named 'sqlalchemy.engine' - imported by pandas.io.sql (delayed)
missing module named 'sqlalchemy.types' - imported by pandas.io.sql (delayed, conditional)
missing module named 'sqlalchemy.schema' - imported by pandas.io.sql (delayed)
missing module named sqlalchemy - imported by pandas.io.sql (delayed, conditional)
missing module named tables - imported by pandas.io.pytables (delayed, conditional)
missing module named 'scipy.stats' - imported by pandas.core.nanops (delayed, conditional)
missing module named scipy - imported by pandas.core.missing (delayed)
missing module named 'pyarrow.compute' - imported by pandas.core.arrays.string_arrow (conditional)
missing module named sets - imported by pytz.tzinfo (optional)
missing module named UserDict - imported by pytz.lazy (optional)
missing module named 'scipy.sparse' - imported by pandas.core.arrays.sparse.array (conditional), pandas.core.arrays.sparse.scipy_sparse (delayed, conditional), pandas.core.arrays.sparse.accessor (delayed), pandas.core.dtypes.common (delayed, conditional, optional)
missing module named hypothesis - imported by pandas.util._tester (delayed, optional)
missing module named pytest - imported by pandas._testing._io (delayed), pandas._testing (delayed), pandas.util._tester (delayed, optional)
missing module named pandas.ExtensionArray - imported by pandas (conditional), pandas.core.construction (conditional)
missing module named pandas.UInt64Index - imported by pandas (conditional), pandas.core.dtypes.generic (conditional)
missing module named pandas.Int64Index - imported by pandas (conditional), pandas.core.dtypes.generic (conditional)
missing module named pandas.Float64Index - imported by pandas (conditional), pandas.core.dtypes.generic (conditional), pandas.core.internals.blocks (conditional), pandas.core.indexes.datetimes (conditional), pandas.core.internals.array_manager (conditional)
I am going to assume that you are using pyinstaller to convert it. Instead of double clicking the exe that you have made from the Python file, I would run it via the terminal or command prompt. Maybe it will show you what error you have. And try using https://pypi.org/project/auto-py-to-exe/
Related
I'm getting following Error while doing terraform apply
[root#terraform ~]# terraform plan -out terraform.tfstate ╷ │ Error: Error acquiring the state lock │ │ Error message: 2 problems: │ │ - Unsupported state file format: The state file could not be parsed as JSON: │ syntax error at byte offset 1. │ - Unsupported state file format: The state file does not have a "version" │ attribute, which is required to identify the format version. │ │ Terraform acquires a state lock to protect the state from being written │ by multiple users at the same time. Please resolve the issue above and try │ again. For most commands, you can disable locking with the "-lock=false" │ flag, but this is not recommended.
#Ajay Kumar hi, problem is you are trying to pass .tfstate file as input and terraform is not able to parse same. terraform apply command expect .out file or .json file as input. Whereas .tfstate file is one where desired state of your infra is maintained (this is auto created by terraform itself). In this case what you can do is Create one .tfvars file with all your configuration parameters (this will be served as input to terraform apply command) touch input.tfvars Next pass this input.tfvars file as input to your terraform apply command using -var-file flag and -out flag to generate .out file terraform plan -var-file=input.tfvars -out inputplan.out Next, you can use terraform apply command for above generated .out file $ terraform apply "inputplan.out" Refer this article for more information https://medium.com/#hegdetapan2609/how-to-install-packages-on-multiple-similar-instances-at-same-time-using-terraform-182582af5260
Using multiple configs in the same group to interpolate values in a yaml file
In Hydra I have the following configuration:
├── conf
│ ├── config.yaml
│ ├── callbacks
│ │ ├── callback_01.yaml
│ │ └── callback_02.yaml
│ └── trainer
│ ├── default.yaml
The callbacks have a structure like this:
_target_: callback_to_instantiate
I need to pass to the trainer/default.yaml both the callbacks through interpolation.
I tried like this:
_target_: pytorch_lightning.Trainer
callbacks:
- ${callbacks.callback_01}
- ${callbacks.callback_02}
With the config.yaml like this:
defaults:
- _self_
- trainer: default
I did also other trials but it doesn't seem to work. Is there a way to interpolate like that in a yaml file by using two or more yaml files that are in the config group?
I would like if possible to keep this structure.
Currently the recommended approach is:
compose a mapping whose values are the desired callbacks, and then
use the oc.dict.values OmegaConf resolver to get a list of values from that dictionary.
# conf/config.yaml
defaults:
- callbacks#_callback_dict.cb1: callback_01
- callbacks#_callback_dict.cb2: callback_02
- trainer: default
- _self_
# conf/trainer/default.yaml
_target_: pytorch_lightning.Trainer
callbacks: ${oc.dict.values:_callback_dict}
# my_app.py
from typing import Any
import hydra
from omegaconf import DictConfig, OmegaConf
#hydra.main(config_path="conf", config_name="config")
def app(cfg: DictConfig) -> Any:
OmegaConf.resolve(cfg)
del cfg._callback_dict
print(OmegaConf.to_yaml(cfg))
if __name__ == "__main__":
app()
At the command line:
$ python my_app.py
trainer:
_target_: pytorch_lightning.Trainer
callbacks:
- _target_: callback_to_instantiate_01
- _target_: callback_to_instantiate_02
For reference, there is an open issue on Hydra's github repo advocating for an improved user experience around
Airflow 2 - ModuleNotFoundError: No module named 'airflow.operators.sensors'
After upgrading to Airflow 2, I got that error in some DAGs: ModuleNotFoundError: No module named 'airflow.operators.sensors'
The new one that works: from airflow.sensors.base import BaseSensorOperator Chosen answer doesn't work for newer versions of Airflow.
I resolved by change the import. old one from airflow.operators.sensors import BaseSensorOperator the new one that works from airflow.sensors import BaseSensorOperator BaseSensorOperator
I was trying to import ExternalTaskSensor and my research led me to this post, it turned out to be this class. The correct import for me was from airflow.sensors.external_task import ExternalTaskSensor Just FYI in case anyone runs into this in the future.
For Airflow 2.1.1 I first installed Amazon provider: pip install apache-airflow-providers-amazon and then imported S3KeySensor: from airflow.providers.amazon.aws.sensors.s3_key import S3KeySensor
Cannot import name 'operator'
I'm trying to create my first custom operator. I have a airflow folder, a dags folder and a plugins folder. Within the plugins folder I have the operators folder. My operator is in the operators folder. In my operators folder I have the following __init__.py file: from operators.my_operator import MyOperator __all__ = [ 'MyOperator', ] In my plugins folder I have the following __init__.py file: from airflow.plugins_manager import AirflowPlugin import operators # Defining the plugin class class NewOperator(AirflowPlugin): name = "test_operator" operators = [ operators.MyOperator ] and then from my dag I'm calling my operator like this from airflow.operators import MyOperator From the UI I see the error cannot import name 'MyOperator' What's wrong?
How are imports chosen for consoleProject?
I'm using sbt 0.13.5-SNAPSHOT built from the sources. When I ran consoleProject command, I could see the following imports: > consoleProject [info] Starting scala interpreter... [info] import sbt._ import Keys._ import currentState._ import extracted._ import cpHelpers._ Why are these imports important and selected for the command? I can explain the first two imports, but vaguely recognise the others.