Problem Running Xgboost in Executable File - pyinstaller

I am using a saved Xgboost model in a executable file created with PyInstaller. I setup a virtual env and downloaded Xgboost and ensured it ran but after I create the exe and run the exe I get an error about xgboost.core:
ModuleNotFoundError: No module nemed 'xgboost.core'

Actually I can't see any import problem with xgboost, first, make sure that you are using the latest version inside your env with pip install -U xgboost next try to add xgboost.core as a hidden-import and add xgboost's DLLs as data-files.
Suppose that your virtualenv is named env, use below command to generate your executable:
├───myscript.py
├───env
Code:
import traceback
try:
from xgboost import core
input("xgboost.core imported successfully!")
except Exception:
traceback.print_exc()
input("Import Error!")
Command:
(env) > pyinstaller myscript.py -F --hidden-import=xgboost.core --add-data "./env/xgboost/*;xgboost/"
--add-data "./env/Lib/site-packages/xgboost/VERSION;xgboost/"

The answer by #Masoud Rahimi did not work for me. What did was running pyinstaller using the --collect-all option:
pyinstaller -D <app_name>.py --noconfirm --collect-all "xgboost"
See this issue and the arguments in the pyinstaller manual

Related

'cfgrib' is not showing in xarray engine list, when the python code is packaged with pyinstaller to get exe file

I am trying to create an executable file from python code with 'Pyinstaller' package. The code runs in conda environment where the cfgrib selfcheck is good:
Found: eccodes v2.27.0.
your system is ready
However when the executable created by pyinstaller runs, the error is:
ValueError: unrecognized engine cfgrib must be one of: ['scipy','store']
I adde 'eccodes', 'python-eccodes', 'cfgrib', 'netcdf4' to the hidden imports, but no success. Does anybody have any idea to fix this issue!

AttributeError: type object 'pandas._libs.tslib._TSObject' has no attribute' _reduce_cython_'

I have a created a tkinter app for Convolution neural network to identify images. I am trying to compile the py file with pyinstaller but i am receiving this error:
AttributeError: type object 'pandas._libs.tslib._TSObject' has no attribute' _reduce_cython_'
i have also attached the screenshot of the error
Just re-start in case you are using Jupyter notebook. It worked for me. Also make sure, you are working in correct python environment.
This problem just happened to me when i try to create an app of an uncompleted project for testing purpose.
I have pandas imported, but i didnt call it in my script.
Removing the import line or call pandas was the solution.
You need to have cython too,
pip install cython or conda install cython
pyinstaller --onefile --hidden-import pandas._libs.tslibs.timedeltas program.py
I solved it using
pyinstaller --onefile --hidden-import pandas._libs.tslibs.timedeltas myScript.py
and it is working now
The other solutions did not work for me but what did is:
add import pandas at the very top of conftest.py
add ignore:numpy\.ufunc size changed.+:RuntimeWarning to the filterwarnings option of pytest.ini
upgraded freezegun to the latest version

Adding the ibapi library to PYTHONPATH module in Spyder (Python 3.6)

I am trying to simply connect to the ibapi (Interactive Brokers API), but I am having some technical troubles with Spyder.
What I did so far:
I installed the latest version for Windows from here
I provided the following path to the PYTHONPATH manager in Spyder: C:\TWS API\source\pythonclient\ibapi afterwards I restarted Spyder
When I simply type import ibapi I get the same error message: ModuleNotFoundError: No module named 'ibapi'
What I am doing wrong here and how can I fix this simple error?
I don't use python very much but I'm pretty sure you have to install the ibapi. I never used the PYTHONPATH and mine works fine in Spyder and Jupyter. I'm using Anaconda.
I run the Anaconda prompt (just activates the conda environment) and navigate to the dir C:\TWS API\source\pythonclient and run python setup.py install
Then everything works. I think many python packages need to be installed like this.
note: That's not my install dir, I just copied yours. There may be a problem with a space in the path. Try renaming to C:\TWSAPI\... if you still have a problem.
Note: after python setup.py install, you might need to restart spyder. If doesn't help, then copy the newly-created ibapi folder to:
C:\Users\iuzeri\AppData\Local\Programs\Python\Python37-32\Lib\site-packages
or
C:\Program Files (x86)\Python37-32\Lib\site-packages
copy your ibapi here
C:/ProgramData/Anaconda3/Lib/site-packages/ibapi

Jupyter Notebook in virtual environment doesn't see the virtual env packages

I'm trying to use a Jupyter Notebook in a virtual environment.
I have created a new virtualenv virtualenv ker12
+ activate + installed a specific version of keras or any other library.
also as mentioned in Using a virtualenv in an IPython notebook I did:
pip install ipykernel
and
python -m ipykernel install --user --name=my-virtualenv-name
when I run the notebook and write
! which jupyter the output is correct
/Users/myname/virtualenv/ker12/bin/python
but when I try to import a library, for example import keras there is an error.
ImportError: No module named keras
But from the other side when I write pip freeze | grep Keras
the output is:
Keras==1.2.0
UPDATE 1:
this problem is not related to Keras it occurs with any other library (for example pandas)
If I print os.path the output is following:
<'module 'posixpath' from /usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/posixpath.pyc>
From a "command line python" the os.path looks correct
<'module 'posixpath' from '/Users/my_name/virtualenv/ker12/lib/python2.7/posixpath.pyc'>
UPDATE 2:
If I print sys.path from terminal and jupyter the output is also different:
from terminal
/Users/myname/virtualenv/ker12/lib/python27.zip
/Users/myname/virtualenv/ker12/lib/python2.7
/Users/myname/virtualenv/ker12/lib/python2.7/plat-darwin
/Users/myname/virtualenv/ker12/lib/python2.7/plat-mac
/Users/myname/virtualenv/ker12/lib/python2.7/plat-mac/lib-scriptpackages
/Users/myname/virtualenv/ker12/lib/python2.7/lib-tk
/Users/myname/virtualenv/ker12/lib/python2.7/lib-old
/Users/myname/virtualenv/ker12/lib/python2.7/lib-dynload
/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7
/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin
/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk
/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac /usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages
/Users/myname/virtualenv/ker12/lib/python2.7/site-packages
from JUPYTER
/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python27.zip
/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7
/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin
/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac /usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages
/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk
/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old
/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload
/usr/local/lib/python2.7/site-packages
/usr/local/lib/python2.7/site-packages/IPython/extensions
/Users/myname/.ipython `
the solution is to open jupyter notebook with following command:
~/virtualenv/my_venv_name/bin/jupyter-notebook
You should not install ipykernel - instead, you should go for a full Jupyter installation (pip install jupyter) inside your virtual environment. Additionally, be sure that you don't create your virtual environment with the --system-site-packages option.
See also this answer.

mpi4py references lipmpi.so.12 as lipmpi.so.1 (says no such file or directory)

I'm new to MPI, but I have been trying to use it on a cluster with OpenMPI. I'm having the following problem:
$ python -c "from mpi4py import MPI"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: libmpi.so.1: cannot open shared object file: No such file or directory
I've done some research and tried a few things including:
module load mpi/openmpi-x86_64
which doesn't seem to change anything. My LD_LIBRARY_PATH seems to be set correctly, but the desired "libmpi.so.1" does not exist. Instead, there is "libmpi.so.12":
$ echo $LD_LIBRARY_PATH
/usr/lib64/openmpi/lib:/usr/local/matio/lib:/usr/lib64/mysql:/usr/local/knitro/lib:/usr/local/gurobi/linux64/lib:/usr/local/lib:
$ls /usr/lib64/openmpi/lib
... libmpi.so libmpi.so.12 libmpi.so.12.0.0 libmpi_usempi.so ...
I can't uninstall/re-install mpi4py because it's outside my virtual environment / I don't have the permissions to uninstall it on the general cluster. I've seen this: Error because file libmpi.so.1 missing, but not sure how I can create a symbolic link / don't think I have permission to modify the folder.
I'm somewhat out of ideas: Not sure if it's possible to have a separate install of mpi4py on my virtualenv, or whether I can specify the correct file to mpi4py temporarily?
Any suggestions would be greatly appreciated!
Update: I ran find /usr -name libmpi.so.1 2>/dev/null as suggested, which returned usr/lib64/compat-openmpi16/lib/libmpi.so.1. Using export LD_LIBRARY_PATH=$LD_LIBRARY_PATH"/usr/lib64/compat-openmpi16/lib/", python -c "from mpi4py import MPI" runs without any problems. However, if I try mpiexec -np 2 python -c 'from mpi4py import MPI', I get the following error:
[[INVALID],INVALID] ORTE_ERROR_LOG: Not found in file ess_env_module.c at line 367
[[INVALID],INVALID]-[[28753,0],0] mca_oob_tcp_peer_try_connect: connect to 255.255.255.255:37376 failed: Network is unreachable (101)
Even adding to LD_LIBRARY_PATH did not work for me. I had to uninstall mpi4py, then install it manually with my mpicc path according to the instructions here - https://mpi4py.readthedocs.io/en/stable/install.html (using distutils section)
I solve this problem by
pip uninstall mpi4py
conda install mpi4py

Resources