How do you make packaged generated applications smaller?
I type in the terminal of Windows
PS C:\Users\Lenovo-pc> PyInstaller -w 123.py
But the package is huge, and the following modules are mainly imported
import time,threading,itertools
from tkinter import Label,Tk,Button,Text,END,PhotoImage,Entry,RAISED
from random import shuffle,choice
from datetime import datetime
from openpyxl import load_workbook
How do I make the resulting executable smaller?
Actually, you can't do much about that, because PyInstaller would bring each dependency beside your application output. On the other side sometimes it brings unnecessary modules with your output which should be ignored.
For a small executable output you can do two things:
Always use virtualenv to buildup your app. This would remove unnecessary packages that you already installed on your main Python library. So they will be ignored on the current build.
According to this using a compression decrease your output executable significantly.
UPX is a free utility available for most operating systems. UPX
compresses executable files and libraries, making them smaller,
sometimes much smaller. UPX is available for most operating systems
and can compress a large number of executable file formats. See the
UPX home page for downloads, and for the list of supported executable
formats.
So bring up a virtualenv install your external dependencies and then install UPX from here and use --upx-dir to pass UPX dir.
Related
My company has a large Git repository that is actively being developed (call it really-big-repo). I would like to use the reticulate package to call some of those python functions in R. Apologies in advance as I still try to get my arms around both python and git.
Single python files can be called using reticulate::source_python("flights.py") (see here). However, the python script I would like to imports modules that are from other parts of the repository. For example, the file that I would like to source great_python_functions.py looks like this:
import datetime
import json
import re
import requests
from bs4 import BeautifulSoup
from SomeRepoDirectory import utils
from SomeRepoDirectory.entity.models import Entity, EntityAlias, EntityBase, Subsidiary
import SomeRepoDirectory.entity.wikipedia
from SomeRepoDirectory.io.es import es_h
...
To further complicate it, the repo is pretty large and this file is just a small part of it. I'm not sure it is wise to load ALL of the repo's functions into my R environment.
And one more bonus question. I really would like to access the functions that are on a branch of the repo, not master. I've cloned the repository to a different directory than my R project using git clone git#github.com:my-company-name/really-big-repo.git
Here are the following steps you can try, but gotta say this is going to be complicated, might I say learning python might be easier :p
Like you said you have cloned the repository:
cd ./cloned_repo
conda activate your_vitual_env
git checkout feature/branch
python setup.py develop # this will install the pkg in virtual env, you can also use install instead of develop
Now in your R, use the virtual env in which you installed the repo, in my example I am using conda env, so you can use: reticulate::use_condaenv('your_virtual_env') and then you should be able to use those functions.
Also, in my experience intermingling python and R has caused a lot of pain for production development especially with package management. So I will advise some caution.
I'm trying to generate working Python modules from the containerd API .proto files as to be found here: https://github.com/containerd/containerd/tree/master/api.
Unfortunately, containerd's own .proto files contain references such as (in api/events/container.proto):
import weak "github.com/containerd/containerd/protobuf/plugin/fieldpath.proto";
Now, this import is actually located in protobuf/plugin/fieldpath.proto, as opposed to (vendor/)github.com/containerd/.... A simple -I ... does not work in this context, as the import uses a "github"-absolute path, whereas the corresponding sources aren't located inside the vendor branch.
Simply copying over the sources inside vendor/github.com/... will cause runtime errors when trying to use the generated Python modules: this is because there are now two separate instances for the same protocol elements trying to register with GRPC with the same protocol element name, yet from two different Python modules. The GRPC Python runtime thus throws an error and terminates.
How can I correctly get this resolved when using python3 -m grpc.tools.protoc ...?
After some trial+error I've finally come up with a working solution that works as follows and which might be helpful for others facing gRPC-based APIs with are more complex than many gRPC examples.
copy over the API .proto files into a directory structure that reflects the final desired Python package and module structure. For containerd, this means having everything within a containerd/ directory structure, avoiding github.com/ folders and aliasing (import aliasing will break things).
fix all import statement paths that would either cause module aliasing or won't fit with the final desired package structure. A good way to do this is with sed while copying over the proto files. In my case, replace "github.com/containerd/containerd/..." with just "containerd/..." import paths.
in case of vendor'ed .proto files somehow related to gRPC infrastructure and where gRPC PyPI packages exist, such as grpcio and protobuf, put then side-by-side with your API .proto files, but do not vendor them into the API directory hierarchy. Ensure to put them in a directory structure that mimics the package structure of the already available PyPI packages.
use protoc via the python3 interpreter to generate the Python modules only for your API .proto files; make sure that the supplemental .proto files from grpc and protobuf are includable, but do not create modules for them. protoc does this already correctly unless you vendor the supplemental .proto files into your API .proto files ... so don't do this.
make sure that your grpcio and protobuf PyPI packages are recent and somehow in sync, avoid especially totally outdated Debian distro packages, install from PyPI ... even if this is a painfully slow process on arm64 because there are no binary wheels for grpcio and grpciotools. Symptoms when doing not so include runtime errors of missing grpc or protobuf object fields.
When I install things with cabal-install, it automatically creates a ~/.ghc/x86_64-freebsd-8.8.3/environments/default with a bunch of stuff in it; this allows me to import my installed packages, but takes away my ability to import the standard library, even the Prelude:
gtk.hs:1:1: error:
Could not load module ‘Prelude’
It is a member of the hidden package ‘base-4.13.0.0’.
You can run ‘:set -package base’ to expose it.
(Note: this unloads all the modules in the current scope.)
Use -v (or `:set -v` in ghci) to see a list of the files searched for.
|
1 |
| ^
(It used to give me this error pointing to the first character of line 1 when line 1 was an unrelated import, so I put a blank line at the start of the file to see if it would still point the error there if there was nothing there. It does.)
If I delete the ~/.ghc/x86_64-freebsd-8.8.3/environments/default, I can import base again but not the installed packages. I need to give it the Cabal database with -package-db on the command line.
While that apparently works, I think there must be a way to configure that environment file to do this automatically. But as long as ~/.ghc/x86_64-freebsd-8.8.3/environments/default even exists, the entire standard library, including Prelude, is rendered hidden, and I can't compile anything at all (even an empty file fails with the above error). Including global-package-db in the environment file also didn't affect it.
Other info: I am not using Cabal to build because a .cabal file and a Setup.hs are added complexity I shouldn't need to compile a one-file throwaway program. I also tried installing the packages with --global passed to cabal install, but it still didn't install them globally.
OS: FreeBSD 12.1
GHC: 8.8.3
cabal-intsall: 3.0.2.0
I have some PyQt5 application with some tr() translations in it. I have a Python script that compiles UI forms and resources, and I want it to also update the ts file with translations, because it's a bit annoying to run pylupdate manually each time.
Alas, I haven't find any information, how to run the Python functions that provide translation file compilation in PyQt5. Haven't you ever got the same problem?
I was able to do it like this
import os
ret = os.system("E:\Anaconda3\Library\\bin\\pylupdate5.exe -verbose Main.py otherfile.py thirdfile.py -ts zh_CN.ts")
Hopefully you found it as well
I am attempting to convert a python file to an executable file. Easy enough right?
I used pyinstaller on a simple program that doesn't import anything. It worked like a charm. Then, I tried it with another dummy program with imported modules, (PyQt4, sys, matplotlib) that my actual program would have. Here I encountered problems.
This error appeared when I ran the application in the 'dist' folder pyinstaller created.
Fatal Python error: Py_Initialize: unable to load the file system codec
ImportError: No module named 'encodings'
I found another site with a possible solution to this problem, but his situation wasn't exactly the same: http://code.activestate.com/lists/python-dev/118463/
This lead me to trying the QT designer that I downloaded earlier. Perhaps if I could convert the .ui file it produced into a .py file, I would be fine. I could use his solution and all would be well.
That's when I got this error:
File "C:\Anaconda3\Lib\site-packages\PyQt4\uic\pyuic.py", line 26, in module
from PyQt4 import QtCore
mportError: No module named 'PyQt4'
I should also probably mention that all the modules I have are through Anaconda 3
I thought installing pyqt in a conda... project? Would fix the problem. It didn't. To be honest I don't entirely know what those are for.
Now I'm entertaining the idea of just using the c++ files that QT designer makes instead of converting them and importing python to tell the gui what to do.
What do you guys think would solve the errors above?
Short solution / workaround:
In your python file import the missing module explicitly. In your case: import encodings.
Proper solution:
By importing every module separately so you might end up imorting many modules and submodules. In this case you need tell pyinstaller where to find the modules (e. g. using compile flags).