Lets say you need to distribute a Scrapy 2.1.0 project to someone and they don't know python.
How do you package this project up into an executable that can be run by double clicking or via command line passing arguments?
project structure:
proj/
scrapy_proj_name/
scrapy_proj_name/
spiders/
my_spider.py
items.py
middelwares.py
pipelines.py
settings.py
venv/
.gitignore
requirements.txt
Now I run the crawler using this scrapy crawl my_spider -a param1=foo -o output.csv from python shell, but I need to do it as program.exe foo or program.exe foo output.csv from command line.
Has anyone ever managed to do this?
Related
jq command not found after adding jq executable
installing jq on git bash
My usecase is more similar with above shared references. I tried to execute a hook that needs to parse a json file. When hook gets executed it throws bash: jq:command not found error. So. I downloaded jq-win64.exe file and copied it to /usr/bin in Git folder. Then from git-bash I run export PATH=$PATH:"/C/Program Files/Git/usr/bin/jq-win64.exe" command and there is no error but when I checked jq --version command it still shows bash: jq:command not found error
Am I missing something? I even tried in windows cmd but is of no use. Hope someone can help me.
Thanks in advance!!!
PATH contains directories. That means what you should do:
Rename jq-win64.exe to jq.exe or just jq. (e.g. cp ~/Downloads/jq-win64.exe /usr/bin/jq).
You don't have to export your path, /usr/bin is already part of it.
If you didn't rename the file to jq (or jq.exe), then you would have to run it as jq-win64 in your console.
You could also put the binary into ~/bin folder, which should be part of PATH too. If it isn't, you can add it. Then you don't need to mess with your global binaries folder.
I have a python file that I am trying to create an executable out of using Pyinstaller on my Mac. This python file imports several different python files. When I run the unix executable that was generated, I get this error:
File "main/__init__.py", line 4, in <module>
ModuleNotFoundError: No module named 'game'
Line 4 reads:
from game.scripts.gui import creator
The command I used to create the executable:
pyinstaller __init__.py --onefile --clean --windowed
The directory:
__init__.py
game
scripts
gui
creator.py
Any ideas on how I could fix this? Thanks
The subdirs are not included by creating an *.exe, so the creator.py is not found inside your *.exe. To avoid that, you have to include the extra files/folders by specifying them. This can be done by a *.spec file
By calling pyinstaller with your *.py file it will create a default *.spec file which you can edit and use next time to create your *.exe. Every option you used when calling
pyinstaller __init__.py --onefile --clean --windowed
is configured here so calling
pyinstaller *.spec
the next time gives the same result.
Edit this in your spec-file to fit your needs by copying single files or even whole folders including their content into the *.exe:
a = Analysis(['your.py'],
pathex=['.'],
binaries=[],
datas=[('some.dll', '.'),
('configurationfile.ini', '.'),
('data.xlsx', '.'),
('../../anotherfile.pdf', '.')
],
....some lines cut ....
a.datas += Tree('./thisfoldershouldbecopied', prefix='foldernameinexe')
More infos to that are found in the docs of pyinstaller regarding spec-files and including data files
https://pyinstaller.readthedocs.io/en/stable/spec-files.html
and for example in this post here:
Pyinstaller adding data files
So, the title basically covers my question. I've created a project using virtualenv, e.g. I have to
source ./env/bin/activate
to run my script.
When I try creating an executable using:
pyinstaller --onefile <myscript.py>
None of the virtualenv packages are included; just the ones that are installed globally. I have a requirements.txt file that contains all of the modules I need. Is there a way to have pyinstaller point to that for the needed modules, or is there another way?
As Valentino pointed out by looking at How can I create the minimum size executable with pyinstaller?
You have to run PyIntaller from inside the virtual environment:
(venv_test) D:\testenv>pyinstaller
How to solve the not importing modules from the virtual environment
The virtual environment saves modules in a different directory than the global module directory. If you are using Windows like me, you can find the modules directory here:
C:\Users\<your username>\.virtualenvs\<your project name>\Lib\site-packages
When you find your virtualenv directory, run this command instead of this simple command(pyinstaller <script>.py):
pyinstaller --paths "C:\Users\<your username>\.virtualenvs\<your project name>\Lib\site-packages" --hidden-import <module name that should be import> <your script name>.py
To export just one file you can add this: -F or --onefile
As many modules as you can add to be imported by adding more --hidden-import flags and module name
Flag description
--paths: The pyinstaller will search for imports here
--hidden-import: Which modules should be imported by pyinstaller from the path
Running python3 xx.py in my specific folder
shows Import Error
But, I go to my home directory ,and add the new files with same code
and then it can work well. Why?
I checked both sys.path in these two, the path is same
find pycache, init.py
ls -l
remove it
rm -rf "pycache" "init.py" it works!
Finally,it is solved
I have a home directory in my unix box. I would like to setup a number or shortcuts in it to point to the latest file in another directory and the link will update if a newer file is created.
Is this possible?
So far I able to get the latest file:
ls -lrt | tail -n1
Thanks
[EDIT]
Perhaps I could even create a shell instead of a softlink which finds the latest file and returns it so I can open/grep/delete etc?
In bash, this will make a link to the latest file or directory in "target-directory" called "latest":
ln -s target-directory/`ls -rt target-directory | tail -n1` latest
And this will wait for a change in "target-directory" before returning:
inotifywait -e attrib target-directory