I am using GoogleScraper to scrape some google results and store them in a json format
I ran this code in terminal:-
$ GoogleScraper -m selenium --sel-browser chrome --browser-mode headless --keyword "corolla" --output-filename ./corolla.json -v debug
I was expecting a .json file as output in the GoogleScrapper Folder.
But instead i do not see any file except google_scraper.db and googlescraper.log files only
I have just started coding in python scrapping.
it is saved under Untitled ▸ Users ▸ Whatever is your username ▸.json by default.
Related
I have been testing with PGP command line to create a batch file that encrypts all zip files in a folder
So far I have tried
GPG -e -r username c:\foldername*.zip
But when I run the bat file nothing happens.
Do I need the path to where PGP is installed adding?
I would also like to delete the zip once it’s encrypted and on a previous batch file used -SDEL, will this work here
Thanks
gpg doesn't seem to work file wildcards. You should use for cycle, see this answer for the examples: How to do something to each file in a directory with a batch script
I am trying to download a pdf file using wget.
When I do:
wget <url> it downloads a corrupted file however if I run wget -i test.txt with the pdf URL inside this test txt file it works and the file is not corrupted.
Does anyone know why?
From the logs I can see the following.
In the first case, it is downloading a note found page.
Length: 11322 (11K) [text/html] Saving to: ‘media.nl?id=39194.1’
In the second it is a proper pdf.
Length: 58272 (57K) [application/pdf] Saving to:
‘media.nl?id=39194&c=4667446&h=34c63dbaaa7adc7c8a33&_xt=.pdf’
Thanks,
Put your URL into quotes. Not quoting the URL can lead to strange effects, in your case the & is interpreted by the shell.
E.g.
wget "https://www.roofingsuppliesuk.co.uk/core/media/media.nl?id=39194&c=4667446&h=34c63dbaaa7adc7c8a33&_xt=.pdf"
or
wget 'https://www.roofingsuppliesuk.co.uk/core/media/media.nl?id=39194&c=4667446&h=34c63dbaaa7adc7c8a33&_xt=.pdf'
or with escaping of &
wget https://www.roofingsuppliesuk.co.uk/core/media/media.nl?id=39194\&c=4667446\&h=34c63dbaaa7adc7c8a33\&_xt=.pdf
I got the same issue but I changed the command to this and then it worked fine when i tested it:
Wget —-no-check-certificate https://www.roofingsuppliesuk.co.uk/core/media/'media.nl?id=39194&c=4667446&h=34c63dbaaa7adc7c8a33&_xt=.pdf'
i just added single quotes beginning at 'media.nl.......pdf'
Make sure the file with same name doesnt exist. You dont need to add --no-check-certificate if you dont get self-signed certificate error
I was trying to deploy my application on Ubuntu 16.04. So i made a package with the following hierarchy -
Package
|
----bin
|
-----application
-----application.sh
-----Qt
|
-----necessary qt libraries
-----platforms
Here is the application.sh file -
#!/bin/sh
export LD_LIBRARY_PATH=`pwd`/Qt
./application
When i execute the application.sh file, it shows me that it cant find the libQt5MultimediaWidgets.so.5 file. But its in the Qt folder. Also when i print the ldd application from the application.sh file after exporting LD_LIBRARY_PATH it gives me following output -
Please check the marked parts. Can anyone please explain why the libraries from the Qt folder are not found even after exporting the LD_LIBARRY_PATH?
Edit:
So as suggested by #Zang, i have checked the debug log and here it is -
Please check the marked parts.
It seems like its actually trying the actual libQt5MultimediaWidgets.so and then report that its unable to find it. Can anyone please help me understand whats happening here?
Edit-2: As per suggestion from #Tarun, i have ran ls -al on my Qt folder. Here is the output -
All files in Your Qt directory are actually simlinks to non-existing files in the same directory, therefore they cannot be found.
If you look at the output of your ls -al
These are soft links that you have. Your softlink libQt5MultimediaWidgets.so.5 points to libQt5MultimediaWidgets.so.5.9.2 in the same directory and the file is not there at all. So you need to either set the correct softlink path or have the file in same directory
First
Could it be that the pwd is not where you assume it is?
You could try adding
# Figure out where the application.sh script is located
scriptpath="$( cd "$(dirname "$0")" ; pwd -P )"
# Make sure our pwd is that location
cd "$scriptpath"
in the top of your script (assumes bash shell, from here)
By doing this all relative paths to Qt folder will be valid.
Second
Maybe you should considder exporting your new LD_LIBRARY_PATH, like so (from here):
LD_LIBRARY_PATH=whatever
export LD_LIBRARY_PATH
Third
It may be useful to run ldconfig command for ld to update after changing the variable (from here):
sudo ldconfig
The file libQt5MultimediaWidgets.so is not present in /Desktop/package/bin/Qt according to the screenshots shown.
I have a dropbox link like https://www.dropbox.com/sh/w4366ttcz6/AAB4kSz3adZ which opens the ususal dropbox site with folders and files.
Is there any chance to download the complete content (tar or directly as sync) to a unix machine using wget?
I have seen some posts here where single files were downloaded but could not find any answer to this. There is an api from Dropbox but that does not work on my server due to the 64 bit issue on my server and http://www.dropboxwiki.com/dropbox-addons/dropbox-gallery-download#BASH_Version does also not work for me.... any other suggestions?
This help article documents some parameters you can use to get different behaviors from Dropbox shared links:
https://www.dropbox.com/help/201
For example, using this link:
https://www.dropbox.com/sh/igoku2mqsjqsmx1/AAAeF57DR2ou_nZGC4JPoQKfa
We can use the dl parameter to get a direct download. Using curl, we can download it as such:
curl -L https://www.dropbox.com/sh/igoku2mqsjqsmx1/AAAeF57DR2ou_nZGC4JPoQKfa?dl=1 > download.zip
(The -L is necessary in order to follow redirects.)
Or, with wget, something like:
wget --max-redirect=20 -O download.zip https://www.dropbox.com/sh/igoku2mqsjqsmx1/AAAeF57DR2ou_nZGC4JPoQKfa
You can use --content-disposition with wget too.
wget https://www.dropbox.com/sh/igoku2mqsjqsmx1/AAAeF57DR2ou_nZGC4JPoQKfa --content-disposition
It will auto-detect the folder name as the zip filename.
Currently, you're probably better off creating an app that you don't publish, which can either access all your files, or just a dedicated app folder (safer). Click the generate API token button about halfway down the app's settings page, and store it securely! You can then use the dedicated download or zip download API calls to get your files from anywhere like so:
curl -X POST https://content.dropboxapi.com/2/files/download_zip \
--header "Authorization: Bearer $MY_DROPBOX_API_TOKEN" \
--header 'Dropbox-API-Arg: {"path": "/path/to/directory"}' \
> useful-name.zip
Adding your token as an environment variable makes it easier & safer to type/script these operations. If you're using BASH, and you have ignorespace in your $HISTCONTROL you can just type + paste your key with a leading space so it's not saved in your history. For frequent use, save it in a file with 0600 permissions that you can source, as you would an SSH key.
export MY_DROPBOX_API_TOKEN='...'
Yes you can as it is pretty wasy follow below steps
Firstly, get the dropbox share link. It will look like this https://www.dropbox.com/s/ad2arn440pu77si/test.txt
Then add a “?dl=1” to the end of that url and a “-O filename” so that you end up with something like this: wget https://www.dropbox.com/s/ad2arn440pu77si/test.txt?dl=1 -O test.txt
Now you can easily get files onto your linux.
I've to use external files because I need to create a page as a web page with StageWebView, and I need these files : 1 HTML file, 2 PNG files, 1 Javascript file.
There is a problem when I want to use these files, I can't access them because I think they are omitted when APK package is installed. However, I can see them into the APK file after I have created it (I can open it as a traditionnal archive).
Unfortunely, after the installation, there is not my files at the expected place : I look at my OS file system, using the file explorer entry into the Android Device Monitor of Android SDK, the tool located at folder AndroidSDK_folder\tools\monitor.bat.
I've writed this command line to package the APK, I don't think to do any errors here, it's very simple...
adt -package -target apk-emulator -storetype pkcs12 -keystore "my-Certificate.p12" -storepass password "my-APK.apk" "src\my-Descriptor-app.xml" "my-SWF.swf" -C src\assets -e src\assets\1.html -e src\assets\image1.png image1.png -e src\assets\image2.png image2.png -e src\assets\my-JS.js my-JS.js
My files are not "embeded" in the code, it seems to be the reason of losing files at the installation step : Is it true ?