Problem running Scrapoxy and Digital Ocean - web-scraping

I'm trying to run Scrapoxy with Digital Ocean. I successfully created a droplet image and configured Scrapoxy.
When I start Scrapoxy, it keeps on creating a new instance and bypassing max limit. It stops only when it reaches 10 droplets. What annoys me is that No instance found in the GUI version. Also, when I test the proxy server I get this message: "Error: No running instance found". It seems that Scrapoxy only creates droplets and can't connect to them.
I installed Scrapoxy manually. Here is my config file:
{
"commander": {
"password": ".........."
},
"instance": {
"port": 3128,
"scaling": {
"min": 1,
"max": 2
}
},
"providers": [
{
"type": "digitalocean",
"token": "5204b9654f301.............c281036bd19e283321c09680ac9c",
"region": "FRA1",
"size": "s-1vcpu-1gb",
"sshKeyName": "scrapoxy",
"imageName": "forward-proxy",
"tags": "Proxy,Amazon"
}
]
}

Did you try to put your region in lowercase in the config file
Example:
like this
"region": "fra1"
instead of
"region": "FRA1"
There are other troubleshooting steps you could take on the following github pages Issue 84 & Issue 62

Related

Debug Next.js App with VSCode in NX monorepo

I'm currently trying to debug a Next.js Application inside a NX monorepo.
I have enabled the Auto Attach setting in VSCode's User Settings.
When I start the Application using the serve command, I can see output in the Debug Console and also print out the current process by typing process or console.log(process) into the Debug Console.
However, I cannot set any breakpoints in the server side code, for example in getServerSideProps.
I checked Next.js Debugging Documentation for the missing pieces, and tried setting the NODE_OPTIONS='--inspect' in my Next.js Application via .env file.
Update: Seems like it's a missing feature on NX.
Got it working, thanks to the information from this Pull Request.
.vscode/launch.json
{
"version": "0.2.0",
"resolveSourceMapLocations": ["${workspaceFolder}/**", "!**/node_modules/**"],
"configurations": [
{
"name": "name-of-the-app – Server",
"type": "node",
"request": "launch",
"runtimeExecutable": "yarn",
"runtimeArgs": [
"nx",
"run",
"name-of-the-app:serve",
"-r",
"ts-node/register",
"-r",
"tsconfig-paths/register"
],
"outputCapture": "std",
"internalConsoleOptions": "openOnSessionStart",
"console": "internalConsole",
"env": {
"TS_NODE_IGNORE": "false",
"TS_NODE_PROJECT": "${workspaceFolder}/apps/name-of-the-app/tsconfig.json"
},
"cwd": "${workspaceFolder}/apps/name-of-the-app/"
}
]
}
Note: I'm using yarn. You might have to replace it with npm instead.

Invalid mint owner during set_collection Candy Machine v2

i was trying to mint some nfts with candy machine but when i try to execute:
ts-node ~/metaplex/js/packages/cli/src/candy-machine-v2-cli.ts set_collection \
-e devnet \
-k ~/.config/solana/devnet.json \
-c example \
-m C2eGm8iQPnKVWxakyo8QhwJUvYrZHKF52DPQuAejpTWG
I got this error:
throw new Error(`Invalid mint owner: ${JSON.stringify(info.owner)}`);
^
Error: Invalid mint owner: "11111111111111111111111111111111"
at Token.getMintInfo (/Users/btk-macmini-01/Desktop/repo/peppermint/docs/metaplex/js/node_modules/#solana/spl-token/client/token.js:731:13)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at async parseCollectionMintPubkey (/Users/btk-macmini-01/Desktop/repo/peppermint/docs/metaplex/js/packages/cli/src/helpers/various.ts:438:5)
at async Command.<anonymous> (/Users/btk-macmini-01/Desktop/repo/peppermint/docs/metaplex/js/packages/cli/src/candy-machine-v2-cli.ts:941:34)
Someone knows why? I have tried putting a different address from the one with that i have created the candymachine and with I made the upload or also the same but the issue is the same, maybe there is something wrong with it or with other things?
This is an example of my json:
{
"name": "#1",
"description": "description",
"external_url": "",
"image": "0.png",
"attributes": [
{
"trait_type": "Background Color Woman",
"value": "Light Blue"
},
{
"trait_type": "Background color man",
"value": "Metal Grey"
}
],
"properties": {
"files": [
{
"uri": "0.png",
"type": "image/png"
}
],
"creators": [
{
"address": "GM1ByqbTfgRwXEQCLJ2N4bsA3P1WcuyL9kZT79gLqYuE",
"share": 100
}
]
},
"compiler": "https://the-nft-generator.com",
"symbol": "Test",
"collection": {
"name": "test",
"family": "test"
}
}
If i upload without execute set_collections it works but with a different name of the collection from the one specified in the jsons file
set_collection is used to set the collection field to all the nfts inside a Candy machine that has not started the mint (0 minted NFTs). To set a collection you can pass any NFT (that is a masterEditionV2) that has the same updateAuthority as the wallet that you used to create ur CandyMachine.
In this case you are trying to set a collection that uses this NFT (-m C2eGm8iQPnKVWxakyo8QhwJUvYrZHKF52DPQuAejpTWG), and you said that ur CM was created with the wallet that has pubkey GM1ByqbTfgRwXEQCLJ2N4bsA3P1WcuyL9kZT79gLqYuE. The NFT has updateAuthority 42NevAWA6A8m9prDvZRUYReQmhNC3NtSZQNFUppPJDRB and thats a completly different pubkey that the one that you used to create the Candy Machine.
You can always use the collection webpage. This webpage allows you to create and mint a collection NFT with certain metadata, and also will migrate (change the onchain collections to the new created collection) the NFTs that you pass on the website and this can be updated at anytime with more NFTs. This website WILL NOT migrate unminted nfts from a candy machine.
If you want to use set_collection make sure to provide, on the -m parameter, an NFT that has the same updateAuthority that ur Candy Machine. Also make sure that your Candy Machine has 0 minted NFTs.

Jupyter Notebook validation failed

When I open a file in jupyter notebook, I get this error.
Notebook validation failed: {'model_id': '47c3f67b01814c2baeeca6efa0a79e2d', 'version_major': 2, 'version_minor': 0} is not valid under any of the given schemas:
{
"model_id": "47c3f67b01814c2baeeca6efa0a79e2d",
"version_major": 2,
"version_minor": 0
}
I had the same problem and this solution works like a charm:
https://github.com/jupyter/nbformat/issues/161
TLDR: open your file on a text editor and replace "nbformat_minor": 1 with "nbformat_minor": 4
Check if you activated conda environment correctly. In my case conda environment was not correctly activated so that was the problem. As I activated correctly everything worked normal.
I think different issues cause the error, one of them is:
The bug happens because the code blocks from Colab are missing "outputs':[] and "execution_count":null.
{
"cell_type": "code",
"metadata": {
"id": "e2769b9e0005"
},
"source": [
"!pip install git+https://github.com/google/starthinker\n"
]
},
To fix add the missing fields to the colab source for any code block only not markdown:
{
"cell_type": "code",
"execution_count": null,
"outputs": [],
"metadata": {
"id": "e2769b9e0005"
},
"source": [
"!pip install git+https://github.com/google/starthinker\n"
]
},

Python 3.6.5: Requests with streaming getting stuck in iter_content even if chunk_length is specified

I have been trying to use requests v2.19.1 in python 3.6.5 to download a ~2GB file from a remote URL. However, I have been repeatedly facing this issue where the code seems to get stuck forever in the for loop in trying to download the data.
My code snippet:
with requests.get(self.model_url, stream=True, headers=headers) as response:
if response.status_code not in [200, 201]:
raise Exception(
'Error downloading model({}). Got response code {} with content {}'.format(
self.model_id,
response.status_code,
response.content
)
)
with open(self.download_path, 'wb') as f:
for chunk in response.iter_content(chunk_size=1024):
if chunk:
f.write(chunk)
Each time I try to run this code, the download seems to stop at different points, and rarely reaches completion.
I have tried playing around with different chunk sizes, but I still keep seeing this issue.
Some additional details:
python -m requests.help
{
"chardet": {
"version": "3.0.4"
},
"cryptography": {
"version": "2.3.1"
},
"idna": {
"version": "2.7"
},
"implementation": {
"name": "CPython",
"version": "3.6.5"
},
"platform": {
"release": "3.10.0-693.11.1.el7.x86_64",
"system": "Linux"
},
"pyOpenSSL": {
"openssl_version": "1010009f",
"version": "18.0.0"
},
"requests": {
"version": "2.19.1"
},
"system_ssl": {
"version": "100020bf"
},
"urllib3": {
"version": "1.23"
},
"using_pyopenssl": true
}
Has anyone else faced a similar issue? If so, how did you resolve it?
It seems like if there is any interruption to the network during the download, the stream hangs up, and the connection goes dead. However, because no timeout is specified, the code seems to expect more packets to arrive over the dead connection. The best way I have found to handle this is to set a reasonable timeout. Once the timeout is reached after the last received package, the code exits the for loop with an exception which can be handled.

events.js:72 meteor up deploy

i tried everything what i found in net but nothing helped me... i try deploy app in my server on Debian 8.2, and every time after: mup deploy i got this:
Meteor Up: Production Quality Meteor Deployments
------------------------------------------------
Building Started: /Volumes/Macintosh HD/Users/myName/Google Drive/_projects/Coda/lottato_com
events.js:72
throw er; // Unhandled 'error' event
^
Error: spawn ENOENT
at errnoException (child_process.js:1011:11)
at Process.ChildProcess._handle.onexit (child_process.js:802:34)
my mup.json look lik:
{
"servers": [
{
"host": "server IP",
"username": "root",
"password": "blablabla"
}
],
"setupMongo": false,
"setupNode": true,
"nodeVersion": "0.10.36",
"enableUploadProgressBar": true,
"appName": "myAppName",
"app": "/Volumes/Macintosh HD/Users/myName/Google Drive/_projects/Coda/myAppName",
"env": {
"MONGO_URL": "//<login>:<password>#ds061464.mongolab.com:61111/myAppdb",
"ROOT_URL": "http://myApp.com"
},
"deployCheckWaitTime": 15
}
i can't handle with this issue almost 3 day! i tried deploy form server, change path, but it still didn't work and don't yet work...
and when i try to look in log, i got this:
Meteor Up: Production Quality Meteor Deployments
------------------------------------------------
[178.63.41.196] tail: cannot open ‘/var/log/upstart/lottato.log’ for reading: No such file or directory
tail: no files remaining
also u tried use mupx insted of mup, n now i got:
Invalid configuration file mup.json: There is no meteor app in the current app path.
new mup.json look like:
{
"servers": [
{
"host": "server IP",
"username": "root",
"password": "blablabla",
"env": {}
}
],
"setupMongo": false,
"appName": "appName",
"app": "~/Google Drive/_projects/Coda/appName",
"env": {
"PORT": 80,
"ROOT_URL": "http://appName.com",
"MONGO_URL": "mongodb://login:pass#ds035735.mongolab.com:35735/appName"
},
"deployCheckWaitTime": 15,
"enableUploadProgressBar": true
}
but i tried any type of path, with ~ or full path, n always the same, installation is starting only when in field path i writ:
"app": ".",
after increasing to 0.10.40 you should run 'mup setup' again followed by 'mup deploy'.
In my project I have mup.json in the project root (same level as .meteor) and
"app": "/Volumes/Macintosh HD/Users/myName/Google Drive/_projects/Coda/myAppName"
looks like
"app": ".",
Not sure if that is important.
i resolve this problem only with mupx + i moved project on server and deploy it from server to the same server.

Resources