I am trying to upload a binary file I have to Openstack Swift. This size of this file is 129MB. I am using the Python client for uploading:
swift_client = swiftclient.Connection(preauthurl=swift_public_auth_url,preauthtoken=swift_access_token,auth_version="2.0",insecure=True,tenant_name="admin")
with open("/path/to/file/file.ckpt", 'r') as read_file:
swift_client.put_object(swift_container_name, 'file.ckpt', contents= read_file.read(),content_type='text/plain')
print "done"
This throws the following error:
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "/usr/local/lib/python2.7/dist-packages/swiftclient/client.py", line 1824, in put_object
response_dict=response_dict)
File "/usr/local/lib/python2.7/dist-packages/swiftclient/client.py", line 1673, in _retry
service_token=self.service_token, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/swiftclient/client.py", line 1322, in put_object
raise ClientException.from_response(resp, 'Object PUT failed', body)
swiftclient.exceptions.ClientException: Object PUT failed: https://abc.xyz/swift/v1/AUTH_123456789abcdefghijklmn/container/file.ckpt 413 Request Entity Too Large [first 60 chars of response] <html>
<head><title>413 Request Entity Too Large</title></h
However, when I tried replacing test.ckpt with a blank text file test.txt, it got uploaded with the same parameters. I know this error occurs when the file size is greater than 5GB but what can be the problem here?
Found out that the problem was because of configuration on my Nginx proxy server. Got it fixed by increasing client_max_body_size. More details here: https://www.cyberciti.biz/faq/linux-unix-bsd-nginx-413-request-entity-too-large/
Related
So I was trying to get Biograkn usecase for BLAST working , following the steps as per this video series. I was able to load the schema , but after that we need to execute python migrate.py to load data into it. I executed the command and i am getting the following error traceback
Traceback (most recent call last):
File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/grakn/service/Session/TransactionService.py", line 161, in send
response = next(self._response_iterator)
File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/grpc/_channel.py", line 364, in __next__
return self._next()
File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/grpc/_channel.py", line 358, in _next
raise self
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
status = StatusCode.INVALID_ARGUMENT
details = "GraqlSemanticException-label 'database' not found. Please check server logs for the stack trace."
debug_error_string = "{"created":"#1582269484.666990683","description":"Error received from peer","file":"src/core/lib/surface/call.cc","file_line":1017,"grpc_message":"GraqlSemanticException-label 'database' not found. Please check server logs for the stack trace.","grpc_status":3}"
>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "migrate.py", line 92, in <module>
init(data_path="uniprot-asthma-proteins.fasta")
File "migrate.py", line 23, in init
session, q_get_database, q_insert_database, "$db"
File "/home/aditya/Projects/RD/biograkn/blast/util.py", line 14, in insert_if_non_existent
found_list = list(read_transaction.query(get_query))
File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/grakn/client.py", line 131, in query
return self._tx_service.query(query, infer)
File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/grakn/service/Session/TransactionService.py", line 49, in query
response = self._communicator.send(request)
File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/grakn/service/Session/TransactionService.py", line 165, in send
raise GraknError("Server/network error: {0}\n\n generated from request: {1}".format(e, request))
grakn.exception.GraknError.GraknError: Server/network error: <_Rendezvous of RPC that terminated with:
status = StatusCode.INVALID_ARGUMENT
details = "GraqlSemanticException-label 'database' not found. Please check server logs for the stack trace."
debug_error_string = "{"created":"#1582269484.666990683","description":"Error received from peer","file":"src/core/lib/surface/call.cc","file_line":1017,"grpc_message":"GraqlSemanticException-label 'database' not found. Please check server logs for the stack trace.","grpc_status":3}"
>
generated from request: query_req {
query: "match $db isa database, has name \"uniprot\"; get $db;"
}
now this is not understandable why loading is not possible. Hence requesting for assistance.
There appears to already be a problem with the schema, since graql is trying to look for the label "database" but cannot find it.
You can check your schema using workbase:
1. connect to your local server (localhost:48555)
2. select your keyspace ("proteins")
3. click on the hierarchy symbol on the top left next to the grakn logo
If the schema is present, it will be visualized automatically.
I am trying to upload a file into a SFTP using a key file. I already configured the connection and I can authenticate without any problem:
{'key_file': '/my_folder/public_key'}
Also I am able to do all the process manually using Cyberduck for example. This is the function that I am calling:
from contextlib import closing
from airflow.contrib.hooks.ssh_hook import SSHHook
# Get connection details
ssh = SSHHook(ssh_conn_id='my conn id')
# Upload the file into sftp
with closing(ssh.get_conn().open_sftp()) as sftp_client:
sftp_client.put('/local_folder/my_file.xlsx', '/sftp_folder/my_file.xlsx')
This is the error I am receiving:
{base_hook.py:80} INFO - Using connection to: xxxxxxx
{transport.py:1687} INFO - Connected (version 2.0, client AWS_SFTP_1.0)
{transport.py:1687} INFO - Authentication (publickey) successful!
PermissionError: [Errno 13] Forbidden
Does anyone have any idea of why this is happening if I am able to do the same manually?
Thank you so much!
The whole stack:
{transport.py:1687} INFO - Authentication (publickey) successful!
{sftp.py:131} INFO - [chan 0] Opened sftp connection (server version 3)
Traceback (most recent call last):
File "/.../airflow/plugins/operators/my_operator.py", line 231, in sftp_upload
client.put(local_path, sftp_path)
File "/.../venv/lib/python3.6/site-packages/paramiko/sftp_client.py", line 727, in put
return self.putfo(fl, remotepath, file_size, callback, confirm)
File "/.../venv/lib/python3.6/site-packages/paramiko/sftp_client.py", line 683, in putfo
with self.file(remotepath, 'wb') as fr:
File "/.../venv/lib/python3.6/site-packages/paramiko/sftp_client.py", line 341, in open
t, msg = self._request(CMD_OPEN, filename, imode, attrblock)
File "/.../venv/lib/python3.6/site-packages/paramiko/sftp_client.py", line 780, in _request
return self._read_response(num)
File "/.../venv/lib/python3.6/site-packages/paramiko/sftp_client.py", line 832, in _read_response
self._convert_status(msg)
File "/.../venv/lib/python3.6/site-packages/paramiko/sftp_client.py", line 863, in _convert_status
raise IOError(errno.EACCES, text)
PermissionError: [Errno 13] Forbidden
The problem I was facing was due to invalid path in sftp folder. Cyberduck was hiding part of the path so I was including an incomplete one into my code. Paramiko was returning Forbidden because probably the path exists but this account doesn't have access to it.
Once I included the full path the code above worked pretty fine!
Thanks!
I have been running R from ST3, but all of a sudden it has stopped working and I cannot figure out why. When I open a R-file in ST3, R does not open after hitting [Cmd + enter]. I have already tried the following things:
1) Check if the path to Rgui is correct in the User settings (it is).
2) Check if the file is in R-extended format (it is).
I have reinstalled ST3, but the problem is still there. I am running ST3 on Windows 7 (64-bit).
In the command-line I get the following error:
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Program Files\Sublime Text 3\sublime_plugin.py", line 818, in run_
return self.run(edit)
File "SendText in C:\Users\Tim\AppData\Roaming\Sublime Text 3\Installed Packages\SendText.sublime-package", line 89, in run
File "SendText in C:\Users\Tim\AppData\Roaming\Sublime Text 3\Installed Packages\SendText.sublime-package", line 27, in send
File "./python3.3/subprocess.py", line 523, in call
File "./python3.3/subprocess.py", line 819, in __init__
File "./python3.3/subprocess.py", line 1110, in _execute_child
FileNotFoundError: [WinError 2] The system cannot find the file specified
Traceback (most recent call last):
File "./python3.3/subprocess.py", line 1104, in _execute_child
FileNotFoundError: [WinError 2] The system cannot find the file specified
Does anyone have any ideas on how to solve this issue?
I am getting "Error handling request" and I can't really trace this down to a particular problem?
I am using gunicorn+nginx and my gunicorn setup is
gunicorn run:app -w 4 -b 0.0.0.0:8080 --workers=1 --timeout=300
here is the error message
2015-10-14 21:27:11,287 DEBG 'myserver' stderr output:
[2015-10-14 21:27:11 +0000] [26725] [ERROR] Error handling request
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/sync.py", line 130, in handle
self.handle_request(listener, req, client, addr)
File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/sync.py", line 171, in handle_request
respiter = self.wsgi(environ, resp.start_response)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1836, in __call__
return self.wsgi_app(environ, start_response)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1820, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1403, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1817, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1478, in full_dispatch_request
response = self.make_response(rv)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1566, in make_response
raise ValueError('View function did not return a response')
ValueError: View function did not return a response
can anybody give me a hint how to debug this? I don't have much experience with server usage...
thanks
carl
tl;dr: This isn't an issue with gunicorn or nginx. Your view function in your Flask app is not returning a response. Check the return statement in your view function for whatever route you are accessing when you get this error.
Beginning at the Line
Traceback (most recent call last):
You can see the stack trace produced by the python interpreter. The stack trace shows the sequence of nested functions leading up to the point where your code failed. In my limited experience, the python interpreter stack trace has reliably directed me to the fault in my code.
In your case here, the final line:
ValueError: View function did not return a response
provides some more detail about the error and should give you pretty good idea of what went wrong (Your view function didn't return a response).
The next line up from the bottom will show you the function that triggered the error, and its exact location in your code:
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1566, in make_response
raise ValueError('View function did not return a response')
In this case the error is raised by a function in the flask source, so unless you edited that, that is probably not where you need to make a fix. Based on the specific ValueError at the end of the trace, I'd go straight to my views function. In Flask that probably looks something like this (example from Flask tutorial):
#app.route('/')
def show_entries():
cur = g.db.execute('select title, text from entries order by id desc')
entries = [dict(title=row[0], text=row[1]) for row in cur.fetchall()]
return render_template('show_entries.html', entries=entries)
That last line seems like a good place to start in your case, since the error says nothing is being returned.
I want to track the HTTP response codes returned by my nginx web-server, using logster.
1) I found and installed logster. I also pip-installed pygtail, which is required for logster.
https://github.com/etsy/logster
2) I found a python script that parses nginx access_log and placed it in the parsers subdir.
https://github.com/metabrainz/logster/blob/master/musicbrainz/logster/NginxStatus.py
...but when I run the logster command, I get a python exception:
Traceback (most recent call last):
File "/usr/local/bin/logster", line 5, in <module>
pkg_resources.run_script('logster==0.0.1', 'logster')
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 505, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 1245, in run_script
execfile(script_filename, namespace, namespace)
File "/usr/local/lib/python2.7/dist-packages/logster-0.0.1-py2.7.egg/EGG-INFO/scripts/logster", line 449, in <module>
main()
File "/usr/local/lib/python2.7/dist-packages/logster-0.0.1-py2.7.egg/EGG-INFO/scripts/logster", line 380, in main
module = __import__(module_name, globals(), locals(), [parser_name])
ImportError: No module named NginxStatus1
````
What am I doing wrong?
The exception error was rather misleading: the file was placed in the right place (the parsers subdir), but - as it turns out- logster must be re-setup after a new parser is added (this isn't documented, unfortunately). so just run:
sudo python setup.py install
in the logster directory and things should start working correctly.