I need push message from backend to frontend in Django, finally I found this Twisted + Django server but in product env we use nginx to proxy request to Django, I'm confused can hendrix and nginx work together?
If yes, then what how to write nginx.conf?
Yep, you can use nginx to proxy pass to hendrix the same way as you do with any other WSGI/ASGI container.
This Question has sample config for doing this.
today i try to change my original nginx config to reverse proxy hendrix, here is some snippets:
upstream django {
server localhost:8000;
}
server {
listen 80;
server_name localhost;
location /static/ {
#root /var/www;
alias /var/www/static;
}
location / {
proxy_pass http://django;
proxy_pass_header Server;
proxy_set_header Host $host;
proxy_redirect off;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}
then i restart nginx and start hendrix: hx start --log ./test.log
it worked finally, but i found some warning message:
*
WebSocket connection to '...' failed: Error during WebSocket
handshake: 'Connection' header value is not 'Upgrade': keep-alive
socket.io.js:2371
*
after some google message, i found nginx version must higher 1.3.13, so i update nginx to 1.10.1(stable), warning message disappear.
but if want use hendrix in my production env, multi hendrix-worker needed, so i try: hx start -w 2, failed output:
root#proxy:/vagrant/hendrix_taste/hendrix/examples/django_hx_chatserver/example_app# hx start -w 2
Adding media resource for URL 'static' at path '/vagrant/hendrix_taste/hendrix/examples/django_hx_chatserver/example_app/servedstatics'
...
Adding media resource for URL 'static/admin' at path '/usr/local/lib/python2.7/dist-
'/vagrant/hendrix_taste/hendrix/examples/django_hx_chatserver/example_app/chat/static/chat/js'
Ready and Listening on port 8000...
{0: 0, 1: 1, 2: 2, 4: 4}
{'main_web_tcp': 4}
Unhandled Error
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/hendrix/deploy/base.py", line 180, in run
getattr(self, action)(fd)
File "/usr/local/lib/python2.7/dist-packages/hendrix/deploy/base.py", line 226, in start
self.launchWorkers(pids)
File "/usr/local/lib/python2.7/dist-packages/hendrix/deploy/base.py", line 265, in launchWorkers
DeployServerProtocol(args), 'hx', args, childFDs=self.childFDs, env=environ
File "/usr/local/lib/python2.7/dist-packages/twisted/internet/posixbase.py", line 340, in spawnProcess
processProtocol, uid, gid, childFDs)
--- <exception caught here> ---
File "/usr/local/lib/python2.7/dist-packages/twisted/internet/process.py", line 756, in __init__
self.proto.makeConnection(self)
exceptions.AttributeError: 'DeployServerProtocol' object has no attribute 'makeConnection'
Unhandled Error
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/hendrix/deploy/base.py", line 180, in run
getattr(self, action)(fd)
File "/usr/local/lib/python2.7/dist-packages/hendrix/deploy/base.py", line 226, in start
self.launchWorkers(pids)
File "/usr/local/lib/python2.7/dist-packages/hendrix/deploy/base.py", line 265, in launchWorkers
DeployServerProtocol(args), 'hx', args, childFDs=self.childFDs, env=environ
File "/usr/local/lib/python2.7/dist-packages/twisted/internet/posixbase.py", line 340, in spawnProcess
processProtocol, uid, gid, childFDs)
--- <exception caught here> ---
File "/usr/local/lib/python2.7/dist-packages/twisted/internet/process.py", line 756, in __init__
self.proto.makeConnection(self)
exceptions.AttributeError: 'DeployServerProtocol' object has no attribute 'makeConnection'
Adding media resource for URL 'static' at path '/vagrant/hendrix_taste/hendrix/examples/django_hx_chatserver/example_app/servedstatics'
...
Ready and Listening on port 8000...
Unhandled error in Deferred:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 317, in addCallback
callbackKeywords=kw)
File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 306, in addCallbacks
self._runCallbacks()
File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 588, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1313, in execute
d = maybeDeferred(f, *args, **kwargs)
--- <exception caught here> ---
File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 150, in maybeDeferred
result = f(*args, **kw)
exceptions.TypeError: _addSubprocess() takes exactly 4 arguments (5 given)
Temporarily disabling observer <twisted.logger._filter.FilteringLogObserver object at 0x20bac50> due to exception: [Failure instance: Traceback: <type 'exceptions.AttributeError'>: 'NoneType' object has no attribute '_formatter_parser'
/usr/local/lib/python2.7/dist-packages/hendrix/deploy/base.py:283:addSubprocess
/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py:707:__del__
/usr/local/lib/python2.7/dist-packages/twisted/logger/_logger.py:178:failure
/usr/local/lib/python2.7/dist-packages/twisted/logger/_logger.py:132:emit
--- <exception caught here> ---
/usr/local/lib/python2.7/dist-packages/twisted/logger/_observer.py:131:__call__
/usr/local/lib/python2.7/dist-packages/twisted/logger/_filter.py:131:__call__
/usr/local/lib/python2.7/dist-packages/twisted/logger/_file.py:50:__call__
/usr/local/lib/python2.7/dist-packages/twisted/logger/_json.py:245:<lambda>
/usr/local/lib/python2.7/dist-packages/twisted/logger/_json.py:196:eventAsJSON
/usr/local/lib/python2.7/dist-packages/twisted/logger/_flatten.py:87:flattenEvent
/usr/lib/python2.7/string.py:621:parse
]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/hendrix/deploy/base.py", line 283, in addSubprocess
self._lock.run(self._addSubprocess, self, fds, name, factory)
File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 707, in __del__
debugInfo=debugInfo)
File "/usr/local/lib/python2.7/dist-packages/twisted/logger/_logger.py", line 178, in failure
self.emit(level, format, log_failure=failure, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/twisted/logger/_logger.py", line 132, in emit
self.observer(event)
--- <exception caught here> ---
File "/usr/local/lib/python2.7/dist-packages/twisted/logger/_observer.py", line 131, in __call__
observer(event)
File "/usr/local/lib/python2.7/dist-packages/twisted/logger/_filter.py", line 131, in __call__
self._observer(event)
File "/usr/local/lib/python2.7/dist-packages/twisted/logger/_file.py", line 50, in __call__
text = self.formatEvent(event)
File "/usr/local/lib/python2.7/dist-packages/twisted/logger/_json.py", line 245, in <lambda>
lambda event: u"{0}{1}\n".format(recordSeparator, eventAsJSON(event))
File "/usr/local/lib/python2.7/dist-packages/twisted/logger/_json.py", line 196, in eventAsJSON
flattenEvent(event)
File "/usr/local/lib/python2.7/dist-packages/twisted/logger/_flatten.py", line 87, in flattenEvent
aFormatter.parse(event["log_format"])
File "/usr/lib/python2.7/string.py", line 621, in parse
return format_string._formatter_parser()
exceptions.At
my python packages:
pip freeze
Warning: cannot find svn location for distribute==0.6.24dev-r0
Automat==0.3.0
Django==1.8.11
GnuPGInterface==0.3.2
Jinja2==2.9.4
MarkupSafe==0.23
PyYAML==3.12
Twisted==16.0.0
apt-xapian-index==0.44
argh==0.26.2
argparse==1.2.1
attrs==16.3.0
characteristic==14.3.0
chardet==2.0.1
command-not-found==0.2.44
configobj==4.7.2
constantly==15.1.0
decorator==3.3.2
## FIXME: could not find svn URL in dependency_links for this package:
distribute==0.6.24dev-r0
hendrix==2.0.2
incremental==16.10.1
ipython==0.12.1
language-selector==0.1
pathtools==0.1.2
pexpect==2.3
pyOpenSSL==16.2.0
pyasn1==0.1.9
pyasn1-modules==0.0.8
pychalk==0.0.5
python-apt==0.8.3ubuntu7.1
python-debian==0.1.21ubuntu1
service-identity==16.0.0
simplegeneric==0.7
six==1.10.0
txsockjs==1.2.2
ufw==0.31.1-1
virtualenv==15.1.0
watchdog==0.8.3
wsgiref==0.1.2
zope.interface==4.0.5
Related
We deployed Apache Airflow 2.3.3 to Azure.
Webserver - Web App
Scheduler - ACI
Celery Worker - ACI
We were seeing errors on the Celery ACI console related to Postgres and Redis connection timeouts as below
[2022-09-22 18:55:50,650: WARNING/ForkPoolWorker-15] Failed operation _store_result. Retrying 2 more times.
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1803, in _execute_context
cursor, statement, parameters, context
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 719, in do_execute
cursor.execute(statement, parameters)
psycopg2.DatabaseError: could not receive data from server: Connection timed out
SSL SYSCALL error: Connection timed out
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.7/site-packages/celery/backends/database/__init__.py", line 47, in _inner
return fun(*args, **kwargs)
File "/home/airflow/.local/lib/python3.7/site-packages/celery/backends/database/__init__.py", line 117, in _store_result
task = list(session.query(self.task_cls).filter(self.task_cls.task_id == task_id))
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 2887, in __iter__
return self._iter().__iter__()
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 2897, in _iter
execution_options={"_sa_orm_load_options": self.load_options},
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 1689, in execute
result = conn._execute_20(statement, params or {}, execution_options)
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1614, in _execute_20
return meth(self, args_10style, kwargs_10style, execution_options)
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 326, in _execute_on_connection
self, multiparams, params, execution_options
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1491, in _execute_clauseelement
cache_hit=cache_hit,
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context
e, statement, parameters, cursor, context
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2027, in _handle_dbapi_exception
sqlalchemy_exception, with_traceback=exc_info[2], from_=e
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
raise exception
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1803, in _execute_context
cursor, statement, parameters, context
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 719, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.DatabaseError: (psycopg2.DatabaseError) could not receive data from server: Connection timed out
SSL SYSCALL error: Connection timed out
[SQL: SELECT celery_taskmeta.id AS celery_taskmeta_id, celery_taskmeta.task_id AS celery_taskmeta_task_id, celery_taskmeta.status AS celery_taskmeta_status, celery_taskmeta.result AS celery_taskmeta_result, celery_taskmeta.date_done AS celery_taskmeta_date_done, celery_taskmeta.traceback AS celery_taskmeta_traceback
FROM celery_taskmeta
WHERE celery_taskmeta.task_id = %(task_id_1)s]
[parameters: {'task_id_1': 'c5f9f53c-8afe-4d67-8d3b-d7ad84875de1'}]
(Background on this error at: https://sqlalche.me/e/14/4xp6)
[2022-09-22 18:55:50,929: INFO/ForkPoolWorker-15] [c5f9f53c-8afe-4d67-8d3b-d7ad84875de1] Executing command in Celery: ['airflow', 'tasks', 'run', 'CS_ALERTING', 'CheckRunningTasks', 'scheduled__2022-09-22T18:00:00+00:00', '--local', '--subdir', 'DAGS_FOLDER/CS_ALERTING.py']
[2022-09-22 18:55:51,241: INFO/ForkPoolWorker-15] Filling up the DagBag from /opt/airflow/platform_pam/dags/CS_ALERTING.py
[2022-09-22 18:55:53,467: INFO/ForkPoolWorker-15] Running <TaskInstance: CS_ALERTING.CheckRunningTasks scheduled__2022-09-22T18:00:00+00:00 [queued]> on host localhost
[2022-09-22 18:55:58,304: INFO/ForkPoolWorker-15] Task airflow.executors.celery_executor.execute_command[c5f9f53c-8afe-4d67-8d3b-d7ad84875de1] succeeded in 960.1964174450004s: None
[2022-09-22 19:29:25,931: WARNING/MainProcess] consumer: Connection to broker lost. Trying to re-establish the connection...
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.7/site-packages/redis/connection.py", line 706, in send_packed_command
sendall(self._sock, item)
File "/home/airflow/.local/lib/python3.7/site-packages/redis/_compat.py", line 9, in sendall
return sock.sendall(*args, **kwargs)
File "/usr/local/lib/python3.7/ssl.py", line 1034, in sendall
v = self.send(byte_view[count:])
File "/usr/local/lib/python3.7/ssl.py", line 1003, in send
return self._sslobj.write(data)
TimeoutError: [Errno 110] Connection timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.7/site-packages/celery/worker/consumer/consumer.py", line 332, in start
blueprint.start(self)
File "/home/airflow/.local/lib/python3.7/site-packages/celery/bootsteps.py", line 116, in start
step.start(parent)
File "/home/airflow/.local/lib/python3.7/site-packages/celery/worker/consumer/consumer.py", line 628, in start
c.loop(*c.loop_args())
File "/home/airflow/.local/lib/python3.7/site-packages/celery/worker/loops.py", line 97, in asynloop
next(loop)
File "/home/airflow/.local/lib/python3.7/site-packages/kombu/asynchronous/hub.py", line 301, in create_loop
poll_timeout = fire_timers(propagate=propagate) if scheduled else 1
File "/home/airflow/.local/lib/python3.7/site-packages/kombu/asynchronous/hub.py", line 143, in fire_timers
entry()
File "/home/airflow/.local/lib/python3.7/site-packages/kombu/asynchronous/timer.py", line 64, in __call__
return self.fun(*self.args, **self.kwargs)
File "/home/airflow/.local/lib/python3.7/site-packages/kombu/asynchronous/timer.py", line 126, in _reschedules
return fun(*args, **kwargs)
File "/home/airflow/.local/lib/python3.7/site-packages/kombu/transport/redis.py", line 557, in maybe_check_subclient_health
client.check_health()
File "/home/airflow/.local/lib/python3.7/site-packages/redis/client.py", line 3522, in check_health
check_health=False)
File "/home/airflow/.local/lib/python3.7/site-packages/redis/connection.py", line 726, in send_command
check_health=kwargs.get('check_health', True))
File "/home/airflow/.local/lib/python3.7/site-packages/redis/connection.py", line 718, in send_packed_command
(errno, errmsg))
redis.exceptions.ConnectionError: Error 110 while writing to socket. Connection timed out.
[2022-09-22 19:29:26,023: WARNING/MainProcess] /home/airflow/.local/lib/python3.7/site-packages/celery/worker/consumer/consumer.py:367: CPendingDeprecationWarning:
I referred the Airflow's documentation and found setting up database
We are modifying Airflow's docker image and adding a python file,
airflow.www.db_utils.db_config (This file is installed to site_packages) and defined the dictionary
keepalive_kwargs = {
"keepalives": 1,
"keepalives_idle": 30,
"keepalives_interval": 5,
"keepalives_count": 5,
}
Finally, we are setting
ENV AIRFLOW__DATABASE__SQL_ALCHEMY_CONNECT_ARGS="airflow.www.db_utils.db_config.keepalive_kwargs"
Unfortunately, the error stills persist. It will be great if someone helps me to resolve this issue.
Our webserver is running and working fine. Although in the logs we are seeing these errors periodically . Can someone provide pointers around these
[2022-04-05 02:53:58 -0400] [129502] [ERROR] Socket error processing request.
Traceback (most recent call last):
File "/airflow/airflowvirt/lib64/python3.6/site-packages/gunicorn/workers/sync.py", line 135, in handle
req = next(parser)
File "/airflow/airflowvirt/lib64/python3.6/site-packages/gunicorn/http/parser.py", line 42, in __next__
self.mesg = self.mesg_class(self.cfg, self.unreader, self.source_addr, self.req_count)
File "/airflow/airflowvirt/lib64/python3.6/site-packages/gunicorn/http/message.py", line 180, in __init__
super().__init__(cfg, unreader, peer_addr)
File "/airflow/airflowvirt/lib64/python3.6/site-packages/gunicorn/http/message.py", line 54, in __init__
unused = self.parse(self.unreader)
File "/airflow/airflowvirt/lib64/python3.6/site-packages/gunicorn/http/message.py", line 192, in parse
self.get_data(unreader, buf, stop=True)
File "/airflow/airflowvirt/lib64/python3.6/site-packages/gunicorn/http/message.py", line 183, in get_data
data = unreader.read()
File "/airflow/airflowvirt/lib64/python3.6/site-packages/gunicorn/http/unreader.py", line 37, in read
d = self.chunk()
File "/airflow/airflowvirt/lib64/python3.6/site-packages/gunicorn/http/unreader.py", line 64, in chunk
return self.sock.recv(self.mxchunk)
File "/usr/lib64/python3.6/ssl.py", line 956, in recv
return self.read(buflen)
File "/usr/lib64/python3.6/ssl.py", line 833, in read
return self._sslobj.read(len, buffer)
File "/usr/lib64/python3.6/ssl.py", line 592, in read
v = self._sslobj.read(len)
OSError: [Errno 0] Error
[2022-04-05 02:53:58 -0400] [129500] [ERROR] Socket error processing request.
I have installed the spacy 3.1.2 and trying to install en_core_web_sm on jupyter notebook in Jupyterlab by using
!python3 -m spacy download en_core_web_sm
but it is showing the following error
2021-09-01 07:00:12.028366: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer.so.6'; dlerror: libnvinfer.so.6: cannot open shared object file: No such file or directory
2021-09-01 07:00:12.028469: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer_plugin.so.6'; dlerror: libnvinfer_plugin.so.6: cannot open shared object file: No such file or directory
2021-09-01 07:00:12.028486: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:30] Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/urllib3/contrib/pyopenssl.py", line 488, in wrap_socket
cnx.do_handshake()
File "/opt/conda/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1934, in do_handshake
self._raise_ssl_error(self._ssl, result)
File "/opt/conda/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1671, in _raise_ssl_error
_raise_current_error()
File "/opt/conda/lib/python3.7/site-packages/OpenSSL/_util.py", line 54, in exception_from_error_queue
raise exception_type(errors)
OpenSSL.SSL.Error: [('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/urllib3/connectionpool.py", line 667, in urlopen
self._prepare_proxy(conn)
File "/opt/conda/lib/python3.7/site-packages/urllib3/connectionpool.py", line 932, in _prepare_proxy
conn.connect()
File "/opt/conda/lib/python3.7/site-packages/urllib3/connection.py", line 371, in connect
ssl_context=context,
File "/opt/conda/lib/python3.7/site-packages/urllib3/util/ssl_.py", line 384, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/opt/conda/lib/python3.7/site-packages/urllib3/contrib/pyopenssl.py", line 494, in wrap_socket
raise ssl.SSLError("bad handshake: %r" % e)
ssl.SSLError: ("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])",)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/opt/conda/lib/python3.7/site-packages/urllib3/connectionpool.py", line 727, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "/opt/conda/lib/python3.7/site-packages/urllib3/util/retry.py", line 439, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /explosion/spacy-models/master/compatibility.json (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])")))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/opt/conda/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/jovyan/.local/lib/python3.7/site-packages/spacy/__main__.py", line 4, in <module>
setup_cli()
File "/home/jovyan/.local/lib/python3.7/site-packages/spacy/cli/_util.py", line 69, in setup_cli
command(prog_name=COMMAND)
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/home/jovyan/.local/lib/python3.7/site-packages/typer/main.py", line 497, in wrapper
return callback(**use_params) # type: ignore
File "/home/jovyan/.local/lib/python3.7/site-packages/spacy/cli/download.py", line 35, in download_cli
download(model, direct, sdist, *ctx.args)
File "/home/jovyan/.local/lib/python3.7/site-packages/spacy/cli/download.py", line 67, in download
compatibility = get_compatibility()
File "/home/jovyan/.local/lib/python3.7/site-packages/spacy/cli/download.py", line 78, in get_compatibility
r = requests.get(about.__compatibility__)
File "/opt/conda/lib/python3.7/site-packages/requests/api.py", line 76, in get
return request('get', url, params=params, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/requests/api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/requests/sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "/opt/conda/lib/python3.7/site-packages/requests/sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/requests/adapters.py", line 514, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /explosion/spacy-models/master/compatibility.json (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])")))
Due to this, I can't use
import spacy
nlp = spacy.load('en_core_web_sm')
Because it is showing error as:
OSError: [E050] Can't find model 'en_core_web_sm'. It doesn't seem to be a Python package or a valid path to a data directory.
I have searched various post and articles related to it but nothing is working for me.
Can anyone tell me how to resolve this error ? It would be very helpful for me.
my ~/.aws/credentials looks like
[default]
aws_access_key_id = XYZ
aws_secret_access_key = ABC
[testing]
source_profile = default
role_arn = arn:aws:iam::54:role/ad
I add my remote like
dvc remote add --local -v myremote s3://bib-ds-models-testing/data/dvc-test
I have made my .dvc/config.local to look like
[‘remote “myremote”’]
url = s3://bib-ds-models-testing/data/dvc-test
access_key_id = XYZ
secret_access_key = ABC/h2hOsRcCIFqwYWV7eZaUq3gNmS
profile=‘testing’
credentialpath = /Users/nyt21/.aws/credentials
but still after running dvc push -r myremote I get
ERROR: unexpected error - Forbidden: An error occurred (403) when calling the HeadObject operation: Forbidden
** Update
here is the output of dvc push -v
2021-07-25 22:40:38,887 DEBUG: Check for update is enabled.
2021-07-25 22:40:39,022 DEBUG: Preparing to upload data to 's3://bib-ds-models-testing/data/dvc-test'
2021-07-25 22:40:39,022 DEBUG: Preparing to collect status from s3://bib-ds-models-testing/data/dvc-test
2021-07-25 22:40:39,022 DEBUG: Collecting information from local cache...
2021-07-25 22:40:39,022 DEBUG: Collecting information from remote cache...
2021-07-25 22:40:39,022 DEBUG: Matched '0' indexed hashes
2021-07-25 22:40:39,022 DEBUG: Querying 1 hashes via object_exists
2021-07-25 22:40:39,644 ERROR: unexpected error - Forbidden: An error occurred (403) when calling the HeadObject operation: Forbidden
------------------------------------------------------------
Traceback (most recent call last):
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/s3fs/core.py", line 246, in _call_s3
out = await method(**additional_kwargs)
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/aiobotocore/client.py", line 154, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/s3fs/core.py", line 1057, in _info
out = await self._simple_info(path)
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/s3fs/core.py", line 970, in _simple_info
out = await self._call_s3(
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/s3fs/core.py", line 265, in _call_s3
raise translate_boto_error(err)
PermissionError: Access Denied
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/s3fs/core.py", line 246, in _call_s3
out = await method(**additional_kwargs)
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/aiobotocore/client.py", line 154, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/dvc/main.py", line 55, in main
ret = cmd.do_run()
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/dvc/command/base.py", line 50, in do_run
return self.run()
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/dvc/command/data_sync.py", line 57, in run
processed_files_count = self.repo.push(
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/dvc/repo/__init__.py", line 51, in wrapper
return f(repo, *args, **kwargs)
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/dvc/repo/push.py", line 44, in push
pushed += self.cloud.push(objs, jobs, remote=remote)
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/dvc/data_cloud.py", line 79, in push
return remote_obj.push(
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/dvc/remote/base.py", line 57, in wrapper
return f(obj, *args, **kwargs)
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/dvc/remote/base.py", line 494, in push
ret = self._process(
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/dvc/remote/base.py", line 351, in _process
dir_status, file_status, dir_contents = self._status(
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/dvc/remote/base.py", line 195, in _status
self.hashes_exist(
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/dvc/remote/base.py", line 145, in hashes_exist
return indexed_hashes + self.odb.hashes_exist(list(hashes), **kwargs)
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/dvc/objects/db/base.py", line 438, in hashes_exist
remote_hashes = self.list_hashes_exists(hashes, jobs, name)
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/dvc/objects/db/base.py", line 389, in list_hashes_exists
ret = list(itertools.compress(hashes, in_remote))
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/concurrent/futures/_base.py", line 619, in result_iterator
yield fs.pop().result()
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/concurrent/futures/_base.py", line 444, in result
return self.__get_result()
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
raise self._exception
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/dvc/objects/db/base.py", line 380, in exists_with_progress
ret = self.fs.exists(path_info)
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/dvc/fs/fsspec_wrapper.py", line 92, in exists
return self.fs.exists(self._with_bucket(path_info))
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/fsspec/asyn.py", line 87, in wrapper
return sync(self.loop, func, *args, **kwargs)
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/fsspec/asyn.py", line 68, in sync
raise result[0]
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/fsspec/asyn.py", line 24, in _runner
result[0] = await coro
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/s3fs/core.py", line 802, in _exists
await self._info(path, bucket, key, version_id=version_id)
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/s3fs/core.py", line 1061, in _info
out = await self._version_aware_info(path, version_id)
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/s3fs/core.py", line 1004, in _version_aware_info
out = await self._call_s3(
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/s3fs/core.py", line 265, in _call_s3
raise translate_boto_error(err)
PermissionError: Forbidden
------------------------------------------------------------
2021-07-25 22:40:39,712 DEBUG: Version info for developers:
DVC version: 2.5.4 (pip)
---------------------------------
Platform: Python 3.8.10 on macOS-10.16-x86_64-i386-64bit
Supports:
http (requests = 2.26.0),
https (requests = 2.26.0),
s3 (s3fs = 2021.6.1, boto3 = 1.18.6)
Cache types: reflink, hardlink, symlink
Cache directory: apfs on /dev/disk3s1s1
Caches: local
Remotes: s3
Workspace directory: apfs on /dev/disk3s1s1
Repo: dvc, git
Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
2021-07-25 22:40:39,713 DEBUG: Analytics is enabled.
2021-07-25 22:40:39,765 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', '/var/folders/4x/xhm22wt16gl6m9nvkl9gllkc0000gn/T/tmpo86jdns5']'
2021-07-25 22:40:39,769 DEBUG: Spawned '['daemon', '-q', 'analytics', '/var/folders/4x/xhm22wt16gl6m9nvkl9gllkc0000gn/T/tmpo86jdns5']'
I can upload through python
import boto3
import os
import pickle
bucket_name = 'bib-ds-models-testing'
os.environ["AWS_PROFILE"] = "testing"
session = boto3.Session()
s3_client = boto3.client('s3')
s3_client.upload_file('/Users/nyt21/Devel/DVC/test/data/iris.csv',
'bib-ds-models-testing',
'data/dvc-test/my_iris.csv')
I don't use aws CLI but the following also gives an access deny !
aws s3 ls s3://bib-ds-models-testing/data/dvc-test
An error occurred (AccessDenied) when calling the ListObjectsV2
operation: Access Denied
but it works if I add --profile=testing
aws s3 ls s3://bib-ds-models-testing/data/dvc-test --profile=testing
PRE dvc-test/
just you know environment variable AWS_PROFILE is already set to 'testing'
UPDATE
I have tried both AWS_PROFILE='testing' and AWS_PROFILE=testing, neither of them worked.
I am trying to use the responses module to mock an HTTP server and I have the feeling it is broken when calls requests are multi-threaded.
Consider the following:
import json
import responses
import requests
import threading
try:
import urlparse
except ModuleNotFoundError:
import urllib.parse as urlparse
server_url = 'http://server'
headers_json = {'content-type': 'application/json'}
def init():
def endpoint(request):
id = urlparse.parse_qs(urlparse.urlparse(request.path_url).query)["id"][0]
data = {"foo": int(id)}
return 200, headers_json, json.dumps(data)
responses.add_callback(
responses.GET, server_url + '/',
callback=endpoint,
)
#responses.activate
def test():
init()
def responses_routine():
resp = requests.get(
server_url + '/?id=456',
headers=headers_json,
)
# {"foo": "456"}
print(resp.json()["foo"])
def print_routine():
print("456")
# This will print "456"
#responses_routine()
# This will print "456" after 2 seconds
#threading.Timer(2, print_routine).start()
# This will fail
threading.Timer(2, responses_routine).start()
test()
It fails with a stack that clearly indicates that no call to responses was made. The requests module did try to make an HTTP code to a remote server.
Exception in thread Thread-1:
Traceback (most recent call last):
File "/opt/python/python3.6.6/lib/python3.6/site-packages/urllib3/connection.py", line 159, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw)
File "/opt/python/python3.6.6/lib/python3.6/site-packages/urllib3/util/connection.py", line 57, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
File "/opt/python/python3.6.6/lib/python3.6/socket.py", line 745, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/python/python3.6.6/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
chunked=chunked)
File "/opt/python/python3.6.6/lib/python3.6/site-packages/urllib3/connectionpool.py", line 354, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/opt/python/python3.6.6/lib/python3.6/http/client.py", line 1239, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/opt/python/python3.6.6/lib/python3.6/http/client.py", line 1285, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/opt/python/python3.6.6/lib/python3.6/http/client.py", line 1234, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/opt/python/python3.6.6/lib/python3.6/http/client.py", line 1026, in _send_output
self.send(msg)
File "/opt/python/python3.6.6/lib/python3.6/http/client.py", line 964, in send
self.connect()
File "/opt/python/python3.6.6/lib/python3.6/site-packages/urllib3/connection.py", line 181, in connect
conn = self._new_conn()
File "/opt/python/python3.6.6/lib/python3.6/site-packages/urllib3/connection.py", line 168, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x2aaab0930438>: Failed to establish a new connection: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/python/python3.6.6/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/opt/python/python3.6.6/lib/python3.6/site-packages/urllib3/connectionpool.py", line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File "/opt/python/python3.6.6/lib/python3.6/site-packages/urllib3/util/retry.py", line 398, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='server', port=80): Max retries exceeded with url: /?id=456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x2aaab0930438>: Failed to establish a new connection: [Errno -2] Name or service not known',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/python/python3.6.6/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/opt/python/python3.6.6/lib/python3.6/threading.py", line 1182, in run
self.function(*self.args, **self.kwargs)
File "tmp.py", line 35, in responses_routine
headers=headers_json,
File "/opt/python/python3.6.6/lib/python3.6/site-packages/requests/api.py", line 116, in post
return request('post', url, data=data, json=json, **kwargs)
File "/opt/python/python3.6.6/lib/python3.6/site-packages/requests/api.py", line 60, in request
return session.request(method=method, url=url, **kwargs)
File "/opt/python/python3.6.6/lib/python3.6/site-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/opt/python/python3.6.6/lib/python3.6/site-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/opt/python/python3.6.6/lib/python3.6/site-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='server', port=80): Max retries exceeded with url: /?id=456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x2aaab0930438>: Failed to establish a new connection: [Errno -2] Name or service not known',))
It fails on Redhat 6.6. I am using Python 3.6.6 (it also fails in 2.7 with a similar stack) with requests 2.22 and responses 0.10.6.
It's like when the main thread exited, all "handles" to responses were lost...
Any clues ?
Actually what's created within the context manager responses.activate should not escape it.
In the code above, the underlying thread of the Timer object would be joined outside the scope, hence potentially run outside the context manager scope, which yield the exception.