Accessing Jupyter notebook on a remote server - jupyter-notebook

I am trying to use the Jupyter notebook from a remote server using the server's IP address and have followed the suggestions from this SO question:
Why I can't access remote Jupyter Notebook server?
The following command has worked in all the server instances but I am facing issue with one particular instance:
jupyter notebook --ip xx.xx.xx.xx --port 8889 --allow-root
In this particular server instance, this throws the following error:
File "/root/anaconda3/bin/jupyter-notebook", line 11, in <module>
sys.exit(main())
File "/root/anaconda3/lib/python3.8/site-packages/jupyter_core/application.py", line 270, in launch_instance
return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
File "/root/anaconda3/lib/python3.8/site-packages/traitlets/config/application.py", line 663, in launch_instance
app.initialize(argv)
File "<decorator-gen-7>", line 2, in initialize
File "/root/anaconda3/lib/python3.8/site-packages/traitlets/config/application.py", line 87, in catch_config_error
return method(app, *args, **kwargs)
File "/root/anaconda3/lib/python3.8/site-packages/notebook/notebookapp.py", line 1769, in initialize
self.init_webapp()
File "/root/anaconda3/lib/python3.8/site-packages/notebook/notebookapp.py", line 1490, in init_webapp
self.http_server.listen(port, self.ip)
File "/root/anaconda3/lib/python3.8/site-packages/tornado/tcpserver.py", line 151, in listen
sockets = bind_sockets(port, address=address)
File "/root/anaconda3/lib/python3.8/site-packages/tornado/netutil.py", line 174, in bind_sockets
sock.bind(sockaddr)
OSError: [Errno 99] Cannot assign requested address
The issue seems to be with the use of IP address and not the port despite the following settings on jupyter_notebook_config.py
c.NotebookApp.ip = '0.0.0.0'
c.NotebookApp.allow_origin = '*'
c.NotebookApp.allow_remote_access = True
I have looked at solutions on SO and Github and most of them are for localhost and not specific IP addresses.
Can someone point to what could be the issue? Thank you.

Related

Error processing http request on raspberry server

I have set up a raspberry webserver running on my school lan network ,other people are connecting with arduino,sometimes when they connect i get this error:
Exception happened during processing of request from ('172.17.17.66', 49153)
Traceback (most recent call last):
File "/usr/lib/pythonz.7/SocketServer.py", line 290, in
_hand1e_request_nobloc k self.process_request(request, client_address) File
"/usr/lib/pythonz.7/SocketServer.py“, line 318, in process_request
self.finish_request(request, client_address) File
"/usr/lib/pythonz.7/SocketServer.py“, line 331, in finish_request
self.RequestHandlerClass(request, client_address, self) File
"/usr/lib/pythonz.7/SocketServer.py", line 652, in __init__ self.hand1e()
File "/usr/lib/pythonz.7/BaseHTTPServer.py“, line 340, in handle
self.handle_one_request() File "/usr/lib/pythonz.7/BaseHTTPServer.py", line
310, in handle_one request self . raw_requestline = self . rfile .
readline(65537) ‘ File "/usr/lib/pythonz.7/socket.py", line 480, in readline
data = self._sock.recv(se1f._rbufsize) error: [Errno 104] Connection reset
by peer
Can someone tell me what means? Is a problem of my server or their socket? If needed i can post my code.
The last line:
error: [Errno 104] Connection reset by peer
Means the client dropped the connection. I'd look into the Arduino code first.

AWS SAM Local dotnetcore2.1 exception when running API Gateway

Setup
Windows 10
Docker for Windows v18.09.0
AWS SAM CLI v0.10.0
Python 3.7.0
AWS CLI v1.16.67
dotnet core sdk v2.1.403
Powershell v5.1.17134.407
Problem
I'm following the quickstart for AWS SAM Local (as well as the readme generated once the init command is executed below), using the dotnetcore2.1 runtime.
I've run the following command to initialise AWS SAM for use with dotnetcore2.1
sam init --runtime dtonetcore2.1
Then I created the package by running
build.ps1 --target=package
Finally I start the local API Gateway service by running
sam local start-api
I then open a browser and navigate to http://localhost:3000/hello where I'm presented with the following:
PS C:\Users\user_name\Documents\Workspace\messaround\aws-sam\sam-app> sam local start-api
2019-01-04 10:39:15 Found credentials in shared credentials file: ~/.aws/credentials
2019-01-04 10:39:15 Mounting HelloWorldFunction at http://127.0.0.1:3000/hello [GET]
2019-01-04 10:39:15 You can now browse to the above endpoints to invoke your functions. You do not need to restart/reload SAM CLI while working on your functions changes will be reflected instantly/automatically. You only need to restart SAM CLI if you update your AWS SAM template
2019-01-04 10:39:16 * Running on http://127.0.0.1:3000/ (Press CTRL+C to quit)
2019-01-04 10:40:10 Invoking HelloWorld::HelloWorld.Function::FunctionHandler (dotnetcore2.1)
2019-01-04 10:40:10 Decompressing C:\Users\user_name\Documents\Workspace\messaround\aws-sam\sam-app\artifacts\HelloWorld.zip
Fetching lambci/lambda:dotnetcore2.1 Docker container image......
2019-01-04 10:40:13 Mounting C:\Users\user_name\AppData\Local\Temp\tmpq0zka7a7 as /var/task:ro inside runtime container
2019-01-04 10:40:14 Exception on /hello [GET]
Traceback (most recent call last):
File "C:\Users\user_name\AppData\Roaming\Python\Python37\site-packages\docker\api\client.py", line 246, in _raise_for_status
response.raise_for_status()
File "C:\Users\user_name\AppData\Roaming\Python\Python37\site-packages\requests\models.py", line 940, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localnpipe/v1.35/containers/102dda11417068e01873242be2383c78c7ad4e2739fd4f8b42c1e0ea494d2bbb/start
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\user_name\AppData\Roaming\Python\Python37\site-packages\flask\app.py", line 2292, in wsgi_app
response = self.full_dispatch_request()
File "C:\Users\user_name\AppData\Roaming\Python\Python37\site-packages\flask\app.py", line 1815, in full_dispatch_request
rv = self.handle_user_exception(e)
File "C:\Users\user_name\AppData\Roaming\Python\Python37\site-packages\flask\app.py", line 1718, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "C:\Users\user_name\AppData\Roaming\Python\Python37\site-packages\flask\_compat.py", line 35, in reraise
raise value
File "C:\Users\user_name\AppData\Roaming\Python\Python37\site-packages\flask\app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
File "C:\Users\user_name\AppData\Roaming\Python\Python37\site-packages\flask\app.py", line 1799, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "C:\Users\user_name\AppData\Roaming\Python\Python37\site-packages\samcli\local\apigw\local_apigw_service.py", line 153, in _request_handler
self.lambda_runner.invoke(route.function_name, event, stdout=stdout_stream_writer, stderr=self.stderr)
File "C:\Users\user_name\AppData\Roaming\Python\Python37\site-packages\samcli\commands\local\lib\local_lambda.py", line 85, in invoke
self.local_runtime.invoke(config, event, debug_context=self.debug_context, stdout=stdout, stderr=stderr)
File "C:\Users\user_name\AppData\Roaming\Python\Python37\site-packages\samcli\local\lambdafn\runtime.py", line 86, in invoke
self._container_manager.run(container)
File "C:\Users\user_name\AppData\Roaming\Python\Python37\site-packages\samcli\local\docker\manager.py", line 98, in run
container.start(input_data=input_data)
File "C:\Users\user_name\AppData\Roaming\Python\Python37\site-packages\samcli\local\docker\container.py", line 187, in start
real_container.start()
File "C:\Users\user_name\AppData\Roaming\Python\Python37\site-packages\docker\models\containers.py", line 390, in start
return self.client.api.start(self.id, **kwargs)
File "C:\Users\user_name\AppData\Roaming\Python\Python37\site-packages\docker\utils\decorators.py", line 19, in wrapped
return f(self, resource_id, *args, **kwargs)
File "C:\Users\user_name\AppData\Roaming\Python\Python37\site-packages\docker\api\container.py", line 1075, in start
self._raise_for_status(res)
File "C:\Users\user_name\AppData\Roaming\Python\Python37\site-packages\docker\api\client.py", line 248, in _raise_for_status
raise create_api_error_from_http_exception(e)
File "C:\Users\user_name\AppData\Roaming\Python\Python37\site-packages\docker\errors.py", line 31, in create_api_error_from_http_exception
raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 500 Server Error: Internal Server Error ("error while creating mount source path '/host_mnt/c/Users/user_name/AppData/Local/Temp/tmpq0zka7a7': mkdir /host_mnt/c/Users/user_name/AppData: permission denied")
2019-01-04 10:40:14 127.0.0.1 - - [04/Jan/2019 10:40:14] "GET /hello HTTP/1.1" 502 -
2019-01-04 10:40:14 127.0.0.1 - - [04/Jan/2019 10:40:14] "GET /favicon.ico HTTP/1.1" 403 -
What I've tried
Resetting the shared drive credentials
Initially I though this was a permissioning error between my Windows drive and the VM running docker... After searching the docker forums I found this article which I've followed. However this doesn't seem to have changed the error message
Any suggestions would be greatly received. Thanks
That's how I fixed my problem:
When SAM CLI sees a zip, it unzip into a temp directory (looks to be C:/Users/user_name/AppData/Local/Temp/tmpq0zka7a7 in your case).
Docker must have access to that folder.
In my case, I've created a local user to give Docker access to shared drives and that local user didn't have access to C:/Users/user_name.
I gave it access and got my problem sorted. Maybe you can fix it the same way.
Try to run the following:
docker run --rm -v c:/Users/user_name:/data alpine ls /data
It should list c:/Users/user_name content if all is fine.
Good luck!

Install CDH 6.0.1 have trouble with install cm-agent

with cloudera install doc step by step I have in trouble with Install Agents
like this:
It said install failed and can not receive signal.
And I find the log like this:
[13/Nov/2018 16:44:19 +0000] 4306 MainThread agent ERROR Heartbeating to ryze-1.bigdata.com:7182 failed.
Traceback (most recent call last):
File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/agent.py", line 1371, in _send_heartbeat
response = self.requestor.request('heartbeat', heartbeat_data)
File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/avro/ipc.py", line 141, in request
return self.issue_request(call_request, message_name, request_datum)
File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/avro/ipc.py", line 254, in issue_request
call_response = self.transceiver.transceive(call_request)
File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/avro/ipc.py", line 483, in transceive
result = self.read_framed_message()
File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/avro/ipc.py", line 489, in read_framed_message
framed_message = response_reader.read_framed_message()
File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/avro/ipc.py", line 417, in read_framed_message
raise ConnectionClosedException("Reader read 0 bytes.")
ConnectionClosedException: Reader read 0 bytes.
I try to solve it with google I already check these setting.
/etc/cloudera-scm-agent/config.ini the port set 7182 and server_host set ryze-1.bigdata.com.
iptable altready shutdown with sudo service iptables stop
ryze-1.bigdata.com is reachable. and telnet ryze-1.bigdata.com 7183 can succeed.
OS: Centos7.4
Platform: AliCloud
So what can I do? Any one can help me ?
I closed the ssl option.
Everything is fine now.......

Why Airflow Worker Tries to connect to local MYSQL server rather than taking from airflow config

When I start a airflow worker with some queue It throws the below Error.
I have configured db details in airflow config. but not sure why it tries to connect to local mysql server
[2018-10-29 18:59:32,012: ERROR/MainProcess] Pool callback raised exception: OperationalError('(_mysql_exceptions.OperationalError) (2002, "Can\'t connect to local MySQL server through socket \'/var/lib/mysql/mysql.sock\' (2)")',)
Traceback (most recent call last):
File "/home/airflow/.local/lib/python2.7/site-packages/billiard/pool.py", line 1747, in safe_apply_callback
fun(*args, **kwargs)
return dialect.connect(*cargs, **cparams)
File "/home/airflow/.local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 393, in connect
return self.dbapi.connect(*cargs, **cparams)
File "/usr/lib64/python2.7/site-packages/MySQLdb/__init__.py", line 81, in Connect
return Connection(*args, **kwargs)
File "/usr/lib64/python2.7/site-packages/MySQLdb/connections.py", line 193, in __init__
super(Connection, self).__init__(*args, **kwargs2)
OperationalError: (_mysql_exceptions.OperationalError) (2002, "Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)")

Event loop is running error on starting jupyter notebook

For some odd reason, after last night my jupyter kernel wouldn't start so I went ahead and removed it and pip3 installed it and now everytime I run jupyter notebook Im getting the following long stack trace in the termninal
[I 14:59:43.566 NotebookApp] Writing notebook server cookie secret to /run/user/1000/jupyter/notebook_cookie_secret
[I 14:59:43.885 NotebookApp] Serving notebooks from local directory: /home/dmitri/machine_learning/airbusShipDetection
[I 14:59:43.885 NotebookApp] The Jupyter Notebook is running at:
[I 14:59:43.885 NotebookApp] http://localhost:8888/?token=b98d3bd1ec284b81559e0a865c14d6a0932e0725a99d811d
[I 14:59:43.885 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 14:59:43.885 NotebookApp]
Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
http://localhost:8888/?token=b98d3bd1ec284b81559e0a865c14d6a0932e0725a99d811d
[I 14:59:44.624 NotebookApp] Accepting one-time-token-authenticated connection from 127.0.0.1
[I 14:59:48.412 NotebookApp] Kernel started: 15342e49-8fc3-402c-9964-eb8f4e2324c5
Traceback (most recent call last):
File "/usr/lib/python3.5/runpy.py", line 184, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/dmitri/.local/lib/python3.5/site-packages/ipykernel_launcher.py", line 16, in <module>
app.launch_new_instance()
File "/home/dmitri/.local/lib/python3.5/site-packages/traitlets/config/application.py", line 658, in launch_instance
app.start()
File "/home/dmitri/.local/lib/python3.5/site-packages/ipykernel/kernelapp.py", line 477, in start
ioloop.IOLoop.instance().start()
File "/usr/local/lib/python3.5/dist-packages/tornado/platform/asyncio.py", line 132, in start
self.asyncio_loop.run_forever()
File "/usr/lib/python3.5/asyncio/base_events.py", line 340, in run_forever
raise RuntimeError('Event loop is running.')
RuntimeError: Event loop is running.
ERROR:tornado.general:Uncaught exception in ZMQStream callback
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/zmq/eventloop/zmqstream.py", line 432, in _run_callback
callback(*args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/tornado/stack_context.py", line 300, in null_wrapper
return fn(*args, **kwargs)
File "/home/dmitri/.local/lib/python3.5/site-packages/ipykernel/kernelbase.py", line 283, in dispatcher
return self.dispatch_shell(stream, msg)
File "/home/dmitri/.local/lib/python3.5/site-packages/ipykernel/kernelbase.py", line 233, in dispatch_shell
self.pre_handler_hook()
File "/home/dmitri/.local/lib/python3.5/site-packages/ipykernel/kernelbase.py", line 248, in pre_handler_hook
self.saved_sigint_handler = signal(SIGINT, default_int_handler)
File "/usr/lib/python3.5/signal.py", line 47, in signal
handler = _signal.signal(_enum_to_int(signalnum), _enum_to_int(handler))
ValueError: signal only works in main thread
ERROR:tornado.general:Uncaught exception in zmqstream callback
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/zmq/eventloop/zmqstream.py", line 450, in _handle_events
self._handle_recv()
File "/usr/local/lib/python3.5/dist-packages/zmq/eventloop/zmqstream.py", line 480, in _handle_recv
self._run_callback(callback, msg)
File "/usr/local/lib/python3.5/dist-packages/zmq/eventloop/zmqstream.py", line 432, in _run_callback
callback(*args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/tornado/stack_context.py", line 300, in null_wrapper
return fn(*args, **kwargs)
File "/home/dmitri/.local/lib/python3.5/site-packages/ipykernel/kernelbase.py", line 283, in dispatcher
return self.dispatch_shell(stream, msg)
File "/home/dmitri/.local/lib/python3.5/site-packages/ipykernel/kernelbase.py", line 233, in dispatch_shell
self.pre_handler_hook()
File "/home/dmitri/.local/lib/python3.5/site-packages/ipykernel/kernelbase.py", line 248, in pre_handler_hook
self.saved_sigint_handler = signal(SIGINT, default_int_handler)
File "/usr/lib/python3.5/signal.py", line 47, in signal
handler = _signal.signal(_enum_to_int(signalnum), _enum_to_int(handler))
ValueError: signal only works in main thread
[W 15:00:48.619 NotebookApp] Timeout waiting for kernel_info reply from 15342e49-8fc3-402c-9964-eb8f4e2324c5
[I 15:01:49.148 NotebookApp] Saving file at /u-net-model-with-submission.ipynb
^C[I 15:02:46.820 NotebookApp] interrupted
Serving notebooks from local directory: /home/dmitri/machine_learning/airbusShipDetection
1 active kernel
The Jupyter Notebook is running at:
The notebook itself says the kernel is loading, but it never finishes loading.
I had the same problem after upgrading notebook.
Upgrading tornado and ipykernel fixed it for me.
If you use the following commands in your terminal, it should work:
pip install tornado==5.1.1
and
pip install -U ipykernel
After rolling back "tornado" and updating "ipykernel" everything worked.

Resources