Using environment variable in jupyter_notebook_config.py - jupyter-notebook

I want to use an environment variable as the location for the c.NotebookApp.notebook_dir parameter in jupyter_notebook_config.py but I have added
c.NotebookApp.notebook_dir = $JUPYTER_NOTEBOOKS
I have also created the variable
>export JUPYTER_NOTEBOOKS=/home/jupyter
but when I run Jupyter notebook I get the below error message and I can tell its not pulling the value of the variable and is trying to use the variable as a literal. Has anyone used environment variables successfully in jupyter notebook config?
[E 20:45:04.513 NotebookApp] Exception while loading config file /root/.jupyter/jupyter_notebook_config.py
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/traitlets/config/application.py", line 562, in _load_config_files
config = loader.load_config()
File "/usr/local/lib/python3.6/site-packages/traitlets/config/loader.py", line 457, in load_config
self._read_file_as_dict()
File "/usr/local/lib/python3.6/site-packages/traitlets/config/loader.py", line 489, in _read_file_as_dict
py3compat.execfile(conf_filename, namespace)
File "/usr/local/lib/python3.6/site-packages/ipython_genutils/py3compat.py", line 198, in execfile
exec(compiler(f.read(), fname, 'exec'), glob, loc)
File "/root/.jupyter/jupyter_notebook_config.py", line 766
c.NotebookApp.notebook_dir = $JUPYTER_NOTEBOOKS
^
SyntaxError: unexpected character after line continuation character
[I 20:45:04.523 NotebookApp] Writing notebook server cookie secret to /root/.local/share/jupyter/runtime/notebook_cookie_secret
Traceback (most recent call last):
File "/usr/local/bin/jupyter-notebook", line 11, in <module>
sys.exit(main())
File "/usr/local/lib/python3.6/site-packages/jupyter_core/application.py", line 266, in launch_instance
return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
File "/usr/local/lib/python3.6/site-packages/traitlets/config/application.py", line 657, in launch_instance
app.initialize(argv)
File "</usr/local/lib/python3.6/site-packages/decorator.py:decorator-gen-7>", line 2, in initialize
File "/usr/local/lib/python3.6/site-packages/traitlets/config/application.py", line 87, in catch_config_error
return method(app, *args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/notebook/notebookapp.py", line 1628, in initialize
self.init_webapp()
File "/usr/local/lib/python3.6/site-packages/notebook/notebookapp.py", line 1407, in init_webapp
self.http_server.listen(port, self.ip)
File "/usr/local/lib/python3.6/site-packages/tornado/tcpserver.py", line 151, in listen
sockets = bind_sockets(port, address=address)
File "/usr/local/lib/python3.6/site-packages/tornado/netutil.py", line 174, in bind_sockets
sock.bind(sockaddr)
OSError: [Errno 99] Cannot assign requested address

It isn't clear that you are using environment variables properly. I glanced at the docs and it looks like %env JUPYTER_NOTEBOOKS is the right way to do it. I checked this on my local system and it works (for a different env variable). Give that a try - good luck! :-) . Below are the docs.
https://ipython.readthedocs.io/en/stable/interactive/magics.html#magic-env

Thanks jimf your link helped. I was able to get things working because I was NOT using environment variables correctly. Instead of modifying the .py file I added the parameter to my command of starting jupyter notebook like this:
--NotebookApp.notebook_dir=$JUPYTER_NOTEBOOKS
I was then able to change the variable value and change where jupyter was looking for notebooks.

You can make use of environment variables in the jupyter_notebook_config.py file, but note that the file is a Python file, so (as you already figured out) bash syntax won't work. Instead you can use Python syntax. You can actually write any Python code you like in this file, so this would be valid syntax:
import os
c.NotebookApp.notebook_dir=os.getenv('JUPYTER_NOTEBOOKS')
This would allow you to do even more complex things if you needed.

Related

Building Plone.4-3.14 gives "need more than 0 values to unpack"

When running buildout with the Plone-4.3.14-version-configs, the errors below are thrown.
Getting distribution for 'feedparser==5.0.1'.
error: Not a recognized archive type: /tmp/tmpuOPdYIget_dist/feedparser-5.0.1.tar.bz2
An error occurred when trying to install /tmp/tmpuOPdYIget_dist/feedparser-5.0.1.tar.bz2. Look above this message for any errors that were output by easy_install.
While:
Installing instance.
Getting distribution for 'feedparser==5.0.1'.
An internal error occurred due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
File "/home/ida/.virtenv/lib/python2.7/site-packages/zc/buildout/buildout.py", line 2127, in main
getattr(buildout, command)(args)
File "/home/ida/.virtenv/lib/python2.7/site-packages/zc/buildout/buildout.py", line 797, in install
installed_files = self[part]._call(recipe.install)
File "/home/ida/.virtenv/lib/python2.7/site-packages/zc/buildout/buildout.py", line 1557, in _call
return f()
File "/home/ida/.buildout/eggs/plone.recipe.zope2instance-4.2.22-py2.7.egg/plone/recipe/zope2instance/__init__.py", line 114, in install
installed.extend(self.install_scripts())
File "/home/ida/.buildout/eggs/plone.recipe.zope2instance-4.2.22-py2.7.egg/plone/recipe/zope2instance/__init__.py", line 618, in install_scripts
requirements, ws = self.egg.working_set(['plone.recipe.zope2instance'])
File "/home/ida/.buildout/eggs/zc.recipe.egg-1.3.2-py2.7.egg/zc/recipe/egg/egg.py", line 101, in working_set
**kw)
File "/home/ida/.virtenv/lib/python2.7/site-packages/zc/buildout/easy_install.py", line 924, in install
return installer.install(specs, working_set)
File "/home/ida/.virtenv/lib/python2.7/site-packages/zc/buildout/easy_install.py", line 726, in install
for dist in self._get_dist(req, ws):
File "/home/ida/.virtenv/lib/python2.7/site-packages/zc/buildout/easy_install.py", line 570, in _get_dist
dists = [_move_to_eggs_dir_and_compile(dist, self._dest)]
File "/home/ida/.virtenv/lib/python2.7/site-packages/zc/buildout/easy_install.py", line 1704, in _move_to_eggs_dir_and_compile
[tmp_loc] = glob.glob(os.path.join(tmp_dest, '*'))
ValueError: need more than 0 values to unpack
Took a moment to figure out why, because the explanation does not lie in the last error-message, but the preceding one:
Not a recognized archive type, which hints that easy_install cannot handle bunzip-files.
It means that the required sys-package bzip-devel was not present when installing Python.
So one must either install that package and install Python again, or in this case also commenting out the pin for feedparser lets the build run without errors.
Afterwards noticed that feedparser is not in the eggs-cache-dir, as it used to be in another build with same versions-configs. At a glance cannot tell the difference why, but the errors are resolved.

PermissionError: [Errno 1] Operation not permitted

I am completely new to python, linux RPI and scapy. I am trying to send some packets using scapy.
On Command Line (only if super user privileges are given to scapy)
send(IP(dst="1.2.3.4")/ICMP())
This works perfectly, while running on python script.
from scapy.all import *
p=send(IP(dst="1.2.3.4")/ICMP())
Throws an error
Traceback (most recent call last):
File "<pyshell#19>", line 1, in <module>
send(IP(dst="1.2.3.4")/ICMP())
File "/usr/local/lib/python3.4/dist-packages/scapy/sendrecv.py",line 255, in send
__gen_send(conf.L3socket(*args, **kargs), x, inter=inter, loop=loop, count=count,verbose=verbose, realtime=realtime)
File "/usr/local/lib/python3.4/dist-packages/scapy/arch/linux.py", line 326, in __init__
self.ins = socket.socket(socket.AF_PACKET, socket.SOCK_RAW, socket.htons(type))
File "/usr/lib/python3.4/socket.py", line 123, in __init__
_socket.socket.__init__(self, family, type, proto, fileno)
PermissionError: [Errno 1] Operation not permitted<br>
I am trying to solve it but cannot, I am new to the environment therefore don't know much. As far as my search goes this issue is relevant to sockets. But I still need some simple explanation to understand.
This means that you need to start your script with sudo/admin rights.

(.py) got converted to exe but while executing fails to find ib\ext\anywrapper.py ... (IBPY)

I am using interactive broker api - ibpy. I have generated an executable using pyinstaller as well as cx_freeze.
The executable gets created in both the cases but while executing it gives the same error that it could not find the file ib\ext\AnyWrapper.py.
File "lib\site-packages\ib\opt\message.py", line 153, in <module>
File "lib\site-packages\ib\opt\message.py", line 31, in __ init__
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\admin\\AppData\\Local\\Temp\\_MEI27882\\ib\\ext\\AnyWrapper.py'
In the above example, the temp folder gets deleted, but while using cx_freeze it could see that the file ib\ext\AnyWrapper.pyc existed.
Could this be some error in ib\opt\message.py itself while looking for AnyWrapper? Should it be looking for AnyWrapper and not AnyWrapper.py?
Is it a bug with IBPY or is it something else?

Any other ways to install heat resource plug-in?

According to the directions of Openstack Official Heat/Plugins wiki https://wiki.openstack.org/wiki/Heat/Plugins, we only need "To install a plugin, copy the Python modules to one of the configured plugin directories. Note that heat-engine must be restarted after this in order to load the new plugins.". But I hit the following error messages after I restart heat-engine..
[root#cs14 heat]# heat-engine
2015-05-04 06:02:09.774 20839 WARNING heat.common.config [-] HT-A65A0DF The "instance_user" option in heat.conf is deprecated and will be removed in the Juno release.
[05/04/2015 06:02:10 EDT]heatCRITICAL : ImportError: No module named my_heat_plugin.client
Traceback (most recent call last):
File "/usr/bin/heat-engine", line 67, in <module>
srv = engine.EngineService(cfg.CONF.host, rpc_api.ENGINE_TOPIC)
File "/usr/lib/python2.6/site-packages/heat/engine/service.py", line 288, in __init__
resources.initialise()
File "/usr/lib/python2.6/site-packages/heat/engine/resources/__init__.py", line 44, in initialise
_load_global_environment(global_env)
File "/usr/lib/python2.6/site-packages/heat/engine/resources/__init__.py", line 49, in _load_global_environment
_load_global_resources(env)
File "/usr/lib/python2.6/site-packages/heat/engine/resources/__init__.py", line 54, in _load_global_resources
manager = plugin_manager.PluginManager(__name__)
File "/usr/lib/python2.6/site-packages/heat/engine/plugin_manager.py", line 58, in __init__
self.modules = list(modules())
File "/usr/lib/python2.6/site-packages/heat/common/plugin_loader.py", line 91, in load_modules
module = _import_module(importer, module_name, package)
File "/usr/lib/python2.6/site-packages/heat/common/plugin_loader.py", line 72, in _import_module
module = loader.load_module(module_name)
File "/usr/lib64/python2.6/pkgutil.py", line 238, in load_module
mod = imp.load_module(fullname, self.file, self.filename, self.etc)
File "/usr/lib/heat/abc_heat_plugin/resources/abc/abc_server.py", line 24, in <module>
from abc_heat_plugin.client import constants as const
ImportError: No module named abc_heat_plugin.client
To solve this problem, I've figured out two ways which are workable.
Method 1. copy abc_heat_plugin to /usr/lib/python2.6/site-packages and restart heat-engine
Method 2. use .pth file.
1) Create a file /usr/lib/python2.6/site-packages/.pth with the following three lines (no need to do this if it exists)
/usr/lib/heat
/usr/lib/heat/abc_heat_plugin
/usr/lib/heat/abc_heat_plugin/client
2) copy plugin "my_heat_plugin" to /usr/lib/heat
3) restart heat service
BUT both ways have to do more than the official guide, so I wonder whether I missed anything important. Any suggestions? Thanks.
(BTW, my_heat_plugin is working well.)
I think the basic idea is that you drop the resource in /usr/lib/heat, but any modules or packages that are used by the resource must be installed the usual way. You sort of did that manually by moving the files to site-packages.
Take a look at any of the plugins in heat/contrib for example. There are a few that have dependencies such as clients to other OpenStack services, and in all cases the assumption is made that those clients were installed separately.
So if you can, just create a setup.py for your client package, and install that separately from the heat plugin.

APIGEETool DeployNodeApp

We are trying to use the apigeetool deploynodeapp utility to deploy a node.js app to apigee.
We think we are following the instructions properly, but are getting a "broken pipe" error every time we try.
The command we are running is...
apigeetool deploynodeapp -n sampleService -d . -m server.js -o ourOrgNameHere -e test -b sample -u ourUserName -p ourPassword
Obviously, ourOrgNameHere, ourUserName, and ourPassword are not the values we really used.
When we run that from the command line, there is a 20 second pause, followed by the output below...
Traceback (most recent call last):
File "/usr/local/bin/apigeetool", line 24, in <module>
deploynodeapp.run()
File "/Library/Python/2.7/site-packages/ApigeePlatformTools/deploynodeapp.py", line 180, in run
revision = deploytools.importBundle(Organization, Name, tf.getvalue())
File "/Library/Python/2.7/site-packages/ApigeePlatformTools/deploytools.py", line 115, in importBundle
resp = httptools.httpCall('POST', uri, hdrs, data)
File "/Library/Python/2.7/site-packages/ApigeePlatformTools/httptools.py", line 24, in httpCall
conn.request(verb, uri, body, hdrs)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 973, in request
self._send_request(method, url, body, headers)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1007, in _send_request
self.endheaders(body)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 969, in endheaders
self._send_output(message_body)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 829, in _send_output
self.send(msg)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 805, in send
self.sock.sendall(data)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 229, in sendall
v = self.send(data[count:])
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 198, in send
v = self._sslobj.write(data)
Any help would be greatly appreciated.
You don't have to tell Apigee about the Node module dependencies. The apigeetool utility is supposed to (by default) resolve your Node.js dependencies automatically after your project is uploaded to Apigee Edge (as if running npm install on Edge itself).
Some users have found that if this fails for some reason, it works if you use the --upload-modules option like this:
apigeetool deploynodeapp --upload-modules
The upload-modules flag tells the tool to zip and upload all the modules from your system (rather than trying to update them on Edge). As long as your Node.js works locally, it should work when you deploy it. Try using --upload-modules to see if that works for you.

Resources