How to add a custom parser to logster? - nginx

I want to track the HTTP response codes returned by my nginx web-server, using logster.
1) I found and installed logster. I also pip-installed pygtail, which is required for logster.
https://github.com/etsy/logster
2) I found a python script that parses nginx access_log and placed it in the parsers subdir.
https://github.com/metabrainz/logster/blob/master/musicbrainz/logster/NginxStatus.py
...but when I run the logster command, I get a python exception:
Traceback (most recent call last):
File "/usr/local/bin/logster", line 5, in <module>
pkg_resources.run_script('logster==0.0.1', 'logster')
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 505, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 1245, in run_script
execfile(script_filename, namespace, namespace)
File "/usr/local/lib/python2.7/dist-packages/logster-0.0.1-py2.7.egg/EGG-INFO/scripts/logster", line 449, in <module>
main()
File "/usr/local/lib/python2.7/dist-packages/logster-0.0.1-py2.7.egg/EGG-INFO/scripts/logster", line 380, in main
module = __import__(module_name, globals(), locals(), [parser_name])
ImportError: No module named NginxStatus1
````
What am I doing wrong?

The exception error was rather misleading: the file was placed in the right place (the parsers subdir), but - as it turns out- logster must be re-setup after a new parser is added (this isn't documented, unfortunately). so just run:
sudo python setup.py install
in the logster directory and things should start working correctly.

Related

Read the Docs with nbsphinx

I created my own docs for Read the Docs. See my repository
Some of my docs files are jupyter notebook so I used nbshpinx for it.
In my computer I installed all the dependencies and it works great when I use make html.
However, Read the docs throws the error:
Running Sphinx v1.8.5
loading translations [en]... done
Traceback (most recent call last):
File "/home/docs/checkouts/readthedocs.org/user_builds/complex-valued-neural-networks/envs/latest/lib/python3.7/site-packages/sphinx/registry.py", line 472, in load_extension
mod = __import__(extname, None, None, ['setup'])
ModuleNotFoundError: No module named 'nbsphinx'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/docs/checkouts/readthedocs.org/user_builds/complex-valued-neural-networks/envs/latest/lib/python3.7/site-packages/sphinx/cmd/build.py", line 303, in build_main
args.tags, args.verbosity, args.jobs, args.keep_going)
File "/home/docs/checkouts/readthedocs.org/user_builds/complex-valued-neural-networks/envs/latest/lib/python3.7/site-packages/sphinx/application.py", line 228, in __init__
self.setup_extension(extension)
File "/home/docs/checkouts/readthedocs.org/user_builds/complex-valued-neural-networks/envs/latest/lib/python3.7/site-packages/sphinx/application.py", line 449, in setup_extension
self.registry.load_extension(self, extname)
File "/home/docs/checkouts/readthedocs.org/user_builds/complex-valued-neural-networks/envs/latest/lib/python3.7/site-packages/sphinx/registry.py", line 475, in load_extension
raise ExtensionError(__('Could not import extension %s') % extname, err)
sphinx.errors.ExtensionError: Could not import extension nbsphinx (exception: No module named 'nbsphinx')
Extension error:
Could not import extension nbsphinx (exception: No module named 'nbsphinx')
Following this tutorial I created two yml files and the error changed to:
Error
Problem in your project's configuration. Invalid "conda.environment": environment not found
Solved it!
I followed this tutorial
I added in readthedocs.yml:
python:
version: 3
install:
- requirements: docs/requirements.txt
system_packages: true
And then in docs/requirements.txt:
ipykernel
nbsphinx
If questions you can always check the repository where I do it.

AttributeError: module 'select' has no attribute 'poll'

I'm running eventlet.monkey_patch() while trying to spin up a flask server which uses flask-socketio. This is the traceback:
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib64/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib64/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/home/alhasan/MeetupPoint/venv/lib/python3.6/site-packages/werkzeug/serving.py", line 777, in inner
srv.serve_forever()
File "/home/alhasan/MeetupPoint/venv/lib/python3.6/site-packages/werkzeug/serving.py", line 612, in serve_forever
HTTPServer.serve_forever(self)
File "/usr/lib64/python3.6/socketserver.py", line 232, in serve_forever
with _ServerSelector() as selector:
File "/usr/lib64/python3.6/selectors.py", line 348, in __init__
self._poll = select.poll()
AttributeError: module 'select' has no attribute 'poll'
I tried using monkey_patch, as previously I encountered the following error:
RuntimeError: You need to use the eventlet server. See the Deployment section of the documentation for more information.
I have eventlet installed.
...
eventlet==0.23.0
Flask==0.12.2
Flask-Migrate==2.1.1
Flask-Script==2.0.6
Flask-SocketIO==3.0.1
...
Is there a fix for this?
My initial problem was that my server returns bad requests everytime I try to emit a message from the client. But, the other way works. Would really appreciate any kind of a solution. :)

Uploading Binary File to Openstack Swift Gives Error

I am trying to upload a binary file I have to Openstack Swift. This size of this file is 129MB. I am using the Python client for uploading:
swift_client = swiftclient.Connection(preauthurl=swift_public_auth_url,preauthtoken=swift_access_token,auth_version="2.0",insecure=True,tenant_name="admin")
with open("/path/to/file/file.ckpt", 'r') as read_file:
swift_client.put_object(swift_container_name, 'file.ckpt', contents= read_file.read(),content_type='text/plain')
print "done"
This throws the following error:
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "/usr/local/lib/python2.7/dist-packages/swiftclient/client.py", line 1824, in put_object
response_dict=response_dict)
File "/usr/local/lib/python2.7/dist-packages/swiftclient/client.py", line 1673, in _retry
service_token=self.service_token, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/swiftclient/client.py", line 1322, in put_object
raise ClientException.from_response(resp, 'Object PUT failed', body)
swiftclient.exceptions.ClientException: Object PUT failed: https://abc.xyz/swift/v1/AUTH_123456789abcdefghijklmn/container/file.ckpt 413 Request Entity Too Large [first 60 chars of response] <html>
<head><title>413 Request Entity Too Large</title></h
However, when I tried replacing test.ckpt with a blank text file test.txt, it got uploaded with the same parameters. I know this error occurs when the file size is greater than 5GB but what can be the problem here?
Found out that the problem was because of configuration on my Nginx proxy server. Got it fixed by increasing client_max_body_size. More details here: https://www.cyberciti.biz/faq/linux-unix-bsd-nginx-413-request-entity-too-large/

How to compile Google Closure code in modules using closure_builder.py?

How can I compile Google Closure code in multiple modules using closure_builder.py?
I tried this command:
python closurebuilder.py
--root closure/closure-library
--root closure/client
--namespace="client.start"
--output_mode=compiled
--compiler_jar=compiler.jar
--output_file closure/compiled.js
--compiler_flags=--compilation_level=ADVANCED_OPTIMIZATIONS
--compiler_flags=--module base:1
--compiler_flags=--js=closure/modules/base.js
--compiler_flags=--module editor:1
--compiler_flags=--js=closure/modules/editor.js
--compiler_flags=--module maps:1
--compiler_flags=--js=closure/modules/maps.js
--compiler_flags=--module uploader:1
--compiler_flags=--js=closure/modules/uploader.js
--compiler_flags=--module visualization:1
--compiler_flags=--js=closure/modules/visualization.js
--compiler_flags=--output_wrapper="(function() {%output%})();"
But the following error is thrown:
closurebuilder.py: Scanning paths...
Traceback (most recent call last):
File "closurebuilder.py", line 256, in <module>
main()
File "closurebuilder.py", line 196, in main
sources.add(_PathSource(js_path))
File "closurebuilder.py", line 167, in __init__
super(_PathSource, self).__init__(source.GetFileContents(path))
File "source.py", line 110, in GetFileContents
fileobj = open(path)
IOError: [Errno 2] No such file or directory: 'base:1'
I actually just ran into a similar issue, except that it wasn't 'base:1' but 'Files'. Turns out, i was executing the script from C:\Program Files (x86)... and those extra spaces messed with the Closure script and how it resolves JS files.
Executing it from a path with no space solved it, hope you can fix it as well, maybe try adding quotes over arguments
Fabien

plone Assertion error in themes

Iam new to Plone. I installed Plone 4 and i have gone through http://noenieto.com/blog/theming-plone-4 and created theme product.
When i run ./bin/buildout,
iam getting the error as
Develop: '/usr/local/Plone/zinstance/src/plonetheme.hanging'
error in plonetheme.hanging setup command: Distribution contains no modules or packages for namespace package 'plonetheme'
While:
Installing.
Processing develop directory '/usr/local/Plone/zinstance/src/plonetheme.hanging'.
An internal error occured due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
File "/usr/local/Plone/buildout-cache/eggs/zc.buildout-1.4.4-py2.7.egg/zc/buildout/buildout.py", line 1683, in main
getattr(buildout, command)(args)
File "/usr/local/Plone/buildout-cache/eggs/zc.buildout-1.4.4-py2.7.egg/zc/buildout/buildout.py", line 417, in install
installed_develop_eggs = self._develop()
File "/usr/local/Plone/buildout-cache/eggs/zc.buildout-1.4.4-py2.7.egg/zc/buildout/buildout.py", line 657, in _develop
zc.buildout.easy_install.develop(setup, dest)
File "/usr/local/Plone/buildout-cache/eggs/zc.buildout-1.4.4-py2.7.egg/zc/buildout/easy_install.py", line 895, in develop
*args) == 0
AssertionError
Can anyone help me to rectify this error?
Create a module or package for namespace package 'plonetheme', e.g.:
mkdir -p src/plonetheme.hanging/plonetheme/hanging
touch src/plonetheme.hanging/plonetheme/__init__.py
touch src/plonetheme.hanging/plonetheme/hanging/__init__.py
src/plonetheme.hanging/plonetheme/__init__.py should contain something like:
from pkg_resources import declare_namespace
declare_namespace(__name__)

Resources