Kibana failure during "Optimizing" step - kibana

[SOLVED] In my case, the logtrail.json was in the wrong directory. It should be in /usr/share/kibana/plugins/logtrail not in /usr/share/kibana/plugins
Docker version: 1.10.3, build 9e83765
OS: SUSE Linux (x86_64)
Docker compose version: 1.7.1, build 0a9ab35
Elastic/Logstash/Kibana/Filebeat version: 7.6.1
Logtrail version: 7.6.1-0.1.31
I have docker-compose.yml file for all of ELK+FB (filebeat) services. For Kibana it's:
kibana:
container_name: kibana
hostname: kibana
build:
context: .
dockerfile: kibanaWithLogtrail.Dockerfile
logging:
driver: "json-file"
options:
max-file: "5"
max-size: "100m"
ports:
- "5601:5601"
networks:
- elk_stack_net
depends_on:
- elasticsearch
- logstash
- filebeat
And kibanaWithLogtrail.Dockerfile is:
FROM docker.elastic.co/kibana/kibana:7.6.1
RUN cd /usr/share/kibana && ./bin/kibana-plugin install https://github.com/sivasamyk/logtrail/releases/download/v0.1.31/logtrail-7.6.1-0.1.31.zip
COPY logtrail.json /usr/share/kibana/plugins/
When I do docker-compose up -d it does:
Creating network "elk_elk_stack_net" with driver "bridge"
Building kibana
Step 1 : FROM docker.elastic.co/kibana/kibana:7.6.1
---> f9ca33465ce3
Step 2 : RUN cd /usr/share/kibana && ./bin/kibana-plugin install https://github.com/sivasamyk/logtrail/releases/download/v0.1.31/logtrail-7.6.1-0.1.31.zip
---> Running in 21143bdfdab7
Attempting to transfer from https://github.com/sivasamyk/logtrail/releases/download/v0.1.31/logtrail-7.6.1-0.1.31.zip
Transferring 4715586 bytes....................
Transfer complete
Retrieving metadata from plugin archive
Extracting plugin archive
Extraction complete
Plugin installation complete
---> 42fbfd43f29c
Removing intermediate container 21143bdfdab7
Step 3 : COPY logtrail.json /usr/share/kibana/plugins/
---> ad684aaad73e
Removing intermediate container 6fce376b8719
Successfully built ad684aaad73e
Creating elasticsearch
Creating logstash
Creating filebeat
Creating kibana
The only problem I have is that kibana can't start. It tries to perform "optimizie" step:
["info","optimize"],"pid":7,"message":"Optimizing and caching bundles
for core, graph, monitoring, space_selector, login,
overwritten_session, logout, logged_out, ml, dashboardViewer, apm,
maps, canvas, infra, siem, uptime and lens. This may take a few
minutes"}
But after a minute or two it fails with:
very long message that start with:
"message": "{ Error: Optimizations failure.\n 6589 modules\n \n ERROR in
./x-pack/legacy/plugins/canvas/public/components/app/index.js\n
Module not found: Error: Can't resolve
'plugins/interpreter/canvas/load_legacy_server_function_wrappers' in
'/usr/share/kibana/x-pack/legacy/plugins/canvas/public/components/app'\n
\n ERROR in
./x-pack/legacy/plugins/canvas/public/lib/run_interpreter.js\n
Module not found: Error: Can't resolve
'plugins/interpreter/canvas/load_legacy_server_function_wrappers' in
'/usr/share/kibana/x-pack/legacy/plugins/canvas/public/lib'\n \n
ERROR in ./x-pack/legacy/plugins/canvas/public/legacy_start.js\n
Module not found: Error: Can't resolve
'plugins/interpreter/interpreter' in
'/usr/share/kibana/x-pack/legacy/plugins/canvas/public'\n \n
ERROR in
./x-pack/legacy/plugins/canvas/public/components/app/index.js\n
Module not found: Error: Can't resolve
'plugins/interpreter/interpreter' in
'/usr/share/kibana/x-pack/legacy/plugins/canvas/public/components/app'\n
\n ERROR in
./x-pack/legacy/plugins/canvas/public/components/datasource/datasource_preview/index.js\n
Module not found: Error: Can't resolve
'plugins/interpreter/interpreter' in
'/usr/share/kibana/x-pack/legacy/plugins/canvas/public/components/datasource/datasource_preview'\n
\n ERROR in
./x-pack/legacy/plugins/canvas/public/components/function_form_list/index.js\n
Module not found: Error: Can't resolve
'plugins/interpreter/interpreter' in
'/usr/share/kibana/x-pack/legacy/plugins/canvas/public/components/function_form_list'\n
\n ERROR in
./x-pack/legacy/plugins/canvas/public/functions/filters.js\n Module
not found: Error: Can't resolve 'plugins/interpreter/interpreter' in
'/usr/share/kibana/x-pack/legacy/plugins/canvas/public/functions'\
and the errors like this:
FATAL Error: Optimizations failure. 6589 modules
ERROR in. / x - pack / legacy / plugins / canvas / public / components
/ app / index.js Module not found: Error: Can 't resolve ' plugins /
interpreter / canvas / load_legacy_server_function_wrappers ' in ' /
usr / share / kibana / x - pack / legacy / plugins / canvas / public /
components / app '
ERROR in ./x-pack/legacy/plugins/canvas/public/lib/run_interpreter.js
Module not found: Error: Can' t resolve 'plugins/interpreter/canvas/load_legacy_server_function_wrappers' in
'/usr/share/kibana/x-pack/legacy/plugins/canvas/public/lib'
ERROR in. / x - pack / legacy / plugins / canvas / public /
legacy_start.js Module not found: Error: Can 't resolve ' plugins /
interpreter / interpreter ' in ' / usr / share / kibana / x - pack /
legacy / plugins / canvas / public '
ERROR in ./x-pack/legacy/plugins/canvas/public/components/app/index.js
Module not found: Error: Can' t resolve 'plugins/interpreter/interpreter' in
'/usr/share/kibana/x-pack/legacy/plugins/canvas/public/components/app'
The weird part is that it works fine when I delete COPY logtrail.json /usr/share/kibana/plugins from dockerfile. But it worked before and I didn't change anything, so I truly doubt that logtrail.json file is wrong or have incorrect configuration
The seconds thing is that I am not low on resources. There is enough storage and RAM.
Other services start properly

Try use different path
COPY logtrail.json /usr/share/kibana/plugins/logtrail/logtrail.json

Related

Can't reach builds.midonet.org when installing Eucalyptus

As the subject says, I am doing the local installing running bash <(curl -Ls https://get.eucalyptus.cloud) but I am getting the following errors:
[Ansible] Installing Eucalyptus ansible package
Failed to set locale, defaulting to C
http://builds.midonet.org/midonet-5.2/stable/el7/repodata/repomd.xml: [Errno 12] Timeout on http://builds.midonet.org/midonet-5.2/stable/el7/repodata/repomd.xml: (28, 'Connection timed out after 30000 milliseconds')
Trying other mirror.
http://builds.midonet.org/midonet-5.2/stable/el7/repodata/repomd.xml: [Errno 12] Timeout on http://builds.midonet.org/midonet-5.2/stable/el7/repodata/repomd.xml: (28, 'Connection timed out after 30002 milliseconds')http://builds.midonet.org/midonet-5.2/stable/el7/repodata/repomd.xml: [Errno 12] Timeout on http://builds.midonet.org/midonet-5.2/stable/el7/repodata/repomd.xml: (28, 'Connection timed out after 30001 milliseconds')
Trying other mirror.
It says it is trying other mirror but it doesn't appear that way.
I tried to ping the domain but no response and navigating to the root domain via a web browser fails to show anything. Is this something on my side or is the host really down?
This is my first time looking at Eucalyptus.cloud
if you have a webserver laying around
---
- name: make sure destination dir exists
file:
path: '/tmp/midonet/'
state: directory
tags:
- localinstall
- name: download a copy of the packages that should have been in the midonet repo
get_url:
url: 'http://pxe.server.lan/packages/midonet/{{ item }}'
dest: '/tmp/midonet/'
with_items:
- libzookeeper-3.4.8-4.x86_64.rpm
- libzookeeper-devel-3.4.8-4.x86_64.rpm
- lldpd-0.9.5-2.1.x86_64.rpm
- lldpd-debuginfo-0.9.5-2.1.x86_64.rpm
- lldpd-devel-0.9.5-2.1.x86_64.rpm
- midolman-5.2.2-1.0.el7.noarch.rpm
- midonet-cluster-5.2.2-1.0.el7.noarch.rpm
- midonet-selinux-1.0-2.el7.centos.noarch.rpm
- midonet-tools-5.2.2-1.0.el7.noarch.rpm
- python-midonetclient-5.2.2-1.0.el7.noarch.rpm
- python-zookeeper-3.4.8-4.x86_64.rpm
- quagga-0.99.23-0.el7.midokura.x86_64.rpm
- zkdump-1.05-1.noarch.rpm
- zkpython-3.4.5-2.x86_64.rpm
- zookeeper-3.4.8-4.x86_64.rpm
- zookeeper-debuginfo-3.4.8-4.x86_64.rpm
- zookeeper-lib-3.4.5-1.x86_64.rpm
register: lidownlowd
retries: 3
delay: 3
until: lidownlowd is not failed
tags:
- localinstall
- name: localinstall all packages from midonet repo
shell: yum -y localinstall *.rpm
args:
chdir: '/tmp/midonet/'
please refer to unable to install eucalyptus in centos 7.9
i'll give it a go and let you know the outcome

.NET Core WebAPI - AWS ElasticBeanstalk Nginx Configuration

I've moved my .NET Core WebAPI to an AWS EB Instance. This works fine, however my file upload has now broken due to the Nginx 'client_max_body_size' configuration.
I've looked up how to override this, however I cannot get this to work. Here is my current set-up:
Within this file I have added:
client_max_body_size 0;
This file has no hidden characters (Notepad++ with show all chars on):
When my EB environment is updated I get the following eb-engine log:
2022/05/17 07:15:57.025994 [ERROR] An error occurred during execution of command [app-deploy] - [FlipProxyForDotNetCore]. Stop running the command. Error: copy proxy conf from staging failed with error validate nginx configuration failed with error Command /bin/sh -c /usr/sbin/nginx -t -c /var/proxy/staging/nginx/nginx.conf failed with error exit status 1. Stderr:nginx: [emerg] unknown directive "client_max_body_size" in /var/proxy/staging/nginx/conf.d/myConf.conf:1
nginx: configuration file /var/proxy/staging/nginx/nginx.conf test failed
My EB instance is running Amazon Linux 2/2.3.1.
What am I doing wrong?
The error msg
"client_max_body_size"
indicates that you have some non-ASCII characters () in your myConf.conf. Please open the file in a regular text editor and verify that it has only ASCII characters.

Run nginx.exe on Windows as another user

I'm working on a project for deploying pentest lab with terraform & ansible. All is working good except that last problem.
In my lab I have a nginx server running on a Windows server. Nginx with php works when I start them as Administrator with ansible but i need them to run with a non admin local account.
For the php i've made a wrapper using this tools : https://github.com/antonioCoco/RunasCs
But it doesn't work with nginx cause of a working directory problem :
Here is the error :
PS C:\Users\Administrator> .\RunAsCs.exe nginx ***** C:\Web\nginx-1.19.6\nginx.exe
[*] Warning: GetUserProfileDirectory failed with error code: 2
[*] Warning: Unable to obtain environment for user 'nginx'.
[*] Warning: Environment of created process might be incorrect.
nginx: [alert] could not open error log file: CreateFile() "logs/error.log" failed (3: The system cannot find the path specified)
2021/03/06 10:18:33 [emerg] 5556#6124: CreateFile() "C:\Windows\system32/conf/nginx.conf" failed (3: The system cannot find the path specified)
And that's normal because as you can see my wrapper start in Windows/System32
I would like to know if there is a solution either with nginx.conf or with ansible to start this exe as the "nginx" user.
This is a working code for starting nginx as Administrator
- name: Starting web server
win_shell: .\nginx.exe
args:
chdir: C:\Web\nginx-1.19.6
async: 180
poll: 0
I know that there is a psexec module in ansible but psexec will work only for Local Admin account and the goal of that is that my nginx don't run as Local Admin.
Thanks for the help !

AWS 'eb init' not a valid key=value pair (missing equal-sign) Bitbucket pipelines

I am setting up Bitbucket pipeline for a Wordpress deployment on Elastic Beanstalk.
This error message pops up in line eb init --region AWS_DEFAULT_REGION --platform php-5.5 :
ERROR: ServiceError - '/20200712/$AWS_DEFAULT_REGION/elasticbeanstalk/aws4_request' not a valid key=value pair (missing equal-sign) in Authorization header: 'AWS4-HMAC-SHA256 Credential=$AWS_ACCESS_KEY_ID/20200712/$AWS_DEFAULT_REGION/elasticbeanstalk/aws4_request, SignedHeaders=content-type;host;x-amz-content-sha256;x-amz-date, Signature=3d7fef....8383d5'.
Other relevant questions and why it doesn't solve my problem :
1 2 : Don't have /.aws/config in Bitbucket environment
3 : Error message mentioned above
The Bitbucket pipeline till now :
image: tyrellsys/aws-ebcli
pipelines:
branches:
master:
- step:
script:
- export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
- export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
- export AWS_DEFAULT_REGION=$AWS_DEFAULT_REGION
- export APPLICATION_NAME=$APPLICATION_NAME
- export ENVIRONMENT_NAME=$ENVIRONMENT_NAME
- eb init --region ap-southeast-1 --platform php-5.5
- eb deploy wordpress-beanstalk
Based on the comments.
The issue was related to incorrectly set $AWS_ACCESS_KEY_ID, $AWS_SECRET_ACCESS_KEY and $AWS_DEFAULT_REGION.
Fixing the values was the solution to the reported issue with key=value pair (missing equal-sign).

No matching sls found for 'swapfile' in env 'base'

I have successfully been using saltstack for managing virtual and bare-metal Ubuntu-14.04-servers for about a year.
On master, I have the following /srv/salt/top.sls:
base:
'*':
- common
- users
- openvpn # openvpn-formula
- openvpn.config # openvpn-formula
- fail2ban # fail2ban-formula
- fail2ban.config # fail2ban-formula
- swapfile # swapfile-formula
- ntp # ntp-formula to set up and configure the ntp client or serv
In /etc/salt/master I have included the following:
gitfs_remotes:
- https://github.com/srbolle/openvpn-formula.git
- https://github.com/srbolle/postgres-formula.git
- https://github.com/srbolle/fail2ban-formula.git
- https://github.com/srbolle/ntp-formula.git
- https://github.com/srbolle/swapfile-formula.git
I have had no problems with saltstack-formulas until now, but when I recently included the "swapfile-formula" I get the following when running
salt '*' state.highstate:
servername:
Data failed to compile:
----------
No matching sls found for 'swapfile' in env 'base'
Also, I get the same error message when running:
salt 'servername' state.show_sls swapfile
servername:
- No matching sls found for 'swapfile' in env 'base'
When I run:
salt servername state.show_top
'swapfile' is listed. I have tried to clear the cache, restarted servers, recreated servers, used 'kwalify -m top.sls' to validate my top.sls-file. I have spent days on this error, and don't know how to further debug (logs don't show anything suspiciuos).
Thankful for any clues on how to proceed.

Resources