Suricata Custom Ruleset will not Load into `suricata.rules` - rules

I am running Suricata in IDS (af-packet) mode on Ubuntu 20.04.5 LTS (Focal Fossa) and deployed as the root user:
NAME="Ubuntu"
VERSION="20.04.5 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.5 LTS"
VERSION_ID="20.04"
Following the Suricata "Adding your own Rules" Doc, I have added a very basic (for complexity ease when troubleshooting) alerting-rule with first available sid:1000000 from custom rules range:
########### Test Rules #############
alert ssh any any -> xxx.xxx.60.6 !22 (msg:"SSH TRAFFIC on non-SSH port"; flow:to_client, not_established; classtype: misc-attack; target: dest_ip; sid:1000000;)
The .rules file for the local rules has sufficient permissions and matches suricata.rules, owned by root:
ls -halt /var/lib/suricata/rules/
total 22M
-rw-r--r-- 1 root root 3.2K Oct 17 00:00 classification.config
drwxr-x--- 2 root root 4.0K Oct 17 00:00 .
-rw-r--r-- 1 root root 22M Oct 17 00:00 suricata.rules
-rw-r--r-- 1 root root 210 Oct 13 21:45 local.rules
Ensured that the rules are added to Suricata suricata.yaml config and processed is restarted:
cat /etc/suricata/suricata.yaml | grep "rule-files" -A 5 -B 5
##
#default-rule-path: /var/lib/suricata/rules
default-rule-path: /etc/suricata/rules
rule-files:
- suricata.rules
- /var/lib/suricata/rules/local.rules
- /etc/suricata/rules/*.rules
AFAIK, the custom ruleset should be loaded into the suricata.rules file? Therefore, I am running the following verification subject to what I am reporting:
cat /var/lib/suricata/rules/suricata.rules | grep sid:1000000
I can test traffic and verify with tcpdump, matching the rule but never see a signature match in fast.log (which is logging other signature-matching traffic):
cat /var/log/suricata/fast.log | grep 1000000
I see no errors following statup of the service that would indicate an error present:
systemctl status suricata.service
● suricata.service - LSB: Next Generation IDS/IPS
Loaded: loaded (/etc/init.d/suricata; generated)
Active: active (running) since Mon 2022-10-17 13:11:39 UTC; 8h ago
Docs: man:systemd-sysv-generator(8)
Process: 2184275 ExecStart=/etc/init.d/suricata start (code=exited, status=0/SUCCESS)
Tasks: 78 (limit: 618963)
Memory: 25.2G
CGroup: /system.slice/suricata.service
└─2184295 /usr/bin/suricata -c /etc/suricata/suricata.yaml --pidfile /var/run/suricata.pid --af-packet -D -v>
Oct 17 13:11:39 sec3 systemd[1]: Starting LSB: Next Generation IDS/IPS...
Oct 17 13:11:39 sec3 suricata[2184275]: Starting suricata in IDS (af-packet) mode... done.
Oct 17 13:11:39 sec3 systemd[1]: Started LSB: Next Generation IDS/IPS.
Can somebody help me with somewhere I may be silly here?
TYIA!

Related

Artifactory failed to initialize due to derby file log size

i'm stuck with an artifactory problem, my artifactory webpage return a 500 status failed to initialize.
First I try to restart the service systemclt restart artifactory.service. The service start with any error output during the operation. Here the output of a systemctl status artifactory after the restart, I try to stop and start too but same result:
➜ log systemctl status artifactory.service -l
● artifactory.service - LSB: Start Artifactory on Tomcat
Loaded: loaded (/etc/init.d/artifactory)
Active: active (running) since mer. 2020-07-22 15:42:06 CEST; 1h 50min ago
Process: 4751 ExecStop=/etc/init.d/artifactory stop (code=exited, status=0/SUCCESS)
Process: 4849 ExecStart=/etc/init.d/artifactory start (code=exited, status=0/SUCCESS)
Main PID: 4915 (java)
CGroup: /system.slice/artifactory.service
‣ 4915 /usr/bin/java -Djava.util.logging.config.file=/home/artifactory/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -server -Xms512m -Xmx2g -Xss256k -XX:+UseG1GC -Djruby.compile.invokedynamic=false -Dfile.encoding=UTF8 -Dartdist=zip -Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true -Dartifactory.home=/home/artifactory -Djava.endorsed.dirs=/home/artifactory/tomcat/endorsed -classpath /home/artifactory/tomcat/bin/bootstrap.jar:/home/artifactory/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/home/artifactory/tomcat -Dcatalina.home=/home/artifactory/tomcat -Djava.io.tmpdir=/home/artifactory/tomcat/temp org.apache.catalina.startup.Bootstrap start
juil. 22 15:42:04 ns337904.ip-**-***-***.** artifactory[4849]: Using ARTIFACTORY_PID: /home/artifactory/run/artifactory.pid
juil. 22 15:42:04 ns337904.ip-**-***-***.** artifactory[4849]: Using CATALINA_BASE: /home/artifactory/tomcat
juil. 22 15:42:04 ns337904.ip-**-***-***.** artifactory[4849]: Using CATALINA_HOME: /home/artifactory/tomcat
juil. 22 15:42:04 ns337904.ip-**-***-***.** artifactory[4849]: Using CATALINA_TMPDIR: /home/artifactory/tomcat/temp
juil. 22 15:42:04 ns337904.ip-**-***-***.** artifactory[4849]: Using JRE_HOME: /usr
juil. 22 15:42:04 ns337904.ip-**-***-***.** artifactory[4849]: Using CLASSPATH: /home/artifactory/tomcat/bin/bootstrap.jar:/home/artifactory/tomcat/bin/tomcat-juli.jar
juil. 22 15:42:04 ns337904.ip-**-***-***.** artifactory[4849]: Using CATALINA_PID: /home/artifactory/run/artifactory.pid
juil. 22 15:42:04 ns337904.ip-**-***-***.** artifactory[4849]: Tomcat started.
juil. 22 15:42:06 ns337904.ip-**-***-***.** artifactory[4849]: Artifactory Tomcat started in normal mode
juil. 22 15:42:06 ns337904.ip-**-***-***.** systemd[1]: Started LSB: Start Artifactory on Tomcat.
But problem isn't fixed. The first error before the restart in the artifactory.log : https://pastebin.com/XEfFmN19
After the restart : https://pastebin.com/EJLAJT91
When I go to the derby.log file same error logged with no more informations. The concerned file(i think) by the error :
➜ cd /home/artifactory/data/derby/log
➜ log ll
total 257M
-rw-r--r-- 1 artifactory artifactory 256M juil. 17 01:05 log3955.dat
-rw-r--r-- 1 root artifactory 0 juil. 22 15:42 log3956.dat
-rw-r--r-- 1 artifactory artifactory 48 juil. 22 15:42 log.ctrl
-rw-r--r-- 1 artifactory artifactory 48 juil. 22 15:42 logmirror.ctrl
-rw-r--r-- 1 artifactory artifactory 532 juil. 10 2016 README_DO_NOT_TOUCH_FILES.txt
So apparently it's a size log file problem. But I don't know how to solve that.
I search in the artifactory conf and see nothing to upgrade the max allow size.
I see nothing in the artifacory docs about that. Anyway I don't know if it's the best way to handle the problem.
Thanks for help
I suggest you consider using an outside db instead of using derby
this link suggests some workarounds on reclaiming space or changing the default storage

Installing devtsack using ./stack.sh fails at devstack#etcd.service

I'm trying to run devstack on Ubuntu 16.04 VM using ./stack.sh
+lib/etcd3:start_etcd3:61 sudo systemctl daemon-reload
+lib/etcd3:start_etcd3:62 sudo systemctl enable devstack#etcd.service
Created symlink from /etc/systemd/system/multi-user.target.wants/devstack#etcd.service to
/etc/systemd/system/devstack#etcd.service.
+lib/etcd3:start_etcd3:63 sudo systemctl start devstack#etcd.service
Job for devstack#etcd.service failed because the control process exited with error code. See "systemctl status
devstack#etcd.service" and "journalctl -xe" for details.
+lib/etcd3:start_etcd3:1 exit_trap
+./stack.sh:exit_trap:515 local r=1
++./stack.sh:exit_trap:516 jobs -p
+./stack.sh:exit_trap:516 jobs=
+./stack.sh:exit_trap:519 [[ -n '' ]]
+./stack.sh:exit_trap:525 '[' -f /tmp/tmp.0IwC5vOcG5 ']'
+./stack.sh:exit_trap:526 rm /tmp/tmp.0IwC5vOcG5
+./stack.sh:exit_trap:530 kill_spinner
+./stack.sh:kill_spinner:425 '[' '!' -z '' ']'
+./stack.sh:exit_trap:532 [[ 1 -ne 0 ]]
+./stack.sh:exit_trap:533 echo 'Error on exit'
Error on exit
+./stack.sh:exit_trap:535 type -p generate-subunit
+./stack.sh:exit_trap:536 generate-subunit 1524706916 269 fail
+./stack.sh:exit_trap:538 [[ -z /opt/stack/logs ]]
+./stack.sh:exit_trap:541 /opt/stack/devstack/tools/worlddump.py -d /opt/stack/logs
World dumping... see /opt/stack/logs/worlddump-2018-04-26-014626.txt for details
+./stack.sh:exit_trap:550 exit 1
When i the run the command sudo systemctl status devstack#etcd.service:
stack#openstack-demo-vm:/opt/stack/devstack$ sudo systemctl status devstack#etcd.service
● devstack#etcd.service - Devstack devstack#etcd.service
Loaded: loaded (/etc/systemd/system/devstack#etcd.service; enabled; vendor preset: enabled)
Active: inactive (dead) (Result: exit-code) since Thu 2018-04-26 01:46:27 UTC; 1min 27s ago
Process: 122376 ExecStart=/opt/stack/bin/etcd --name openstack-demo-vm --data-dir /opt/stack/data/etcd --initial-
cluster-state new --initial-cluster-token etcd-cluster-01 --initial-cluster openst
Main PID: 122376 (code=exited, status=1/FAILURE)
Apr 26 01:46:26 openstack-demo-vm systemd[1]: devstack#etcd.service: Main process exited, code=exited, status=1/FAILURE
Apr 26 01:46:26 openstack-demo-vm systemd[1]: Failed to start Devstack devstack#etcd.service.
Apr 26 01:46:26 openstack-demo-vm systemd[1]: devstack#etcd.service: Unit entered failed state.
Apr 26 01:46:26 openstack-demo-vm systemd[1]: devstack#etcd.service: Failed with result 'exit-code'.
Apr 26 01:46:27 openstack-demo-vm systemd[1]: devstack#etcd.service: Service hold-off time over, scheduling restart.
Apr 26 01:46:27 openstack-demo-vm systemd[1]: Stopped Devstack devstack#etcd.service.
Apr 26 01:46:27 openstack-demo-vm systemd[1]: devstack#etcd.service: Start request repeated too quickly.
Apr 26 01:46:27 openstack-demo-vm systemd[1]: Failed to start Devstack devstack#etcd.service.
lines 1-14/14 (END)
While etcd runs on the VM:
stack#openstack-demo-vm:/opt/stack/devstack$ systemctl status etcd
● etcd.service - etcd - highly-available key value store
Loaded: loaded (/lib/systemd/system/etcd.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2018-04-26 00:12:18 UTC; 1h 58min ago
Docs: https://github.com/coreos/etcd
man:etcd
Main PID: 4425 (etcd)
Tasks: 9
Memory: 9.3M
CPU: 22.020s
CGroup: /system.slice/etcd.service
└─4425 /usr/bin/etcd
What am i missing?
Well, you may simply add the following in your local.conf:
disable_service etcd3
And then, run stack.sh.
Afte some researcha nd asking aroundin Openstack forums and devstack, disabling etcd' instack.sh' helped to resole this.
Steps:
Edit the file /opt/stack/devstack/stach.sh
Comment the below lines (you will find them at around 1035 line)
# Start Services
# ==============
# Dstat
# -----
# A better kind of sysstat, with the top process per time slice
#start_dstat
# Etcd
# -----
# etcd is a distributed key value store that provides a reliable way
to store data across a cluster of machines
#if is_service_enabled etcd3; then
# start_etcd3
#fi
Save the above file.
Run ./unstack.sh
Run ./stack.sh

mount encrypted loopback in suse/systemd: unknown filesystem type 'crypt'

This might be specific to openSUSE / systemd.
I'm having trouble mounting an encrypted loopback file using the procedure described on the SDB:Encrypted filesystems knowledge base. I get this behaviour:
[mjl#tesla:~]
[11:12] $ sudo systemctl start /home/mjl/key
Job for home-mjl-key.mount failed. See "systemctl status home-mjl-key.mount" and "journalctl -xe" for details.
[mjl#tesla:~]
[11:12] 1 $ sudo systemctl status home-mjl-key.mount
● home-mjl-key.mount - /home/mjl/key
Loaded: loaded (/etc/fstab; bad; vendor preset: disabled)
Active: failed (Result: exit-code) since Sun 2018-03-11 11:12:41 AEDT; 3s ago
Where: /home/mjl/key
What: /home/mjl/.tomb
Docs: man:fstab(5)
man:systemd-fstab-generator(8)
Process: 12949 ExecMount=/usr/bin/mount /home/mjl/.tomb /home/mjl/key -t crypt -o loop,user,acl,user_xattr (code=exited, status=32)
Mar 11 11:12:41 tesla systemd[1]: Mounting /home/mjl/key...
Mar 11 11:12:41 tesla mount[12949]: mount: unknown filesystem type 'crypt'
Mar 11 11:12:41 tesla systemd[1]: home-mjl-key.mount: Mount process exited, code=exited status=32
Mar 11 11:12:41 tesla systemd[1]: Failed to mount /home/mjl/key.
Mar 11 11:12:41 tesla systemd[1]: home-mjl-key.mount: Unit entered failed state.
[mjl#tesla:~]
[11:12] 3 $
The /home/mjl/.tomb loopback file was created using YaST Partitioner; I specified that I did not want it mounted at system start time, but that users should be allowed to mount it.
So it created the file, added an entry to /etc/cryptab and also this entry to /etc/fstab:
[mjl#tesla:~]
[11:12] 3 $ tail -n1 /etc/fstab
/home/mjl/.tomb /home/mjl/key crypt loop,user,noauto,acl,user_xattr,nofail 0 0
[mjl#tesla:~]
[11:15]$
There is the 'crypt' filesystem type.
My question is: how should I be mounting this as a user? Is systemd failing because of the filesystem type, or because I haven't told it the encryption key?
I've also tried mounting directly:
[mjl#tesla:~]
[11:16]$ sudo mount /home/mjl/key
mount: unknown filesystem type 'crypt'
[mjl#tesla:~]
The same error. So I guess I'm not mounting it correctly. Do I need to do something with cryptsetup

how to apply salt states to just one environment

I'm trying to apply a salt state to my non prod environment at /srv/salt/non-prod
I'm getting this result:
[root#salt non-prod]# salt '*' state.apply
salt.localdomain:
----------
ID: states
Function: no.None
Result: False
Comment: No Top file or external nodes data matches found.
Changes:
Summary for salt.localdomain
------------
Succeeded: 0
Failed: 1
I have this location defined in my master config
non-prod:
- /srv/non-prod
- /srv/salt/non-prod/services
- /srv/salt/non-prod/states
I have a top file located here:
[root#salt ~]# cat /srv/salt/non-prod/top.sls
base:
'*':
- apache
- python
- ssh
- users
These are the contents of the non-prod directory
[root#salt ~]# ls -lh /srv/salt/non-prod/
total 16K
drwxr-xr-x. 2 root root 4.0K Oct 3 21:02 apache
drwxr-xr-x. 2 root root 45 Oct 3 20:57 python
drwxr-xr-x. 2 salt salt 6 Oct 3 14:10 services
drwxr-xr-x. 2 root root 54 Oct 3 18:23 ssh
drwxr-xr-x. 2 salt salt 6 Oct 3 14:10 states
-rw-r--r--. 1 root root 80 Oct 3 15:29 state.template
-rw-r--r--. 1 root root 174 Oct 3 15:30 test.sls
-rw-r--r--. 1 root root 61 Oct 3 21:14 top.sls
drwxr-xr-x. 2 root root 22 Oct 3 21:03 users
drwxr-xr-x. 2 salt salt 99 Oct 3 18:28 webserver
it contains a few salt modules
How can I apply salt states to just the non-prod environment?
If you check the syntax using some yaml validation tools, then we can go to next step.
Read saltstack top documentation thoroughly, you will notice setting different environment, you first explicitly define alternate environment name on /etc/salt/master and also specify it under top.sls
i.e., you file_roots specify the non-prod environment
file_roots:
#non-prod environment
non-prod:
- /srv/non-prod
- /srv/salt/non-prod/services
- /srv/salt/non-prod/states
Thus your top.sls should use the environment name non-prod , not base
non-prod:
'*':
- apache
- python
- ssh
- users
Since saltstack always use "base" environment by default, you should apply the state explicitly.
salt '*' state.highstate saltenv=non-prod

uWSGI, Nginx, Flask app service keeps failing

Going to my app produces a 502 gateway error. Found out that it was because my how_lit.service is failing. But I am having trouble finding out why.
Tried editing the application and the ini document. Cannot figure out whats wrong.
The Nginx and uWSGI services are up and running fine.
Service Status:
lit#digitalocean:~/howlit$ sudo service how_lit status
[sudo] password for lit:
● how_lit.service - uWSGI instance to serve how lit rest api
Loaded: loaded (/etc/systemd/system/how_lit.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Thu 2016-08-04 00:30:44 EDT; 5 days ago
Process: 14294 ExecStart=/home/lit/howlit/env/bin/uwsgi --ini /home/lit/howlit/howlit.ini (code=exited, status=1/FAILURE)
Main PID: 14294 (code=exited, status=1/FAILURE)
Aug 04 00:30:44 digitalocean systemd[1]: Started uWSGI instance to serve how lit rest api.
Aug 04 00:30:44 digitalocean uwsgi[14294]: [uWSGI] getting INI configuration from /home/lit/howlit/howlit.ini
Aug 04 00:30:44 digitalocean systemd[1]: how_lit.service: Main process exited, code=exited, status=1/FAILURE
Aug 04 00:30:44 digitalocean systemd[1]: how_lit.service: Unit entered failed state.
Aug 04 00:30:44 digitalocean systemd[1]: how_lit.service: Failed with result 'exit-code'.
Directory and Permissions:
lit#digitalocean:~/howlit$ ls -l .
total 16
drwx---r-x 6 lit www-data 4096 Jul 29 11:47 env
-rwx---r-x 1 lit www-data 202 Aug 3 23:29 howlit.ini
-rwx---r-x 1 lit www-data 1203 Aug 3 23:01 how_lit_restapi.py
-rwxr-xr-x 1 lit www-data 72 Aug 3 23:27 wsgi.py
/etc/systemd/system/how_lit.service:
lit#digitalocean:~/howlit$ cat /etc/systemd/system/how_lit.service
[Unit]
Description=uWSGI instance to serve how lit rest api
After=network.target
[Service]
User=lit
Group=www-data
WorkingDirectory=/home/lit/howlit/
Environment="PATH=/home/lit/howlit/env/bin"
ExecStart=/home/lit/howlit/env/bin/uwsgi --ini /home/lit/howlit/howlit.ini
[Install]
WantedBy=multi-user.target
howlit.ini file:
lit#digitalocean:~/howlit$ cat howlit.ini
[uwsgi]
module = wsgi:app
uid = lit
gid = www-data
master = true
processes = 5
socket = how_lit_restapi.sock
chmod-sock = 666
vacum = true
die-on-term = true
gto = /var/log/uwsgi/%n.log
Tried running it by hand:
lit#digitalocean:~/howlit$ /home/lit/howlit/env/bin/uwsgi --ini /home/lit/howlit/howlit.ini
[uWSGI] getting INI configuration from /home/lit/howlit/howlit.ini
*** Starting uWSGI 2.0.13.1 (64bit) on [Tue Aug 9 18:28:25 2016] ***
compiled with version: 5.4.0 20160609 on 29 July 2016 11:48:08
os: Linux-4.4.0-31-generic #50-Ubuntu SMP Wed Jul 13 00:07:12 UTC 2016
nodename: digitalocean
machine: x86_64
clock source: unix
detected number of CPU cores: 1
current working directory: /home/lit/howlit
detected binary path: /home/lit/howlit/env/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
your processes number limit is 1896
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
bind(): Permission denied [core/socket.c line 230]
permission error again?
SOLVED IT: By sending my socket into tmp, but still getting bad gateway error when I navigate to my site :(
Solved my own problem.
First I checked my services.
sudo service nginx status
sudo service uwsgi status
sudo service how_lit status
then I saw them all running and up but was still getting the bad gateway error. Well after checking the logs had no errors. I had to assume my configs.
Then I realized my mistake....I never restarted all of it, just certain parts at certain times. So I restarted every single one as such:
sudo service nginx restart
sudo service uwsgi restart
sudo service how_lit restart
now it works.
About the permission issue I tried it by putting the socket into the /tmp directory that way www-data group users can access it as well as root. I learned that you need to be able to create the socket and allow access to the system for it.
I moved it out of tmp btw later for production as I was told that was not best practice.

Resources