Syslog-NG two relay server issue - syslog

I am trying to forward logs through two syslog-ng relay server, which adds the first relay server IP as a source and in my SIEM, I am seeing all logs are coming from the first syslog relay server.
Setup is below.
Client --> Syslog-Relay1 ---> Syslog-Relay2 ---> SIEM
In SIEM I am seeing all the log source as Syslog-Relay1. I have played with multiple option, but no hope yet. Any idea what I am missing here ? I am not finding any proper documents / forums which explains this setup. This we are looking to meet some specific log flow, in case if you have a question why I am trying to achieve this. Thanks in advance
Following is my configuration:
Syslog-Relay1
#version:3.5
#include "scl.conf"
# syslog-ng configuration file.
#
# This should behave pretty much like the original syslog on RedHat. But
# it could be configured a lot smarter.
#
# See syslog-ng(8) and syslog-ng.conf(5) for more information.
#
# Note: it also sources additional configuration files (*.conf)
# located in /etc/syslog-ng/conf.d/
options {
time-reap(30);
mark-freq(10);
# keep-hostname(yes);
keep-hostname(no);
log_msg_size(65536);
log_fifo_size(10000);
threaded(yes);
flush_lines(100);
use_dns(no);
stats_freq(60);
mark_freq(36400);
use_fqdn(no);
# chain-hostnames(yes);
chain-hostnames(no);
};
source s_syslog_over_network {
network(
ip(0.0.0.0)
log-fetch-limit(200)
log-iw-size(1000000)
keep-alive(yes)
max_connections(10000)
port(9999)
transport("tcp")
flags(no-parse)
);
};
destination d_syslog_tcp {
network(
"10.12.86.98"
transport("tcp")
port(12229)
);
};
log {
source(s_syslog_over_network);
destination(d_syslog_tcp);
};
Syslog-Relay2
#version:3.5
#include "scl.conf"
# syslog-ng configuration file.
#
# This should behave pretty much like the original syslog on RedHat. But
# it could be configured a lot smarter.
#
# See syslog-ng(8) and syslog-ng.conf(5) for more information.
#
# Note: it also sources additional configuration files (*.conf)
# located in /etc/syslog-ng/conf.d/
options {
time-reap(30);
mark-freq(10);
# keep-hostname(yes);
keep-hostname(no);
log_msg_size(65536);
log_fifo_size(10000);
threaded(yes);
flush_lines(100);
use_dns(no);
stats_freq(60);
mark_freq(36400);
use_fqdn(no);
# chain-hostnames(yes);
chain-hostnames(no);
};
source s_syslog_over_network {
network(
ip(0.0.0.0)
log-fetch-limit(200)
log-iw-size(1000000)
keep-alive(yes)
max_connections(10000)
port(12229)
transport("tcp")
flags(no-parse)
);
};
destination d_syslog_tcp {
network(
"10.12.86.76"
transport("tcp")
port(12221)
);
};
log {
source(s_syslog_over_network);
destination(d_syslog_tcp);
};

If you want to use the Client's IP address in SIEM, you have to:
set keep-hostname(no) and use-dns(no) on Syslog-Relay1
This will discard the orginal HOST field of the messages of Client
and use the IP address of Client instead.
set keep-hostname(yes) on Syslog-Relay2
On Syslog-Relay1, the HOST field of the message was overwritten. You
want to keep this and forward to SIEM.
remove flags(no-parse) from s_syslog_over_network on Syslog-Relay2
The IP of Client is stored in the message, so it has to be parsed before forwarding towards SIEM.

Related

Syslog-ng 3.30.1 over TLS

I want to configure Syslog-ng server to use TLS. Im not very familiar with writing syslog-ng statement, but here is what i have so far. Need help adding the TLS portion and destination
# logs
source s_net2 {
tcp(ip(0.0.0.0) port(6514));
};
destination d_net2 {
file(
"/var/log/syslog-ng/net1/$HOST/$YEAR-$MONTH-$DAY-ess-hbss.log"
perm(644)
create_dirs(yes)
);
};
log { source(s_net2); destination(d_net2); };
source s_net2 {
network(
port(6514) transport("tls")
tls(
ca-file("/path/ca.crt")
# ca_dir(/path/ca.d/)
key-file("/path/server.key")
cert-file("/path/server.crt")
)
);
};
Alternatively, you can use the pkcs12-file() option to specify a PKCS #12 archive containing the CA certificates, server certificate, and the TLS server key.
Docs:
https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edition/3.26/administration-guide/61#TOPIC-1431136
https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edition/3.26/administration-guide/62#TOPIC-1431143

syslog-ng not filtering on tags on remote server

I have an nginx server using syslog-ng to send access and error logs to a remote syslog-ng server. I am having it tag the messages so that the remote server can filter on the tags to put them into files. But the filter statements seem to be not working. On the local client I did a test, sending the messages to a local file using the filters and they work there. But they seem to break somehow when being sent remote.
The config on the client is:
#version: 3.13
#include "scl.conf"
## global options.
options { chain_hostnames(off);
flush_lines(0);
use_dns(no);
use_fqdn(no);
owner("root");
group("adm");
perm(0640);
stats_freq(0);
bad_hostname("^gconfd$");
};
source s_qa_nginx_access {
file("/var/log/nginx/access.log" follow-freq(1)
tags("qa_nginx_access")
flags(no-parse));
};
source s_qa_nginx_error {
file("/var/log/nginx/error.log" follow-freq(1)
tags("qa_nginx_error")
flags(no-parse));
};
destination d_syslog-ng_central { syslog("10.0.0.50" transport("tcp") port(514)); };
log { source(s_qa_nginx_access); destination(d_syslog-ng_central);};
log { source(s_qa_nginx_error); destination(d_syslog-ng_central);};
On the remote syslog-ng server I have
#version: 3.13
#include "scl.conf"
options {
flush_lines(0);
use_dns(no);
use_fqdn(no);
owner("root");
group("adm");
perm(0640);
stats_freq(0);
bad_hostname("^gconfd$");
time-reap(30);
mark-freq(10);
keep-hostname(yes);
};
source s_network { syslog(transport(tcp) port(514)); };
filter f_qa_nginx_access { tags("qa_nginx_access"); };
filter f_qa_nginx_error { tags("qa_nginx_error"); };
destination d_qa_nginx_access {
file(
"/var/log/remote/qa_nginx_access.log"
owner("root")
group("adm")
perm(0640)
);
};
destination d_qa_nginx_error {
file(
"/var/log/remote/qa_nginx_error.log"
owner("root")
group("adm")
perm(0640)
);
};
log { source(s_network); filter(f_qa_nginx_access); destination(d_qa_nginx_access); };
log { source(s_network); filter(f_qa_nginx_error); destination(d_qa_nginx_error); };
If I remove the filter from the log statement all of the log messages go to both files. but with the filter in place nothing makes it to any of the files on the remote server. Is it somehow not sending the tags to remote?
You might want to refer to syslog-ng administration guide. Below are some of the important notes from the guide. If you need to send the tags remotely, use SDATA.meta.tags instead or you can use the template to write is as part of the message too.
Full admin guide can be find at the following link:
https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edition/3.22/administration-guide/58
Tags are available locally, that is, if you add tags to a message on the client, these tags will not be available on the server.
To include the tags in the message, use the ${TAGS} macro in a template. Alternatively, if you are using the IETF-syslog message format, you can include the ${TAGS} macro in the .SDATA.meta part of the message. Note that the ${TAGS} macro is available only in syslog-ng OSE 3.1.1 and later.

syslog-ng revice json string

Now I used syslog-ng recive json-format log and store to local file, but the log was be changed.
pro log:
{"input_name":"sensor_alert","machine":"10.200.249.27"}
currently store log:
"sensor_alert","machine":"10.200.249.27"}`
the key "input_name" was be deleted
syslog-ng config:
source test_src {
udp(
ip(0.0.0.0) port(5115)
);
};
destination test_dest {
file("/data/test_${YEAR}${MONTH}${DAY}.log"
template("$MSG\n")
template-escape(no));
};
log {
source(test_src);
destination(test_dest);
};
Who can tell me the reason, thks.
If you only send the above mentioned string (without any other framing) probably you should turn of parsing in the source with:
udp(... flags(no-parse));
This is going to put everything it received into the MSG macro.
If you have some kind of framing (like syslog) please provide an sample message, because otherwise I can only guess.

syslog NG not starting up when specifying an ip address but works as a catch all and write to file setup

I am trying to setup a syslog NG server where i could collect all the logs. now ive managed to create the settings where the server will collect all the logs from all the servers and write it to a single file. but i was wondering if its possible to create a separate log file for each ip address. my config file is as below and every time i mention network it fails to start. can you please let me know where im going wrong?
log { source(s_src); filter(f_console); destination(d_console_all);
destination(d_xconsole); };
log { source(s_src); filter(f_crit); destination(d_console); };
log {
source(s_src);
};
destination Windest {
file("/var/log/test");
};
source forwarder {
network( ip(192.168.1.140));
};
destination forwarderonedest {
file("/var/log/forwarder1");
};
log {
source(forwarder);
destination(forwarderonedest);
};
the
error i get when i try to restart is
/etc/init.d/syslog-ng restart
[....] Restarting syslog-ng (via systemctl): syslog-ng.serviceJob for syslog-ng.service failed because the control process exited with error code. See "systemctl status syslog-ng.service" and "journalctl -xe" for details.
failed!
what works for me is
};
destination Windest {
file("/var/log/test");
};
source forwarder {
tcp();
udp();
};
destination forwarderonedest {
file("/var/log/forwarder1");
};
log {
source(forwarder);
destination(forwarderonedest);
};
and it works. but all the logs from all the machines get written on to a single file.
You can try the below configuration in order to split logs in two/more files:
As per teh config below , syslog-ng server will be running on 2 different ports (your choice) i.e., 514 and 515.
So, on client you can configure application logs to be forwarded to port 514 and system logs to port number 515.
Syslog-ng server will handle the logs in two different files.
#### Local Logs ####
source s_local { system(); internal(); };
#### Source : Application Logs ####
source s_xyz_network {
network(transport(tcp) ip(192.168.1.140) port (514) flags(syslog-protocol));
};
#### Source: System Logs #####
source s_sys_network {
network(transport(tcp) ip(192.168.1.140) port (515) flags(syslog-protocol));
};
destination d_local {
file("/var/log/syslog-ng/local_sys_logs.log"); };
destination d_xyz_logs {
file(
"/var/log/syslog-ng/centralized_logs_xyz.log"
owner("root")
group("root")
perm(0777)
); };
destination d_sys_logs {
file(
"/var/log/syslog-ng/centralized_sys_logs.log"
owner("root")
group("root")
perm(0777)
); };
log { source(s_xyz_network); destination(d_xyz_logs);};
log { source(s_local); destination(d_local);};
log { source (s_sys_network);destination(d_sys_logs);};
##### Config Ends ########
Hope this will help you :)

The following code for setting proxy in Qt fails in case of manual proxy settings

What is wrong with the code:
if i use system proxy the error displayed is “connection refused”
and if i use manual proxy (proxy address being same) error displayed is “Host not found”
The proxy server is squid with proxy-address:172.16.28.11 and port:3128
Besides, it also doesn’t work for localhost proxy like the one created using "tor" or dynamic port forwarding!
if(settDialog.ui->no_proxy->isChecked())
{
QNetworkProxyFactory::setUseSystemConfiguration (false);
QNetworkProxy::setApplicationProxy(QNetworkProxy::NoProxy);
}
else if(settDialog.ui->use_s_proxy->isChecked())
{
QNetworkProxyFactory::setUseSystemConfiguration (true);
}
else if(settDialog.ui->man_proxy->isChecked())
{
QNetworkProxyFactory::setUseSystemConfiguration (false);
proxy.setHostName(settDialog.ui->proxy_addr->text());
proxy.setPort(settDialog.ui->port_num->value());
if(settDialog.ui->proxyType->currentIndex()==0)
proxy.setType(QNetworkProxy::HttpProxy);
else if(settDialog.ui->proxyType->currentIndex()==1)
proxy.setType(QNetworkProxy::Socks5Proxy);
else if(settDialog.ui->proxyType->currentIndex()==2)
proxy.setType(QNetworkProxy::FtpCachingProxy);
proxy.setHostName(settDialog.ui->username->text());
proxy.setPassword(settDialog.ui->pwd->text());
QNetworkProxy::setApplicationProxy(proxy);
}
I may be over-simplifying things, but this looks like this is a simple incorrect API call.
proxy.setHostName is where you define the host name of the proxy server, you set the user name through the proxy.setUser API, i.e:
proxy.setUser(settDialog.ui->username->text());

Resources