I have the following NXLog config that reads a file from disk and forwards it to another host (syslog). By default, it appears NXLog only forwards the data in the log file if it has changed.
How can you configure 'nxlog.conf' so that it always forwards the file, even if the data is the same?
<Input in>
Module im_file
File "C:\\myfile.txt"
</Input>
<Output out>
Module om_tcp
Host 127.0.0.1
Port 12345
</Output>
<Route 1>
Path in => out
</Route>
After some digging on the NXLog forum, this cannot be done. Basically, NXLog will monitor the file (you can configure the frequency) for changes and re-transmit when it has appended values (or changed).
So, a workaround would be to delete the file and re-write the contents (on some schedule perhaps) to get NXLog to forward it again.
NXLog won't re-read data from a file unless that file is updated while nxlog is running.
There is another workaround however.
Add the ReadFromLast FALSE and SavePos FALSE to the im_file module definition and then periodically run nxlog -c <custom config>
The custom config will have to specify a different pid location though.
Related
i would like cloud-init to not use 50-cloud-init.yaml. I have prepared my own file.
Do you know how to do this
You can add
network:
config: disabled
to /etc/cloud/cloud.cfg or a file in /etc/cloud/cloud.cfg.d.
Another option is to add
network-config=disabled
to the kernel command line.
While the network config yaml technically works as userdata, the network configuration will have already been written out before userdata is read.
One other option is to write your netplan configuration into /etc/netplan/99-some-name.yaml. If you have configuration there that overlaps with what is in 50-cloud-init.yaml, your configuration will override what is in the default configuration.
See https://cloudinit.readthedocs.io/en/latest/topics/network-config.html#disabling-network-configuration .
We're got a pre-existing rsyslog config file which is working for papertrail e.g.
/etc/rsyslog.d/20-papertrail.conf which has
*.* #logs4.papertrailapp.com:44407
However we've got a couple of NGINX websites on the server so would like to have it also monitor their error logs.
The paths to them are:
/var/log/nginx/www.website-one.com-error.log
/var/log/nginx/www.website-two.com-error.log
/var/log/nginx/www.website-three.com-error.log
However this /var/log/nginx also contains a bunch of .log files which we do not want to monitor e.g.
/var/log/nginx/error.log
/var/log/nginx/access.log
/var/log/nginx/error.log1
/var/log/nginx/nginx.log
In my head we need to add something like...
/var/log/nginx/*-error.log
And make sure they pipe to the papertrail url as well.
However I'm struggling to decipher the rsyslog documentation to figure out how to do this.
Thanks!
In rsyslog documentation it seems that you can use wildcards in files.
File
The file being monitored. So far, this must be an absolute name (no macros or templates). Note that wildcards are supported at the file name level (see WildCards below for more details).
WildCards
Before Version: 8.25.0
Wildcards are only supported in the filename part, not in directory names.
/var/log/*.log works.
/var/log/*/syslog.log does not work.
Since Version: 8.25.0
Wildcards are supported in filename and paths which means these samples will work:
/var/log/*.log works.
/var/log/*/syslog.log works.
/var/log/*/*.log works.
All matching files in all matching subfolders will work. Note that this may decrease performance in imfile depending on how many directories and files are being watched dynamically.
If you want to forward your vhosts logs you can change configuration directly in NGINX vhosts configuration, you should change/add access_log and error_log policies as explained here or use custom facilities to forward your logs (using rsyslog).
HOW TO DO IT USING RSYSLOG?
Create a new custom file in /etc/rsyslog.d/nginx_custom.conf:
module(load="imfile" PollingInterval="1") #needs to be done just once
# File 1
input(type="imfile"
File="/var/log/nginx/www.website-*.com-error.log"
Tag="websites"
Facility="local0")
local0.* #logs4.papertrailapp.com:44407
#Just to test that logs are forwarded, comment the line once you've tested it
local0.* /var/log/test.log
And restart rsyslog service
NOTE: Line local0.* /var/log/test.log is just to test that you can see forwarded logs into your local server, comment this line after you've tested that everything works.
I'm trying to send multiple nginx logs to loggly...
Config file: /etc/rsyslog.d/21-nginx.conf
$ModLoad imfile
#$InputFilePollInterval 10
$InputFileMode inotify
$WorkDirectory /var/spool/rsyslog
$PrivDropToGroup adm
# nginx access file:
$InputFileName /var/log/nginx/*access.log
$InputFileTag nginx-access:
$InputFileStateFile stat-nginx-access
$InputFileSeverity info
$InputFilePersistStateInterval 20000
$InputRunFileMonitor
# other stuff continues......
after restart i get this error in log syslog:
imfile: The to-be-monitored file "/var/log/nginx/*access.log" contains wildcards. This is not supported in polling mode. [v8.16.0 try http://www.rsyslog.com/e/2420 ]
activation of module imfile failed [v8.16.0 try http://www.rsyslog.com/e/-3 ]
did i make something wrong?
Are there other places in your rsyslog configuration where the file mode is changed to pulling or the file poll interval is active? The problem with using this kind of legacy syntax is that all the configuration is loaded globally, so things in other configuration files can interact. You might consider using the new action syntax so that the inotify mode is applied to the specific source. You can see an example of it here http://www.rsyslog.com/doc/v8-stable/configuration/modules/imfile.html
I want to send my weblogic log to syslog. here is what I have done so far.
1.Included following log4j.properties in managed server classpath -
log4j.rootLogger=DEBUG,syslog
log4j.appender.syslog=org.apache.log4j.net.SyslogAppender
log4j.appender.syslog.Threshold=DEBUG
log4j.appender.syslog.Facility=LOCAL7
log4j.appender.syslog.FacilityPrinting=false
log4j.appender.syslog.Header=true
log4j.appender.syslog.SyslogHost=localhost
log4j.appender.syslog.layout=org.apache.log4j.PatternLayout
log4j.appender.syslog.layout.ConversionPattern=[%p] %c:%L - %m%n
2. added following command to managed server arguments -
-Dlog4j.configuration=file :<path to log4j properties file> -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.Log4JLogger -Dweblogic.log.Log4jLoggingEnabled=true
3. Added wllog4j.jar and llog4j-1.2.14.jar into domain's lib folder.
4.Then, from Admin console changed logging information by doing the following. "my_domain_name"--->Configuration--->Logging--->(Advanced options)-->Logging implementation: Log4J
Restart managed server.
I used this as refernce. But didnt get anaything in syslog(/var/log/message). What am I doing wrong?
I would recommend a couple items to check:
Remove the space in DEBUG, syslog in the file
Your last two server arguments have a space between the - and the D so make sure that wasn't just a copy and paste error in this post.
Double check that the log files are in the actual classpath.
Double check from a ps command, that the -D options made it correctly into the start command that was executed.
Make sure that the managed server has a copy of the JARs correctly as they would get synchornized from admin during the restart.
Hopefully something in there will help or give an idea of what to look for.
--John
I figured out the problem. My appender was working fine, the problem was in rsyslog.conf. Just uncommented following properties
# Provides UDP syslog reception
#$ModLoad imudp
#$UDPServerRun 514
We were appending the messages, but the listner was abesnt, so it didnt knew what to do with it.
and from *.debug;mail.none;authpriv.none;cron.none /var/log/messages it figures out where to redirect any (debug in this case) information to messages file.
has anyone tried to use a log4j.xml reference within a WinRun4j service configuration. here is a copy of my service.ini file. I have tried many configuration combinations. this is just my latest attempt
service.class=org.boris.winrun4j.MainService
service.id=SimpleBacnetIpDataTransfer
service.name=Simple Backnet IP DataTransfer Service
service.description=This is the service for the Simple Backnet IP DataTransfer.
service.startup=auto
classpath.1=C:\Inbox\DataTransferClient-1.0-SNAPSHOT-jar-with-dependencies.jar
classpath.2=WinRun4J.jar
classpath.3=C:\Inbox\log4j-1.2.16.jar
arg.1=C:\Inbox\DataTransferClient.xml
log=C:\WinRun4J-Service\SimpleBacnetIpDataTransfer\NBP-DT-service.log
log.overwrite=true
log.roll.size=10MB
[MainService]
class=com.shiftenergy.ws.App
vmarg.1=-Xdebug
vmarg.2=-Xnoagent
vmarg.3=-Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n
vmarg.4=-Dlog4j.configuration=file:C:\Inbox\log4j.xml
within the log4j.xml file, there is reference to a log file for when the application runs. if I run the java -jar -Dlog4j.configuration=file:C:\Inbox\log4j.xml ...., the log file is created accordingly. if I register my service and start the service, the log file does not get created.
has anyone had success using the -D log4j configuration, using winrun4j?
thanks
I think that you provided the vmarg.4 parameter incorrectly. In your case it has to be like:
vmarg.4=-Dlog4j.configurationFile=[Path for log4j.xml]
I am also using the same and in my case, it is working perfectly fine. Please see below example:
vmarg.1=-Dlog4j.configurationFile=.\log4j2.xml
Have you tried setting the path in your code instead:
System.setProperty("log4j.configurationFile", "config/log4j.xml");
I'm using a relative path to a folder named config that contains log4j.xml. An absolute path is not recommended, but may work as well.
Just be sure to set this before making any calls to log4j, including any log4j config settings or static method calls!
System.setProperty("log4j.configurationFile", "config/log4j.xml");
final Logger log = Logger.getLogger(Main.class);
log.info("Starting up");
I didn't specify the log4j path in the ini file, only placed log4j.xml file at the same place the jar was placed.
Also without specify the
System.setProperty("log4j.configurationFile", "config/log4j.xml");
In the Java project it was stored in (src/main/resources) and will be included in the jar, but it will not be that one used if placed outside the jar.