I am using wso2 esb-4.8.1.
I have read about http_ access logs blog but I have small consult like how we can customize the pattern.
Using below line in log4j.properties file I am getting Http,NHttp transports access logs.
log4j.logger.org.apache.synapse.transport.http.access=INFO
Using this property I am getting Http,NHttp transport logs like below.
- 127.0.0.1 - [27/Jan/2015:14:59:54 +0330] "- - " 202 - "-" "-"
But I wish log the values in below format.
%{yyyy-MM-dd HH:mm:ss}t %m %U %T %s %b %h %l %u
2015-01-26 14:36:03 POST /middleware/services/AuditService 0.022 200 428 127.0.0.1 - -
I have tried above format in tomcat 7 Its working fine,So I am expecting in wso2esb Because its using same flavor.
Where can I configure about Custom format for Http ,NHttp transport access logs
I have searched for http_access_log file in wso2esb folder but i haven't get that.
Thanks in advance.
If you need to modify the http_access logs, you can use this WSO2 doc for it. However, i just want to mentioned that, ESB is using pass through transport to communicate. Therefore if you need to see the HTTP request/response that is going through ESB, you can enable the wire logs using log4j.properties file. You just need to uncomment following and log pattern can be modified with log4j.properties
log4j.logger.org.apache.synapse.transport.http.wire=DEBUG
log4j.logger.org.apache.synapse.transport.http.headers=DEBUG
Related
I'm trying to send logs into datadog using rsyslog. Ideally, I'm trying to do this without having the logs stored on the server hosting rsyslog. I've run into an error in my config that I haven't been able to find out much about. The error occurs on startup of rsyslog.
omfwd: could not get addrinfo for hostname '(null)':'(null)': Name or service not known [v8.2001.0 try https://www.rsyslog.com/e/2007 ]
Here's the portion I've added into the default rsyslog.config
module(load="imudp")
input(type="imudp" port="514" ruleset="datadog")
ruleset(name="datadog"){
action(
type="omfwd"
action.resumeRetryCount="-1"
queue.type="linkedList"
queue.saveOnShutdown="on"
queue.maxDiskSpace="1g"
queue.fileName="fwdRule1"
)
$template DatadogFormat,"00000000000000000 <%pri%>%protocol-version% %timestamp:::date-rfc3339% %HOSTNAME% %app-name% - - - %msg%\n "
$DefaultNetstreamDriverCAFile /etc/ssl/certs/ca-certificates.crt
$ActionSendStreamDriver gtls
$ActionSendStreamDriverMode 1
$ActionSendStreamDriverAuthMode x509/name
$ActionSendStreamDriverPermittedPeer *.logs.datadoghq.com
*.* ##intake.logs.datadoghq.com:10516;DatadogFormat
}
First things first.
The module imudp enables log reception over udp.
The module omfwd enables log forwarding over (tcp, udp, ...)
So most probably - or atleast as far as i can tell - with rsyslog you just want to log messages locally and then send them to datadog.
I don't know anything about the $ActionSendStreamDriver tags, so I can't help you there. But what is jumping out is, that in your action you haven't defined where the logs should be sent to.
ruleset(name="datadog"){
action(
type="omfwd"
target="10.100.1.1"
port="514"
protocol="udp"
...
)
...
}
I need to forward messages from a log file to another IP - let's say 127.0.0.1 514. How do I achieve this?
I used this example from the docs of rsyslog:
module(load="imfile" PollingInterval="10") #needs to be done just once
# File 2
input(type="imfile"
File="/path/to/file2"
Tag="tag2")
As well as providing it with the following rule:
*.* #127.0.0.1:514
But this ended up sending all of the system's logs including journald.
So how do I correctly use ruleset, input blocks and *.* #127.0.0.1:514 to send logs from file /path/to/file2 to 127.0.0.1:514?
Thanks
When specifying the input, also say which ruleset to apply. Input outside the ruleset will not be processed by the ruleset.
module(load="imfile")
input(type="imfile" File="/path/to/file2" Tag="tag2" ruleset="remote")
ruleset(name="remote"){
action(type="omfwd" target="127.0.0.1" port="514" protocol="udp")
# or use legacy syntax:
# *.* #127.0.0.1:514
}
I'm trying to setup monitoring (http-ecv) with authorization header, but I'm getting illegal character or header folding error (jetty 9.3)
1. example
GET /somepath/somepage.html HTTP/1.1
Server Running
\r\nAuthorization: Basic somestring=\r\n
Response "HTTP1.1 400 Illegal character SPACE=''\r\n"
2. example
GET /somepath/somepage.html
Server Running
HTTP/1.1\r\nAuthorization: Basic somestring=\r\n
Response HTTP/1.1 400 Illegal character VCHAR='/'\r\n
This example was working on older jetty version
3.example
GET /somepath/somepage.html
Server Running
\r\nHTTP/1.1\r\nAuthorization: Basic somestring=\r\n
Response HTTP1.1 400 Header Folding\r\n
Any ideas?
try the solution on the article https://support.citrix.com/article/CTX117142
edit to add more context: the article describes how to create a monitor for a back end server that requests basic authentication with a user name and password.
Summarized:
add lb monitor test_login_tcp TCP-ECV -send "GET / HTTP/1.1\r\nAuthorization: Basic YOURBASE64USERPW\r\nHost: IP_or_FQDN\r\n\r\n" -recv 200 -LRTM ENABLED
I am a bit confused about syslog message format. I have to write a program that parses syslog messages. When I read what I get in my syslog-ng instance I get messages like this:
Jan 12 06:30:00 1.2.3.4 apache_server: 1.2.3.4 - - [12/Jan/2011:06:29:59 +0100] "GET /foo/bar.html HTTP/1.1" 301 96 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.2.12) Gecko/20101026 Firefox/3.6.12 ( .NET CLR 3.5.30729)" PID 18904 Time Taken 0
I can clearly determine the real message (which is, in this case an Apache access log message) The rest is metadata about the syslog message itself.
However when I read the RFC 5424 the message examples look like:
without structured data
<34>1 2003-10-11T22:14:15.003Z mymachine.example.com su - ID47 - BOM'su root' failed for lonvick on /dev/pts/8
or with structured data
<165>1 2003-10-11T22:14:15.003Z mymachine.example.com evntslog - ID47 [exampleSDID#32473 iut="3" eventSource="Application" eventID="1011"] BOMAn application event log entry...
So now I am a bit confused. What is the correct syslog message format ? It is a matter of spec version where RFC 5424 obsoleted RFC 3164 ?
The problem in this case is that apache is logging via the standard syslog(3) or via logger. This only supports the old (RFC3164) syslog format, i.e. there is no structured data here.
In order to have the fields from the apache log show up as RFC5424 structured data, apache would need to format the log that way.
The first example is not proper RFC3164 syslog, because the priority value is stripped from the header. Proper RFC3164 format would look like this:
<34>Jan 12 06:30:00 1.2.3.4 apache_server: 1.2.3.4 - - [12/Jan/2011:06:29:59 +0100] "GET /foo/bar.html HTTP/1.1" 301 96 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.2.12) Gecko/20101026 Firefox/3.6.12 ( .NET CLR 3.5.30729)" PID 18904 Time Taken 0
Traditionally rfc3164 syslog messages are saved to files with the priority value removed.
The other two are in RFC5424 format.
If you have access to the installed syslog-daemon on the system you could configure it to write the logs (received both locally or via network) in a different format. rsyslogd for instance allows to configure your own format (just write a template) and also if I remember correctly has a built-in template to store in json format. And there are libraries in almost any language to parse json.
EDIT: You could also make rsyslogd part of your program. rsyslog is very good in reading incoming syslogs in either of the two RFC formats. You can then use rsyslog to output the message in JSON. This way rsyslog does all the decompositioning of the message for you.
Whenever a URL that has %2F which is the hex code for / is posted to my JBOSS Server, I get an error:
HTTP 400 Bad Request error message.
Here is the URL:
http://localhost:8080/application/**abc%2Fhi**?msg=hello"
If I remove the %2F from the URL the link works fine.
This %2F has to be part of the URL and cannot be a request parameter.
Finally figured out the cause of this (both for JBoss and Apache). Both applications intentionally reject URIs with an encoded slash (%2F for / and %5C for \) to prevent possible security vulnerabilities.
Links:
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2007-0450
http://securitytracker.com/id/1018110 (Look at section 4. Solution)
And here are the instructions they provide for enabling this behavior in JBoss:
Note: In response to CVE-2007-0450, JBoss AS considers encoded slashes and backslashes in URLs invalid and its usage will result in HTTP 400 error. It is possible to allow encoded slashes and backslashes by following the steps outlined below, however doing so will expose you to CVE-2007-0450 related attacks:
a) If you use the /var/lib/jbossas/bin/run.sh setup, please edit /etc/jbossas/run.conf and append
- -Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true
- -Dorg.apache.catalina.connector.CoyoteAdapter.ALLOW_BACKSLASH=true to the string assigned to JAVA_OPTS
b) If you use the init script setup to run multiple JBoss AS services and you wish to allow encoding by default on all services, please edit /etc/jbossas/jbossas.conf and add the line JAVA_OPTS="${JAVA_OPTS}
- -Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true
- -Dorg.apache.catalina.connector.CoyoteAdapter.ALLOW_BACKSLASH=true"
c) If you use the init script setup to run multiple JBoss AS services and want to allow encoding of slashes and backslashes for a particular service, please edit /etc/sysconfig/${NAME} (where NAME is the name of your service) and add the line JAVA_OPTS="${JAVA_OPTS}
- -Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true
- -Dorg.apache.catalina.connector.CoyoteAdapter.ALLOW_BACKSLASH=true"
For Apache, it's as simple as setting "AllowEncodedSlashes NoDecode" somewhere in your apache conf or vhost conf (doesn't work in an .htaccess, however).
Apache link: http://httpd.apache.org/docs/current/mod/core.html#allowencodedslashes