I am using curl to open a URL. If worked for few URLs. But for few its giving me an error report. When I open the same url in browser its working fine. The output of both the browser and curl command should be the same, but its not.What could be the reason?
$ curl 'http://server:port/ABC_Service/app'
<html><head><title>VMware vFabric tc Runtime 2.6.4.RELEASE/6.0.35.A.RELEASE - Error report</title><style><!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}HR {color : #525D76;}--></style> </head><body><h1>HTTP Status 401 - </h1><HR size="1" noshade="noshade"><p><b>type</b> Status report</p><p><b>message</b> <u></u></p><p><b>description</b> <u>This request requires HTTP authentication ().</u></p><HR size="1" noshade="noshade"><h3>VMware vFabric tc Runtime 2.6.4.RELEASE/6.0.35.A.RELEASE</h3></body></html>
Expected Output:
$ curl 'http://server:port/ABC_Service/app'
<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
Output in Browser (1st 2 lines):
<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
- <appMetadata>
The browser is just helpfully giving you an interactive view on the XML. Use View Source to see the actual response.
A lot of websites try to detect if a browser can support XML / XSLT . If the user agent is something they know supports it, they send that which you see. If not, they send normal HTML (in your case, an error in HTML ).
You should try setting your user agent :
curl -A "Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_3_3 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8J2 Safari/6533.18.5" http://server:port/ABC_Service/app
You can find a list of user agent string from different devices / programs here
Related
I'm trying to setup monitoring (http-ecv) with authorization header, but I'm getting illegal character or header folding error (jetty 9.3)
1. example
GET /somepath/somepage.html HTTP/1.1
Server Running
\r\nAuthorization: Basic somestring=\r\n
Response "HTTP1.1 400 Illegal character SPACE=''\r\n"
2. example
GET /somepath/somepage.html
Server Running
HTTP/1.1\r\nAuthorization: Basic somestring=\r\n
Response HTTP/1.1 400 Illegal character VCHAR='/'\r\n
This example was working on older jetty version
3.example
GET /somepath/somepage.html
Server Running
\r\nHTTP/1.1\r\nAuthorization: Basic somestring=\r\n
Response HTTP1.1 400 Header Folding\r\n
Any ideas?
try the solution on the article https://support.citrix.com/article/CTX117142
edit to add more context: the article describes how to create a monitor for a back end server that requests basic authentication with a user name and password.
Summarized:
add lb monitor test_login_tcp TCP-ECV -send "GET / HTTP/1.1\r\nAuthorization: Basic YOURBASE64USERPW\r\nHost: IP_or_FQDN\r\n\r\n" -recv 200 -LRTM ENABLED
I've just started working with the Quectel MC60 and I am having some issues:
About HTTP GET method, I make the following commands:
AT+QIFGCNT=0
AT+QICSGP=1,"my_apn"
AT+QIREGAPP
AT+QIACT
AT+QSSLCFG="https",1
AT+QHTTPURL=39,40
my_url_39_bytes_long
AT+QHTTPGET=60
AT+QHTTPREAD=30
AT+QIDEACT
When using the QCOM software, I make a script running all the above commands sequentially. When it comes to the AT+QHTTPREAD command, the response is always "+CME ERROR: 3822" (HTTP response failed). What can it be? I'm sure the HTTP server is working properly.
The answer is that it is necessary to configure the request header
AT+QIFGCNT=0
AT+QICSGP=1,"my_apn"
AT+QIREGAPP
AT+QIACT
AT+QHTTPURL=39,40
my_url_39_bytes_long
AT+QHTTPCFG="requestheader",1
AT+QHTTPPOST=77
GET path HTTP/1.1
User-Agent: Fiddler
Host: www.my_host.com
AT+QHTTPREAD=30
AT+QIDEACT
NOTE: in AT+HTTPPOST=77, 77 is the size of the POST message (last two \r\n are required and count)
NOTE2: after GET you're supposed to write the path to the url inserted in AT+QHTTPURL. For example, if you specified your URL as https://www.my_host.com/debug/main/port, your AT+HTTPPOST request should look like this (don't forget the last two \r\n):
GET /debug/main/port HTTP/1.1
User-Agent: Fiddler
Host: www.my_host.com
I use the following command to send SMS messages through SMS Adapters:
telnet localhost 0000 <<-EOF
helo x
MAIL FROM: Test
RCPT TO: 447999999999
DATA
Test £1234
.
logout
quit
EOF
However when ever I get the Message through it will be in the format:
Test ?£1234
Appending the ? to the front of the £ symbol, I have tried investigating a few methods including MIME however not quite sure how they can be implemented.
Any ideas on how I can stop and allow the successful passthroughs of £
Have you tried encoding the message first? You can do this using base64 in UTF-8 charset:-
e.g.
Convert:
msg="£1234"
To:
msg="wqMxMjM0"
NOTE:Try testing encoding/decoding using the online converter - https://www.base64encode.org/
Once you have encoded your text you can send the message via telnet by adding the MIME details after the DATA command in telnet by specifying MIME types, example script below:-
telnet localhost 0000 <<-EOF
helo x
MAIL FROM: Test
RCPT TO: 447999999999
DATA
Subject: Test
Mime-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: base64
${msg}
.
logout
quit
EOF
Hope that helps.
In some installations, telnet is not 8-bit clean by default (It does not recognize 8-bit character encodings such as Unicode). In order to be able to send Unicode keystrokes to the remote host, you need to set telnet into "outbinary" mode. There are two ways to do this:
$ telnet -L <host>
and
$ telnet
telnet> set outbinary
telnet> open <host>
Source
I am a bit confused about syslog message format. I have to write a program that parses syslog messages. When I read what I get in my syslog-ng instance I get messages like this:
Jan 12 06:30:00 1.2.3.4 apache_server: 1.2.3.4 - - [12/Jan/2011:06:29:59 +0100] "GET /foo/bar.html HTTP/1.1" 301 96 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.2.12) Gecko/20101026 Firefox/3.6.12 ( .NET CLR 3.5.30729)" PID 18904 Time Taken 0
I can clearly determine the real message (which is, in this case an Apache access log message) The rest is metadata about the syslog message itself.
However when I read the RFC 5424 the message examples look like:
without structured data
<34>1 2003-10-11T22:14:15.003Z mymachine.example.com su - ID47 - BOM'su root' failed for lonvick on /dev/pts/8
or with structured data
<165>1 2003-10-11T22:14:15.003Z mymachine.example.com evntslog - ID47 [exampleSDID#32473 iut="3" eventSource="Application" eventID="1011"] BOMAn application event log entry...
So now I am a bit confused. What is the correct syslog message format ? It is a matter of spec version where RFC 5424 obsoleted RFC 3164 ?
The problem in this case is that apache is logging via the standard syslog(3) or via logger. This only supports the old (RFC3164) syslog format, i.e. there is no structured data here.
In order to have the fields from the apache log show up as RFC5424 structured data, apache would need to format the log that way.
The first example is not proper RFC3164 syslog, because the priority value is stripped from the header. Proper RFC3164 format would look like this:
<34>Jan 12 06:30:00 1.2.3.4 apache_server: 1.2.3.4 - - [12/Jan/2011:06:29:59 +0100] "GET /foo/bar.html HTTP/1.1" 301 96 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.2.12) Gecko/20101026 Firefox/3.6.12 ( .NET CLR 3.5.30729)" PID 18904 Time Taken 0
Traditionally rfc3164 syslog messages are saved to files with the priority value removed.
The other two are in RFC5424 format.
If you have access to the installed syslog-daemon on the system you could configure it to write the logs (received both locally or via network) in a different format. rsyslogd for instance allows to configure your own format (just write a template) and also if I remember correctly has a built-in template to store in json format. And there are libraries in almost any language to parse json.
EDIT: You could also make rsyslogd part of your program. rsyslog is very good in reading incoming syslogs in either of the two RFC formats. You can then use rsyslog to output the message in JSON. This way rsyslog does all the decompositioning of the message for you.
I'm requesting a HTML report from an OpenVAS server the following way:
<?xml version="1.0" encoding="utf-8"?>
<commands><authenticate><credentials><username>***</username><password>***</password></credentials></authenticate><get_reports report_id="454a3397-b8a4-408e-9f55-f08972765d30" format_id="b993b6f5-f9fb-4e6e-9c94-dd46c00e058d"/></commands>
However when I read the response I receive only 8192 bytes, which happens not to hold all the HTML code. Is this is a hardcoded limit? How do I get around it?
I have found a similar issue. The cause was related to running
openvas-mkcert-client -n admin -i
Note the trailing -i, this is what causes the break. I have found no solution.