This probably has been asked before, but I couldn't find relevant information and this is the first time it happens to me. Trying to capture/decrypt traffic from ASP.NET web application. Web.config modified accordingly to point to Fiddler as a proxy:
<system.net>
<defaultProxy enabled = "true" useDefaultCredentials = "true">
<proxy autoDetect="False" bypassonlocal="False" proxyaddress="http://127.0.0.1:8888" usesystemdefault="False" />
</defaultProxy>
</system.net>
This has worked many times before, but this time Fiddler cannot decrypt HTTPS traffic, getting this:
HTTP/1.1 200 Connection Established
FiddlerGateway: Direct
StartTime: 08:10:40.453
Connection: close
EndTime: 08:10:40.547
ClientToServerBytes: 432
ServerToClientBytes: 3286
This is a CONNECT tunnel, through which encrypted HTTPS traffic flows.
Fiddler's HTTPS Decryption feature is enabled, but this specific tunnel was configured not to be decrypted. Settings can be found inside Tools > Fiddler Options > HTTPS.
A SSLv3-compatible ServerHello handshake was found. Fiddler extracted the parameters below.
Version: 3.1 (TLS/1.0)
SessionID: 54 87 1E 5C 03 C7 16 81 E6 25 E8 8F 48 C0 42 52 23 B6 5B 04 4F 4D 69 67 11 F2 9E 09 D0 27 77 2D
Random: 54 87 1F 00 0D 37 FF 22 3E 6A 10 BF 4D 4F 67 ED C2 D2 97 A9 66 B2 CF 56 0D 8C 7C E5 6B 2E 2E A1
Cipher: TLS_RSA_AES_128_SHA [0x002F]
CompressionSuite: NO_COMPRESSION [0x00]
Extensions:
renegotiation_info 00
Any idea what is going on here and how to make Fiddler decrypt traffic?
Related
I am trying to build a TCP checking on my backend servers using HAProxy version 1.5.8.
The behavior should be as follows:
Send binary data to server
Receive ACK as first packet
Receive confirmation data in second packet
So I need to check that after sending binary data I received ACK and after that other binary data in a second packet in a row.
Is it possible to do it with HAProxy.
I am trying to find it in documentation and also trying to create different configurations, unsuccessfully:
option tcp-check
tcp-check connect
tcp-check send-binary 303030303030
tcp-check expect binary 303030303030
Every time I received back from server ACK, connection is terminated by HAProxy with the result that the backend server is down.
EDIT:
I will receive the following:
First packet after sending data
0000 a0 66 10 09 2e 46 9c af ca bb aa 47 08 00 45 00 f...F.¯Ê»ªG..E.
0010 00 28 40 58 40 00 3e 06 d7 04 0a 1e 0b 34 0a 02 .(#X#.>.×....4..
0020 06 20 25 1c d5 80 91 0a f8 87 db 03 25 8f 50 10 . %.Õ...ø.Û.%.P.
0030 01 c9 03 d6 00 00 00 00 00 00 00 00 .É.Ö........
Second packet right after the above
0000 a0 66 10 09 2e 46 9c af ca bb aa 47 08 00 45 00 f...F.¯Ê»ªG..E.
0010 00 39 40 59 40 00 3e 06 d6 f2 0a 1e 0b 34 0a 02 .9#Y#.>.Öò...4..
0020 06 20 25 1c d5 80 91 0a f8 87 db 03 25 8f 50 18 . %.Õ...ø.Û.%.P.
0030 01 c9 2d 2e 00 00 00 0f 30 30 30 30 30 30 42 33 .É-.....000000B3
0040 30 30 43 48 45 43 4b 00CHECK
The first is without any data and I need to check that the second contains 000000.
EDIT2:
PCAP provided:
Normal behavior when communication goes directly from client to server, without HAProxy:
Normal behavior - client to server
Using HAProxy as load balancer, connecting to the same server and checking with the same command, failing to check:
failing check - HAProxy to server
backend configuration:
backend nodes
mode tcp
balance roundrobin
default-server inter 10s fall 3 rise 2
option tcp-check
tcp-check connect
tcp-check send-binary 303030303030423230303035434845434b
tcp-check expect binary 000f30303030303042333030434845434b
server server1 10.30.11.52:9500 check
server server2 10.30.11.52:9501 check
server server3 10.30.11.52:9502 check
Receive ACK as first packet
HA proxy does not work at the raw packet level but at the TCP level. At this level there is no such thing as an ACK as a single packet. There is not even the concept of a packet at this level. Instead there is only the concept of a data stream consisting of the received bytes.
Every time I received back from server ACK, connection is terminated by HAProxy with the result that the backend server is down.
Given that HA proxy does not care about packets with zero payload in the first place it is likely that your "ACK as first packet" is actually some packet which contains an ACK (as almost all TCP packets do) but also contains some payload, but not the one you expect with the "next packet". Since the payload does not match the payload you specify as expected the check fails.
Note that this is only an assumption made based on incomplete information about your "ACK as first packet". To prove the assumption one would actually need to see what is really going on on the wire, for example by having a packet capture.
EDIT#1: after the OP provided a some (undocumented) dump of the packets and some figuring out where the actual IP header in these packets starts (offset 14, i.e. prefixed with layer 2 ethernet header) it is clear that the first packet has no payload which means it gets completely ignored by the check. The second packet then has the following payload of 17 bytes:
0030 00 0f 30 30 30 30 30 30 42 33 ..000000B3
0040 30 30 43 48 45 43 4b 00CHECK
Given that the OP checks for binary 303030303030 but the actual payload is 00 0f 30 30 30 30 30 30 .... the given tcp-check expect ... does not match the actual payload and thus the check fails.
EDIT#2:
After the OP has provided the pcap of a connection without and with haproxy a difference in the behavior of both client/haproxy and server can be seen:
without haproxy:
client sends 2 bytes \x00\x11 to the server followed by 17 bytes \x30\x30....
server replies immediately with 17 bytes \x00\x0f\x30\x30....
with haproxy:
haproxy send 17 bytes \x30\x30... to the server. It does not send the initial 2 bytes \x00\x11 as done by the original server !!!
Server does not reply (except an ACK with no payload). After 6 seconds of inactivity haproxy closes the connection to the server and likely considers the check failed.
In summary: I think the haproxy check fails to send the proper request to the server, i.e. the first 2 bytes are missing. That's why the server will not respond at all and the check will fail after some timeout.
tl;dr "What would the bytes 0x33 0x39 0x0d 0x0a between the end of HTTP headers and the start of HTTP response body refer to?"
I'm using the thoroughly excellent libcurl to make HTTP requests to various 3rd party endpoints. These endpoints are not under my control and are required to implement a specification. To help debug and develop these endpoints I have implemented the text output functionality you might see if you make a curl request from the command line with the -v flag using curl.setopt(pycurl.VERBOSE, 1) and curl.setopt(pycurl.DEBUGFUNCTION, debug_function)
This has been working great but recently I've come across a request which my debug function does not handle in the same way as curl's debug output. I'm sure is due to me not understanding the HTTP spec.
If making a curl request from the command line with --verbose I get the following returned.
# redacted headers
< Via: 1.1 vegur
<
{"code":"InvalidCredentials","message":"Bad credentials"}*
Connection #0 to host redacted left intact
If making the same request with --trace the following is returned
0000: 56 69 61 3a 20 31 2e 31 20 76 65 67 75 72 0d 0a Via: 1.1 vegur..
<= Recv header, 2 bytes (0x2)
0000: 0d 0a ..
<= Recv data, 1 bytes (0x1)
0000: 33 3
<= Recv data, 62 bytes (0x3e)
0000: 39 0d 0a 7b 22 63 6f 64 65 22 3a 22 49 6e 76 61 9..{"code":"Inva
0010: 6c 69 64 43 72 65 64 65 6e 74 69 61 6c 73 22 2c lidCredentials",
0020: 22 6d 65 73 73 61 67 65 22 3a 22 42 61 64 20 63 "message":"Bad c
0030: 72 65 64 65 6e 74 69 61 6c 73 22 7d 0d 0a redentials"}..
<= Recv data, 1 bytes (0x1)
0000: 30 0
<= Recv data, 4 bytes (0x4)
0000: 0d 0a 0d 0a ....
== Info: Connection #0 to host redacted left intact
All HTTP client libs I've tested don't include these parts of the bytes in the response body so I'm guessing these are part of the HTTP spec I don't know about but I can't find a reference to them and I don't know how to handle them.
If it's helpful I think curl is using this https://github.com/curl/curl/blob/master/src/tool_cb_dbg.c for building the output in the first example bit I'm not really a c/c++ programmer and I haven't been able to reverse engineer the logic.
Does anyone know what these bytes are?
0d 0a are ASCII control characters representing carriage return and line feed, respectively. CRLF is used in HTTP to mark the end of a header field (there are some historic exceptions you should not worry about at this point). A double CRLF is supposed to mark the end of the fields section of a message.
The 33 39 you observe there is "39" in ascii. This is the chunk size indicator - treated as a hexdecimal number. The presence of Transfer-Encoding: chunked in the response headers may support this.
I'm checking ALPN support on NGINX with latest libressl 2.5.3. I think I have all correctly setup, but when I try to test from:
https://tools.keycdn.com/http2-test
Tell me : Yeah! DOMAIN supports HTTP/2.0
"ALPN is not supported"
Instead when I try to test locally with opnessl we can see alpn is correct with version h2
#openssl s_client -alpn h2 -connect domain:port | grep ALPN
depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
verify return:1
depth=0 CN = fbrqx.com
verify return:1
ALPN protocol: h2
What I'm doing wrong ?
Are correct the information provided by keycdn ?
Thanks.
Ricardo / Brqx.
Today keycdn works and tell us ALPN is supported.
Here we can see libressl information ( so with 2.5.3 version [April 2017] you can test openssl ):
openssl version -a
LibreSSL 2.5.3
built on: date not available
platform: information not available
options: bn(64,64) rc4(16x,int) des(idx,cisc,16,int) idea(int) blowfish(idx)
compiler: information not available
OPENSSLDIR: "/PATH/libressl/libressl-2.5.3/.openssl/etc/ssl"
How to test server :
1.From server ( with openssl client )
openssl s_client -alpn h2 -connect DOMAIN:PORT -status
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
ALPN protocol: h2
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES128-GCM-SHA256
Session-ID: 1FB23766E93F4983F3218F3E1C0058947DD60BEEB414CA50BDDF9009299A273B
Session-ID-ctx:
Master-Key: BBCC316C78C85C90066F660A5C1018F5F71CEADFF13AC406758BA06B0D07A5FFCF21395657CFF1B42760C371EE70281D
TLS session ticket lifetime hint: 300 (seconds)
TLS session ticket:
0000 - 06 48 61 b7 a7 63 23 ad-6a cf f9 47 39 49 c0 1a .Ha..c#.j..G9I..
0010 - b3 04 db ea 02 2e 46 a9-9a 6b 02 de b5 e9 bb c4 ......F..k......
0020 - 68 71 ba b2 62 27 08 76-f0 cb cb e6 e5 7d 4a 50 hq..b'.v.....}JP
0030 - f8 4d 40 24 2a 35 2a 7e-03 5b bc 27 73 7d 1a 14 .M#$*5*~.[.'s}..
0040 - f3 86 9d d9 33 42 49 c7-93 28 a0 f9 e3 4a cb 93 ....3BI..(...J..
0050 - 4d e7 43 bf 8d 8b 9d 3b-06 47 04 77 ca ca 0b 56 M.C....;.G.w...V
0060 - fc 6c ff 0d ac 69 7c b3-bb 6f fb 35 d5 75 7d aa .l...i|..o.5.u}.
0070 - d3 34 e1 04 f9 85 06 50-a2 3d 12 8d 69 39 53 32 .4.....P.=..i9S2
0080 - c7 c4 f3 84 93 fb 87 8c-48 a2 7b 8f 35 5a c0 4f ........H.{.5Z.O
0090 - 3c 58 39 3d 5d cf b1 10-b4 fa 19 9d 43 f2 09 c7 <X9=].......C...
Start Time: 1493112423
Timeout : 7200 (sec)
Verify return code: 0 (ok)
From client - Browser Google Chrome ( will tell us current connections what listen http2)
chrome://net-internals/#http2
This is the configuration on nginx :
server {
index index.html index.htm index.php;
# Direct http2
listen IP:PORT ssl http2;
# Haproxy redirection
listen IP:PORT http2 proxy_protocol;
...
And this one is the configuration on haproxy ( My idea is to use SSL layer between HA and Nginx so could be in different servers around the world).
frontend ABC
mode tcp
# Works with http11
# bind 0.0.0.0:PORT ssl crt /etc/haproxy/certs/DOMAIN.pem alpn http/1.1
# Works with http2 and http11
bind 0.0.0.0:PORT ssl crt /etc/haproxy/certs/DOMAIN.pem alpn h2,http/1.1 ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA
reqadd X-Forwarded-Proto:\ https
use_backend proxy_nginx if { ssl_fc_alpn -i h2 }
default_backend nginx
backend nginx
mode tcp
server server_nginx IP:PORT check ssl verify none
backend proxy_nginx
mode tcp
server proxy_nginx IP:PORT check send-proxy
...
Now everybody could have working Haproxy, Nginx with libressl 2.5.3 using http2 and alpn.
Interesting URLs to check :
https://gist.github.com/thisismitch/7c91e9b2b63f837a0c4b
https://www.mare-system.de/blog/page/1405201517/
https://jve.linuxwall.info/blog/index.php?post/2015/10/04/SHA1/SHA256-certificate-switching-with-HAProxy
https://wiki.mozilla.org/Security/Server_Side_TLS#Recommended_configurations
Best regards.
Ricardo / Brqx.
Out of curiosity, I spent some time looking through TCP dumps of an https web connection I made. I have been able to make sense of most of it, but I am stuck on one particular TLS Record. Here is the hex dump:
16 03 01 00 24 ae f5 83 cb 35 db dd 67 f5 bf 4a
c7 52 b5 16 56 59 52 40 fa 7b f8 f6 40 a7 13 74
0a f3 b0 6e 5b 4f 2b 88 a3
The previous Record is a Change Cipher Spec Record (i.e. Content Type 0x14) if that helps. Also, I used wget to make the request.
As far as I can tell, this should follow the handshake subprotocol (16), uses TLS 1.0 (03 01), the message length is 36 Bytes (00 24). And here is where I am stuck: what does the ae mean?! At first I thought it might have something to do with SNI or some other TLS extension, but so far no luck there either.
Any help interpreting this would be appreciated.
There is no HandshakeType with a value of 174. The 174 shows up because the TLS connection just finished negotiating a cipher suite, and is now encrypting the record's payload!
I'm trying to implement certificate signature verification on a Microchip pic controller (certificates are generated and signed using OpenSSL). The Microchip PIC controller doesn't support OpenSSL libraries, but it does have an encryption/decryption function. I was successful in getting a SSL connection between PIC controller and a web server. My next step is to setup signature verification on the PIC controller.
After reading PKCS#1 V2.1 RSA Cryptography Standard (http://www.rsa.com/rsalabs/node.asp?id=2125)
I realized that encryption is essentially the same as signature verification and decryption is the same as signing. More specifically both encryption and verification uses the public key and the following formula:
m = s ^ e mod n
Where s is the signature or the message, e is the public exponent, n is the modulus and m is the encrypted message or decoded signature. Therefore, I'm trying to use the encryption algorithm provided to perform signature verification.
In order to verify the certificate, I generated the SHA1 hash of the certificate; Decoded signature using CA's public key and encryption algorithm. Remove the padding from the decoded signature, the result hash should be equal to the SHA1 hash of the certificate.
However, I cannot get the two hash values to be equal. I tried to verify my assumption and PIC controller results using OpenSSL command line.
This is the hash value I got from both OpenSSL command line and PIC controller
openssl rsautl -in signature.txt -verify -asn1parse -inkey pubkey.pem
-pubin
db e8 c6 cb 78 19 3c 0f-fd 96 1c 4f ed bd b2 34 45 60 bf 65
This is what I got from Signature verification using OpenSSL. After removing "ff" paddings I'll end up with asn1 format of the certificate hash.
openssl rsautl -verify -in signature.txt -inkey pubkey.pem -pubin
-raw -hexdump
00 01 ff ff ff ff ff ff-ff ff ff ff 00 30 21 30
09 06 05 2b 0e 03 02 1a-05 00 04 14 db e8 c6 cb
78 19 3c 0f fd 96 1c 4f-ed bd b2 34 45 60 bf 65
However this is what I got from the PIC controller which is much different from the above
8e fb 62 0e 09 c8 0b 49 40 1f 4d 2d a7 7d d6 8c
9b bc 95 e6 bc 98 4b 96 aa 74 e5 68 90 40 bf 43
b5 c5 02 6d ab e3 ad 7b e6 98 fd 10 22 af b9 fb
This is my signature
7951 9b3d 244a 37f6 86d7 dc02 dc18 3bb4
0f66 db3a a3c1 a254 5be5 11d3 a691 63ef
0cf2 ec59 c48b 25ad 8881 9ed2 5230 bcd6
This is my public key (I'm using a very small key just for testing, will make it larger once everything works)
96 FE CB 59 37 AE 8C 9C 6C 7A 01 50 0F D6 4F B4
E2 EC 45 D1 88 4E 1F 2D B7 1E 4B AD 76 4D 1F F1
B0 CD 09 6F E5 B7 43 CA F8 14 FE 31 B2 06 F8 7B
Exponent is 01 00 01
I'm wondering are my assumptions wrong that I cannot use encryption algorithm for decoding signature? or I'm doing something else wrong?
It turned out the method I described above is correct. I was able to get the matching result from hashing the certificate and unsigning the signature using encryption.
The problem that caused my previous failing attempts was the endianess used by Microchip Pic controller. They use small-endian instead of big-endian. I did not pay attention to the endianness of the exponent since 01 00 01 is the same in either format. However I was wrong, it turns out Microchip looks at a 4 byte value as the exponent (RSA standard??). So it pads 00 in the front resulting 00 01 00 01. Therefore, the endianness matters now since 00 01 00 01 is different from 01 00 01 00. And 01 00 01 00 is the small-endian format that Microchip Pic uses.