We created a .NET 6 gRPC server that uses the Kestrel web server and a server certificate for HTTPS. We also created a .NET 6 gRPC client that connects to the aforementioned server and supplies a client certificate for authentication.
We successfully tested both the client and the server on Windows 10 and Windows Server 2016. We're now deploying the client and server to Windows Server 2012 R2 servers and are running into TLS 1.2 issues.
Using Wireshark, we can see the client (running on Server 2012 R2) send a TLSv1.2 "Client Hello" to begin the handshake. The server (running on a different Server 2012 R2 server) immediately responds with a TLSv1.2 fatal alert that has a description of "Protocol Version (70)". According to this article from Microsoft, that alert means "The protocol version the client attempted to negotiate is recognized, but not supported. For example, old protocol versions might be avoided for security reasons."
We have explicitly disabled via registry TLS 1.0 and 1.1, but not TLS 1.2. According to this article from Microsoft, TLS 1.2 is enabled by default on Windows Server 2012 R2. I even confirmed that neither of the following registry keys are present, which could disable TLS 1.2 if specified and set to 1 and 0 respectively:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Client\DisabledByDefault
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Client\Enabled
That leads me to believe that the cipher suites could be the problem. This is where my very limited understanding of TLS tapers off. I can see in the Wireshark capture that these are the cipher suites passed from the client:
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (0xc028)
Cipher Suite: TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 (0x009f)
Cipher Suite: TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (0x009e)
Cipher Suite: TLS_RSA_WITH_AES_256_GCM_SHA384 (0x009d)
Cipher Suite: TLS_RSA_WITH_AES_128_GCM_SHA256 (0x009c)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (0xc02b)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 (0xc023)
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (0xc027)
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)
Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA256 (0x003d)
Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA256 (0x003c)
Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA (0x0035)
Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA (0x002f)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 (0xc02c)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 (0xc024)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (0xc00a)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA (0xc009)
Cipher Suite: TLS_DHE_DSS_WITH_AES_256_CBC_SHA256 (0x006a)
Cipher Suite: TLS_DHE_DSS_WITH_AES_128_CBC_SHA256 (0x0040)
Cipher Suite: TLS_DHE_DSS_WITH_AES_256_CBC_SHA (0x0038)
Cipher Suite: TLS_DHE_DSS_WITH_AES_128_CBC_SHA (0x0032)
On the server, in gpedit.msc > Computer Configuration > Administrative Templates > Network > SSL Configuration Settings, I can see that the "SSL Cipher Suite Order" setting state is not configured, so it should use the factory default cipher suite order. According to the description of the setting, the default order should be the following:
TLS 1.2 SHA256 and SHA384 cipher suites:
TLS_RSA_WITH_AES_128_CBC_SHA256
TLS_RSA_WITH_AES_256_CBC_SHA256
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P384
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P521
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P521
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P256
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P384
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P521
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384_P384
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384_P521
TLS_DHE_DSS_WITH_AES_128_CBC_SHA256
TLS_DHE_DSS_WITH_AES_256_CBC_SHA256
TLS_RSA_WITH_NULL_SHA256
TLS 1.2 ECC GCM cipher suites:
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P256
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P384
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P521
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384_P384
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384_P521
There are four SHA256 cipher suites that exist in both the client and the server (TLS_RSA_WITH_AES_256_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA256, and TLS_DHE_DSS_WITH_AES_128_CBC_SHA256), so I believe that means the TLS negotiation should find a match and use it.
That leaves me stumped as to what else could be the problem. Any suggestions would be greatly appreciated.
My suspicion about cipher suites was correct. Per this article:
Kestrel has limited support for HTTP/2 on Windows Server 2012 R2 and Windows 8.1. Support is limited because the list of supported TLS cipher suites available on these operating systems is limited. A certificate generated using an Elliptic Curve Digital Signature Algorithm (ECDSA) may be required to secure TLS connections.
Related
I have a very old shell script running on a production machine i just got access to from my customer and my job is to convert it to a curl equivalent. The script is pretty simple and all it does is downloading a file from remote FTP to a local filesystem:
lftp -u Username,'pass' xxx.xxx.xx.xx << !
echo 'Connected'
get dir/file.csv
exit
!
First of all - is that even possible to replace that with curl? It looks like a simple FTP fetching script to me but i might be not aware of any nuances of downloading FTP files using curl.
Second of all, here is what i tried so far based on dozen of threads i've found on the internet and none of these worked:
curl ftp://Username:pass#xxx.xxx.xx.xx/dir/file.csv --ftp-ssl
#=> curl: (67) Access denied: 550
curl ftps://Username:pass#xxx.xxx.xx.xx/dir/file.csv --ftp-ssl
#=> curl: (67) Access denied: 550
curl -P - --insecure "ftp://xxx.xxx.xx.xx/dir/file.csv" --user "Username:pass" --ftp-ssl
#=> curl: (67) Access denied: 550
Edit: after adding -v, i realized that there was some issue with the certificate so i added --insecure flag and it now tells that login was incorrect while I'm 100% sure both login and passwords are correct. Output:
* Trying xxx.xxx.xx.xx...
* TCP_NODELAY set
* Connected to xxx.xxx.xx.xx (xxx.xxx.xx.xx) port 21 (#0)
< 220 NASFTPD Turbo station 1.3.5a Server (ProFTPD)
> AUTH SSL
< 234 AUTH SSL successful
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* Server certificate:
* subject: C=TW; ST=xxx; L=xxx; O=xxx, Inc.; OU=xxx; CN=xxx; emailAddress=xxx
* start date: Mar 11 10:45:27 2016 GMT
* expire date: Mar 9 10:45:27 2026 GMT
* issuer: C=TW; ST=xxx; L=Taipei; O=xxx, Inc.; OU=QTS; CN=xxx; emailAddress=xxx
* SSL certificate verify result: self signed certificate (18), continuing anyway.
> USER Username
< 331 Password required for Username
> PASS s
< 530 Login incorrect.
* Access denied: 530
* Closing connection 0
* TLSv1.2 (OUT), TLS alert, Client hello (1):
curl: (67) Access denied: 530
The issue was caused by the fact that password contained a $ character which made password parsing misbehaving and was using just a first letter of a password (next one was a dollar sign). Wrapping password inside single quotation marks solved the issue for me.
I use Delphi 2010, newest Indy 10.6.2 (5498) and OpenSSL 1.0.2r (from indy.fulgan.com) and I am not able to force it to use TLS 1.2. When I open a page mentioned later, it returns EIdOSSLUnderlyingCryptoError:
Error connecting with SSL. error:1408F10B:SSL
routines:SSL3_GET_RECORD:wrong version number
Here is the code I use to open web
var
IdHTTP: TIdHTTP;
begin
IdHTTP:=TIdHTTP.Create;
try
IdHTTP.IOHandler := TIdSSLIOHandlerSocketOpenSSL.Create(IdHTTP);
TIdSSLIOHandlerSocketOpenSSL(IdHTTP.IOHandler).SSLOptions.Method := sslvTLSv1_2;
TIdSSLIOHandlerSocketOpenSSL(IdHTTP.IOHandler).SSLOptions.SSLVersions := [sslvTLSv1_2];
IdHTTP.Get('https://...');
finally
IdHTTP.Free;
end;
end;
When I open https:// www.google.com, then it correctly uses TLS 1.2, here are details from Wireshark
Secure Sockets Layer
TLSv1.2 Record Layer: Handshake Protocol: Client Hello
Content Type: Handshake (22)
Version: TLS 1.0 (0x0301)
Length: 512
Handshake Protocol: Client Hello
Handshake Type: Client Hello (1)
Length: 508
Version: TLS 1.2 (0x0303)
But when I open this page https:// www.downloadtb.com, Indy uses TLS 1.0. The site uses TLS 1.2 only and this cipher: ECDHE-RSA-AES256-GCM-SHA384.
Secure Sockets Layer
TLSv1 Record Layer: Handshake Protocol: Client Hello
Content Type: Handshake (22)
Version: TLS 1.0 (0x0301)
Length: 329
Handshake Protocol: Client Hello
Handshake Type: Client Hello (1)
Length: 325
Version: TLS 1.2 (0x0303)
It seems that it mixes TLS 1.0 and 1.2. Is this some kind of fallback? How do I fix it?
EDIT: Something changed on the site and Indy now correctly uses TLS 1.2
Secure Sockets Layer
TLSv1.2 Record Layer: Handshake Protocol: Client Hello
Content Type: Handshake (22)
Version: TLS 1.0 (0x0301)
Length: 512
Handshake Protocol: Client Hello
Handshake Type: Client Hello (1)
Length: 508
Version: TLS 1.2 (0x0303)
I'm a bit at a loss on how to further investigate this, so pointers would be highly appreciated.
I'm running Ubuntu 17.04, and I believe roughly since around my upgrade time (was running 16.10 before) I can no longer update (or use anything "from the internet") anything from within R -- it fails on SSL for everything. All of the "normal" SSL traffic outside of R works fine.
For instance, doing install.packages("curl"), I get this error message:
Warning in install.packages :
URL 'https://cran.rstudio.com/src/contrib/PACKAGES.rds': status was 'SSL connect error'
Warning in install.packages :
URL 'https://cran.rstudio.com/src/contrib/PACKAGES.gz': status was 'SSL connect error'
Warning in install.packages :
URL 'https://cran.rstudio.com/src/contrib/PACKAGES': status was 'SSL connect error'
Warning in install.packages :
... [etc] ...
However, if I run curl -v "https://cran.rstudio.com/src/contrib/PACKAGES.rds" -o test.curl on command line, everything works.
* Trying 10.26.0.19...
* TCP_NODELAY set
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to (nil) (10.26.0.19) port 3128 (#0)
* Establish HTTP proxy tunnel to cran.rstudio.com:443
* Proxy auth using Basic with user '[redacted]'
> CONNECT cran.rstudio.com:443 HTTP/1.1
> Host: cran.rstudio.com:443
> Proxy-Authorization: Basic [redacted]
> User-Agent: curl/7.52.1
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection established
<
* Proxy replied OK to CONNECT request
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:#STRENGTH
* successfully set certificate verify locations:
* CAfile: /home/csafferling/programs/anaconda3/ssl/cacert.pem
CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):} [5 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client hello (1):} [512 bytes data]
* TLSv1.2 (IN), TLS handshake, Server hello (2):{ [76 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):{ [4787 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):{ [333 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):} [70 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):} [16 bytes data]
* TLSv1.2 (IN), TLS change cipher, Client hello (1):{ [1 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use http/1.1
* Server certificate:
* subject: OU=Domain Control Validated; CN=cran.rstudio.com
* start date: Jun 30 19:59:41 2015 GMT
* expire date: Jun 30 19:59:41 2018 GMT
* subjectAltName: host "cran.rstudio.com" matched cert's "cran.rstudio.com"
* issuer: C=US; ST=Arizona; L=Scottsdale; O=GoDaddy.com, Inc.; OU=http://certs.godaddy.com/repository/; CN=Go Daddy Secure Certificate Authority - G2
* SSL certificate verify ok.} [5 bytes data]
> GET /src/contrib/PACKAGES.rds HTTP/1.1
> Host: cran.rstudio.com
> User-Agent: curl/7.52.1
> Accept: */*
> { [5 bytes data]
< HTTP/1.1 200 OK
< Content-Length: 251020
< Connection: keep-alive
< Date: Wed, 12 Jul 2017 14:11:48 GMT
< Server: Apache/2.2.22 (Ubuntu)
< Last-Modified: Wed, 12 Jul 2017 13:02:43 GMT
< ETag: "d78fc54-3d48c-5541e6e7d22c0"
< Accept-Ranges: bytes
< Cache-Control: max-age=1800
< Expires: Wed, 12 Jul 2017 14:41:48 GMT
< Age: 1045
< X-Cache: Hit from cloudfront
< Via: 1.1 67284fcf464f6f1529cc1e521669622c.cloudfront.net (CloudFront)
< X-Amz-Cf-Id: CqpfjeemEcxkxFYJueqzwUEu8Yh-qSenHJJiR2BdmqmAYLpu2_54dA==
< { [15891 bytes data]
* Curl_http_done: called premature == 0 100 245k 100 245k 0 0 583k 0 --:--:-- --:--:-- --:--:-- 589k
* Connection #0 to host (nil) left intact
One thing I notice is that command-line curl uses the CAs of my anaconda install, which is very weird indeed. Perhaps R uses the default CAs, and they don't work? Like I said, only R fails to work with SSL, everything else works.
Any help is highly appreciated!
Dear Christoph Saffering,
My sense is that you have hit the CRAN ssh by default issue with RStudio / R.
Solution
Add the following to your target machines .Rprofile
options(download.file.method = "wget")
local({
r<- getOption("repos");
r["CRAN"] <-"https://cran.rstudio.com/"
options(repos=r)
})
Explanation
Secure Download Methods
When R transfers files over HTTP (e.g. using the install.packages or download.file function) a download method is chosen based on the download.file.method option. There are several methods available and the default behavior if no option is explicitly specified is to use R’s internal HTTP implementation. In many circumstances this internal method will not support HTTPS connections so you’ll need to override the default.
R 3.2
R 3.2 includes two new download methods (“libcurl” and “wininet”) that both support HTTPS connections. We recommend that you use these new methods when running under R 3.2. The requisite code to add to .Rprofile or Rprofile.site is as follows:
Windows
options(download.file.method = "wininet")
Note that in the upcoming R 3.2.2 release this will no longer be necessary, as the default method is equivalent to “wininet”.
OS X and Linux
options(download.file.method = "libcurl")
Note that if you built R from source the “libcurl” method may or may not have been compiled in. In the case that it wasn’t (i.e. capabilities("libcurl") == FALSE), you can follow the directions for earlier versions of R below to configure an alternate secure method.
R 3.1 and Earlier
Windows
utils::setInternet2(TRUE)
options(download.file.method = "internal")
Note that setInternet2(TRUE) is the default value in RStudio however is not for R GUI. If you don’t want to use setInternet2(TRUE) on Windows then the only other way to configure secure downloads is to have the “wget” or “curl” utility on your PATH as described for OS X and Linux below.
OS X
options(download.file.method = "curl")
Linux
options(download.file.method = "wget")
Note that the “curl” and “wget” methods will work on any platform so long as the requisite binary is in the system PATH. The recommendations above are based on the fact that “curl” is included in OS X and “wget” is included in most Linux distributions.
ref: https://support.rstudio.com/hc/en-us/articles/206827897-Secure-Package-Downloads-for-R
Right now I use curl -I to retrieve headers.
Will sites adopt a different way of serving headers with HPACK in the upcoming adoption of HTTP/2 by browsers that will render my use of the curl command ineffective?
Yes, you can use curl to see and send HTTP headers with HTTP/2 just as you do with HTTP/1.
curl supports HTTP/2 and it is implemented as a sort of translation layer. It means it shows and "pretends" that headers work 1.1 style. It shows headers as text and it sends headers in callbacks like they were done with 1.1. We made it this way to make scripts and applications get a very smooth and basically invisible transition path to HTTP/2 with curl.
Internally that is of course done by decompressing received headers before showing them, and showing them before compressing them when sending them.
I believe it depends on curl version. HTTP/2 was added in curl 7.36.x IIRC ? not all distros would have that version ?
This is with curl 7.41.0 over HTTP/2 against https://google.com
curl --http2 -I -v https://google.com
* Rebuilt URL to: https://google.com/
* Trying 173.194.123.1...
* Connected to google.com (173.194.123.1) port 443 (#0)
* ALPN, offering h2-14, http/1.1
* ALPN, server accepted to use h2-14
* Server certificate:
* subject: C=US; ST=California; L=Mountain View; O=Google Inc; CN=*.google.com
* start date: 2015-03-11 16:13:43 GMT
* expire date: 2015-06-09 00:00:00 GMT
* subjectAltName: google.com matched
* issuer: C=US; O=Google Inc; CN=Google Internet Authority G2
* SSL certificate verify ok.
* Using HTTP2
edit: correction, curl --http2 needs nghttp2 compiled for it to work https://nghttp2.org/
curl --version
curl 7.41.0 (x86_64-unknown-linux-gnu) libcurl/7.41.0 OpenSSL/1.0.2b zlib/1.2.8 nghttp2/0.7.8-DEV
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets
We have an Windows 2008 server with IIS 7 to test sites we develop for our clients. Each site has a binding on a subdomain:
clienta.example.com
clientb.example.com
clientc.example.com
(* Using example.com to protect the innocent)
For one of these sites we now have to test if it works over https.
So I have created a certificate request with *.example.com as the common name. I have received the certificate (issued by PositiveSSL SA) and completed the request. The certificate is now installed in IIS.
Now I have added an https binding to the second site with the following settings:
type: https
IP address: All Unassigned
Port: 443
Host name: clientb.example.com
SSL certificate: *.example.com
Browsing the site over regular http works fine. When I try to browse the site over https I get the following errors (depending on the browser used):
Chrome
This webpage is not available
Error 102 (net::ERR_CONNECTION_REFUSED): Unknown error.
Firefox
Unable to connect
Firefox can't establish a connection to the server at clientb.example.com
Firebug says Status: Aborted
Internet Explorer
Internet Explorer cannot display the webpage
I have checked Failed Request Tracing, and according to the log the request was completed with status 200.
I have run the SSL Diagnostics Tool with the following result:
System time: Fri, 04 Mar 2011 14:04:35 GMT
Connecting to 192.168.2.95:443
Connected
Handshake: 115 bytes sent
Handshake: 3877 bytes received
Handshake: 326 bytes sent
Handshake: 59 bytes received
Handshake succeeded
Verifying server certificate, it might take a while...
Server certificate name: *.example.com
Server certificate subject: OU=Domain Control Validated, OU=PositiveSSL Wildcard, CN=*.example.com
Server certificate issuer: C=GB, S=Greater Manchester, L=Salford, O=Comodo CA Limited, CN=PositiveSSL CA
Server certificate validity: From 2-3-2011 1:00:00 To 2-3-2012 0:59:59
1:00:00 To 2-3-2012 0:59:59
HTTPS request:
GET / HTTP/1.0
User-Agent: SSLDiag
Accept:*/*
HTTPS: 85 bytes of encrypted data sent
HTTPS: 533 bytes of encrypted data received
Status:
HTTP/1.1 404 Not Found
HTTP/1.1 404 Not Found
Content-Type: text/html; charset=us-ascii
Server: Microsoft-HTTPAPI/2.0
Date: Fri, 04 Mar 2011 14:04:35 GMT
Connection: close
Content-Length: 315
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Not Found</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Not Found</h2>
<hr><p>HTTP Error 404. The requested resource is not found.</p>
</BODY></HTML>
HTTPS: server disconnected
Final handshake: 37 bytes sent successfully
Q: What can I do to make this work?
For the IP-binding to work when combined with a host header it's a good idea to first make sure the firewall is forwarding https requests to the webserver (TCP/443).
Doh.