Requests ignoring setting OP_NO_SSLv3 - python-requests

i'm trying to get a domain on a private server, but requests get continuously fails with sslv3 alert handshake failure.
I have configured the session to ignore sslv3 according to the blog post here https://lukasa.co.uk/2017/02/Configuring_TLS_With_Requests/
but it continues using it as statet in the exception message.
Any ideas what i am making wrong here?
Heres my minimal script:
import ssl
import requests
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.ssl_ import create_urllib3_context
CIPHERS = (
'ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+HIGH:'
'DH+HIGH:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+HIGH:RSA+3DES:!aNULL:'
'!eNULL:!MD5'
)
class DESAdapter(HTTPAdapter):
"""
A TransportAdapter that re-enables 3DES support in Requests.
"""
def create_ssl_context(self):
ctx = ssl.create_default_context()
# opt out the SSLv3
ctx.options |= ssl.OP_NO_SSLv3
ctx.set_ciphers( CIPHERS )
return ctx
def init_poolmanager(self, *args, **kwargs):
kwargs['ssl_context'] = self.create_ssl_context()
return super(DESAdapter, self).init_poolmanager(*args, **kwargs)
def proxy_manager_for(self, *args, **kwargs):
kwargs['ssl_context'] = self.create_ssl_context()
return super(DESAdapter, self).proxy_manager_for(*args, **kwargs)
s = requests.Session()
s.mount('https://my-broken-intranet-domain.net:4942', DESAdapter())
# This will throw the error:
r = s.get('https://my-broken-intranet-domain.net:4942')
Exception has occurred: requests.exceptions.SSLError
HTTPSConnectionPool(host='my-broken-intranet-domain.net', port=4942): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1056)')))
With Curl, the request goes trough and the negotiated CIPHER and TLS are:
curl -v https://my-broken-intranet-domain.net:4942
...
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / DES-CBC3-SHA
* ALPN, server did not agree to a protocol
Versions:
Python 3.7.2
requests
Version: 2.21.0
urllib3
Version: 1.24.1
OpenSSL
1.1.0j
Thanks for the help!


Related to Python 3.7.2.
Closing this, as it is not present with 3.6.8

Related

NGINX drops SSE connections every minute or so

got a express based SSE server running behind an NGINX proxy. It keeps dropping connections every minute. Here is a curl session with much detail. Please help to diagnose.
curl -N -verbose --http2 -H "Accept:text/event-stream" https://umx.safuyi.com/api/rooms/960/events
* Trying 49.235.227.189:443...
* Connected to umx.safuyi.com (49.235.227.189) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
* CApath: none
* (304) (OUT), TLS handshake, Client hello (1):
* (304) (IN), TLS handshake, Server hello (2):
* (304) (IN), TLS handshake, Unknown (8):
* (304) (IN), TLS handshake, Certificate (11):
* (304) (IN), TLS handshake, CERT verify (15):
* (304) (IN), TLS handshake, Finished (20):
* (304) (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / AEAD-AES256-GCM-SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
* subject: CN=umx.safuyi.com
* start date: Jan 5 00:00:00 2023 GMT
* expire date: Jan 5 23:59:59 2024 GMT
* subjectAltName: host "umx.safuyi.com" matched cert's "umx.safuyi.com"
* issuer: C=CN; O=TrustAsia Technologies, Inc.; CN=TrustAsia RSA DV TLS CA G2
* SSL certificate verify ok.
> GET /api/rooms/960/events HTTP/1.1
> Host: umx.safuyi.com
> User-Agent: curl/7.79.1
> Referer: rbose
> Accept:text/event-stream
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Server: nginx/1.14.0 (Ubuntu)
< Date: Sun, 29 Jan 2023 12:31:33 GMT
< Content-Type: text/event-stream
< Transfer-Encoding: chunked
< Connection: keep-alive
< X-Powered-By: Express
< Access-Control-Allow-Origin: *
< Cache-Control: no-cache
<
data: [{"who":"0.36735210198439616/2023-01-29T12:30:25.938Z","did":"left room","room":"960"},{"who":"0.6205505230339852/2023-01-29T12:30:23.668Z","did":"left room","room":"960"},{"who":"0.10731778838068662/2023-01-29T12:31:26.900Z","did":"entered room","room":"960"}]
event: clientId
data: {"clientId": "0.9871521861164745/2023-01-29T12:31:33.839Z"}
* transfer closed with outstanding read data remaining
* Closing connection 0
I put some headers in the response to disable buffering on the NGINX part:
Content-Type: text/event-stream;
Cache-Control: no-cache;
X-Accel-Buffering: no;
did not seems to work

Try changing proxy_read_timeout by default it is 60 seconds.
Source

Nginx: peer closed connection in SSL handshake while SSL handshaking to upstream

I have this new client which only can talk TLSv1.2 which communicate with a old service which can only talk TLSv1. It is about to be upgraded. It is an internal system without access to internet.
To solve the problem I have placed a Nginx proxy in between.
client =TLS1.2=> Nginx =TLSv1=> oldservice
Even with proxy_ssl_verify=off I get this error:
==> /var/log/nginx/ngx-error.log <==
2021/09/15 14:48:26 [error] 13231#0: *3 peer closed connection in SSL handshake while SSL handshaking to upstream, client: 10.140.164.9, server: ngx.example.org, request: "GET /net/EXT/microservice.svc HTTP/1.1", upstream: "https://172.23.149.10:443/EXT/microservice.svc", host: "ngx.example.org"
If I on ngx.example.org run curl -v https://endp.example.org/EXT/microservice.svc I get this message:
* NSS error -8016 (SEC_ERROR_CERT_SIGNATURE_ALGORITHM_DISABLED)
* The certificate was signed using a signature algorithm that is disabled because it is not secure.
* Closing connection 0
curl: (60) The certificate was signed using a signature algorithm that is disabled because it is not secure.
If I run curl --insecure it is working and this is the behavior I want Nginx to do.
My Nginx configuration:
server {
listen 80;
listen 443 ssl;
server_name ngx.example.org;
access_log /var/log/nginx/ngx-access.log;
error_log /var/log/nginx/ngx-error.log;
root /var/www/nginx/ngx;
ssl_protocols TLSv1.2;
ssl_certificate /etc/pki/nginx/private/ngx.example.org.crt;
ssl_certificate_key /etc/pki/nginx/private/ngx.example.org.key;
ssl_verify_client off;
proxy_ssl_server_name on;
proxy_ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
#proxy_ssl_ciphers PSK-3DES-EDE-CBC-SHA;
location /net/ {
proxy_ssl_name endp.example.org;
proxy_ssl_server_name on;
proxy_ssl_verify off;
proxy_ssl_trusted_certificate /etc/pki/ca-trust/source/anchors/wsgw.root.ca.pem;
proxy_set_header Host endp.example.org;
proxy_set_header X_FORWARDED_PROTO https;
proxy_pass https://endp.example.org/;
}
}
Edit:
Here is the output from openssl without -tlsv1:
openssl s_client -connect endp.example.org:443
139801673865104:error:140790E5:SSL routines:ssl23_write:ssl handshake failure:s23_lib.c:177:
CONNECTED(00000003)
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 289 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : 0000
Session-ID:
Session-ID-ctx:
Master-Key:
Key-Arg : None
Krb5 Principal: None
PSK identity: None
PSK identity hint: None
Start Time: 1631880896
Timeout : 300 (sec)
Verify return code: 0 (ok)
Here is the output from openssl with -tlsv1:
openssl s_client -connect endp.example.org:443 -tls1
depth=0 CN = endp.example.org
verify error:num=18:self signed certificate
verify return:1
depth=0 CN = endp.example.org
verify return:1
CONNECTED(00000003)
---
Certificate chain
0 s:/CN=endp.example.org
i:/CN=endp.example.org
---
Server certificate
-----BEGIN CERTIFICATE-----
MIICIzCCAYygAwIBAgIQZs9VZsp/iZRC2rL+oYTf0DANBgkqhkiG9w0BAQQFADAc
...lines deleted
OS+ih0tnXEzJKqtSC52Z+zAIFWdEipA=
-----END CERTIFICATE-----
subject=/CN=endp.example.org
issuer=/CN=endp.example.org
---
No client certificate CA names sent
---
SSL handshake has read 694 bytes and written 363 bytes
---
New, TLSv1/SSLv3, Cipher is RC4-MD5
Server public key is 1024 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1
Cipher : RC4-MD5
Session-ID: A01E00002B40C49CA7D7CD8EE73A9FB8DF44BF155300CDFCC98712657F697F88
Session-ID-ctx:
Master-Key: A90284275AA0067A47836A269592213E419F5286E0D30EA38F1094B50536F67913FCE7BA5D43025D1AC7FBDCA769FBE4
Key-Arg : None
Krb5 Principal: None
PSK identity: None
PSK identity hint: None
Start Time: 1631862396
Timeout : 7200 (sec)
Verify return code: 18 (self signed certificate)
Content of certificate:
$ openssl x509 -in endp.example.org.crt -text -noout
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
66:cf:55:66:ca:7f:89:94:42:da:b2:fe:a1:84:df:d0
Signature Algorithm: md5WithRSAEncryption
Issuer: CN=endp.example.org
Validity
Not Before: Nov 9 11:08:50 2010 GMT
Not After : Dec 30 23:00:00 2030 GMT
Subject: CN=endp.example.org
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (1024 bit)
Modulus:
00:eb:ea:04:f0:53:87:26:b5:19:13:c6:a5:7b:27:
75:7e:f5:9a:84:74:59:d3:ec:0d:40:42:78:a4:c6:
1b:42:b2:50:19:3e:90:7b:20:73:f4:71:b8:3b:c3:
b3:dd:84:94:78:64:64:ac:5c:dc:a3:e4:8c:36:10:
32:ec:12:90:89:6c:e9:44:0e:fb:2e:84:0f:df:16:
c2:ae:b8:2c:d6:d0:73:b3:cf:4b:f8:56:91:e6:30:
c1:15:34:9f:70:6d:e1:e9:33:de:d6:9b:4d:2e:c8:
03:7b:eb:ed:6b:9c:8e:0b:80:a2:ef:29:5f:18:4e:
bf:e3:9a:81:e3:57:ae:c5:3f
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Extended Key Usage:
TLS Web Server Authentication
2.5.29.1:
0D....M..w....T..."...0.1.0...U....endp.example.org..f.Uf....B.......
Signature Algorithm: md5WithRSAEncryption
83:a7:c9:cf:a5:d3:91:78:65:d0:43:24:84:c5:77:62:be:ba:
52:db:7f:c6:ca:59:40:50:91:5f:48:fe:77:4a:94:26:36:23:
3a:82:6b:54:46:c1:a4:0d:bd:8f:96:bc:04:c8:54:f7:74:94:
83:3a:9e:71:61:8c:d4:a0:77:be:fc:50:e8:3f:12:a3:00:01:
9d:d6:06:a0:77:c2:84:24:96:03:c1:6a:05:57:bb:5e:d0:47:
e5:ff:a9:6c:a1:e1:cc:a4:d0:4b:b0:9d:1b:0b:d4:39:2f:a2:
87:4b:67:5c:4c:c9:2a:ab:52:0b:9d:99:fb:30:08:15:67:44:
8a:90

Not a solution but a work-around: Use Apache
ServerName ngx.example.org:443
SSLEngine on
SSLProxyEngine on
SSLProxyVerifyDepth 0
SSLProtocol TLSv1.2
SSLProxyProtocol TLSv1
SSLCACertificateFile /etc/pki/tls/certs/ca-bundle.crt
SSLCertificateFile /etc/pki/nginx/private/ngx.example.org.crt
SSLCertificateKeyFile /etc/pki/nginx/private/ngx.example.org.key
DocumentRoot /var/www/httpd/ngx
CustomLog /var/log/httpd/ngx/access.log combined
ErrorLog /var/log/httpd/ngx/error.log
ProxyPass "/net/" "https://endp.example.org/"

Determining the physical host of a website using dig and curl

This is based on a question asked in the book "Computer Networking: Principles, Protocols and Practice" by Olivier Bonaventure. I've read the man pages of both dig and curl on my Linux terminal, but I can't seem to understand how it will help in finding out the physical host of a given website. Do we use both statements separately? Or do we pipe them into one statement?

I'm pretty sure what is meant here is finding out what IP is being used.
You would use dig to get the A record:
~ → dig www.info.ucl.ac.be
; <<>> DiG 9.10.6 <<>> www.info.ucl.ac.be
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 58340
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;www.info.ucl.ac.be. IN A
;; ANSWER SECTION:
www.info.ucl.ac.be. 7200 IN A 130.104.228.160
;; Query time: 459 msec
;; SERVER: 2001:558:feed::1#53(2001:558:feed::1)
;; WHEN: Wed Oct 06 12:27:23 PDT 2021
;; MSG SIZE rcvd: 63
That would give you the physical host of: 130.104.228.160
You can do the same with curl:
~ → curl -svo /dev/null https://www.info.ucl.ac.be
* Trying 130.104.228.160...
* TCP_NODELAY set
* Connected to www.info.ucl.ac.be (130.104.228.160) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [232 bytes data]
* TLSv1.2 (IN), TLS handshake, Server hello (2):
{ [108 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [6286 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [300 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [37 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
{ [1 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
* subject: C=BE; postalCode=1348; ST=Brabant wallon; L=Louvain-la-Neuve; street=Place de l'Universit�, 1; O=Universit� catholique de Louvain; OU=INGI; CN=www.info.ucl.ac.be
* start date: Jul 13 00:00:00 2020 GMT
* expire date: Jul 13 23:59:59 2022 GMT
* subjectAltName: host "www.info.ucl.ac.be" matched cert's "www.info.ucl.ac.be"
* issuer: C=NL; O=GEANT Vereniging; CN=GEANT OV RSA CA 4
* SSL certificate verify ok.
> GET / HTTP/1.1
> Host: www.info.ucl.ac.be
> User-Agent: curl/7.64.1
> Accept: */*
>
< HTTP/1.1 302 Found
< Date: Wed, 06 Oct 2021 19:27:40 GMT
< Server: Apache/2.4.37 (centos) OpenSSL/1.1.1g
< X-Powered-By: PHP/7.2.24
< Location: http://www.uclouvain.be/ingi.html
< Content-Length: 0
< Content-Type: text/html; charset=UTF-8
<
* Connection #0 to host www.info.ucl.ac.be left intact
* Closing connection 0
Pay special attention to the the output:
Connected to www. info. ucl. ac. be (130.104.228.160) port 443 (#0)

QWebSocket doesn't connect over TLS

I have a WebSocket which works good over WebSocket protocol, but I can not switch to WebSocketSecure protocol, It doesn't generate any errors on server side, client says error:141970DF:SSL routines:tls_construct_cke_psk_preamble:psk identity not found. The certificate was generated by certbot and is used of for https web site on same domain.
Server code:
QSslConfiguration conf = server.sslConfiguration();
QFile * privkey =
new QFile{"/etc/letsencrypt/live/example.com/privkey.pem"};
privkey->open(QFile::ReadOnly);
conf.setCaCertificates(QSslCertificate::fromPath(
"/etc/letsencrypt/live/example.com/fullchain.pem"));
conf.setPrivateKey(QSslKey(privkey));
conf.setProtocol(QSsl::TlsV1_0);
server.setSslConfiguration(conf);
if (server.listen(QHostAddress::Any, 54045)) {
connect(
&server, &QWebSocketServer::newConnection, this,
&Server::onNewConnection);
connect(&server, &QWebSocketServer::closed, this, &Server::closed);
qDebug() << "server started";
}
The client code:
import QtQuick 2.13
import QtWebSockets 1.13
WebSocket {
active: true
url: "wss://example.com:54045"
}
Output of openSSL:
$ openssl s_client -connect example.com:54045
CONNECTED(00000003)
140623606740288:error:14095126:SSL routines:ssl3_read_n:unexpected eof while reading:ssl/record/rec_layer_s3.c:302:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 325 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---

install.packages doesn't work with proxy when using R installed with conda

I am working on a Linux server RHEL6 and I installed anaconda.
I have the following setup
conda-env version : 4.3.13
conda-build version : 2.1.4
python version : 2.7.13.final.0
rpy2 : 2.8.5
I installed rpy2 to use R in python
> R.home()
[1] "/anaconda2/envs/py27CCA/lib/R"
> R.version
version.string R version 3.3.2 (2016-10-31)
I setup my proxy in the following way:
> Sys.getenv("https_proxy")
[1] "https://login:pwd#xxx.net:8080/"
But downloading R packages doesn't work
> options(internet.info = 0)
> install.packages("httr")
* error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol
....
Warning: unable to access index for repository https://stat.ethz.ch/CRAN/src/contrib:
cannot download all files
Warning message:
package 'httr' is not available (for R version 3.3.2)
But If I installed the same standalone R version with the exact same proxy setup it works without any issue
> R.version
version.string R version 3.3.2 (2016-10-31)
> install.packages("httr")
...
** testing if installed package can be loaded
* DONE (httr)
Making 'packages.html' ... done
...
What is creating this issue ? I check the openssl version and I have the same version in the 2 environments!
This link explain the possible reason of such proxy issue link stackoverflow discussion.
I have the same issue and error messages if I do it inside python
>>> from rpy2.robjects.packages import importr
>>> utils = importr('utils')
>>> utils.install_packages('httr')

TL;DR:
Instead of setting https_proxy to...:
https://login:pwd#xxx.net:8080/
...try setting it to:
http://login:pwd#xxx.net:8080/
Also, by doing this, if someone sniffs packets of the initial connection you make with the proxy server, you will be leaking your credentials. Read further to know more.
IMO, this question has nothing to do with Conda. This is a very common mistake which I find quite prevalent on the internet.
The reason why this happens, is because of the confusion lying around the term "HTTPS Proxy".
IIUC, here is what the two environment variables mean:
http_proxy|HTTP_PROXY: The proxy server that you wish to use, for all
your HTTP requests to the outside world.
https_proxy|HTTPS_PROXY: The proxy server that you wish to use, for
all your HTTPS requests to the outside world.
http(s?)://proxy.mydomain.com:3128
^^^^^ ^^^^^ ^^^^
| | |
scheme proxy domain/IP proxy port
Now, ideally, the scheme specified in the value for these environment variables determines the protocol over which the client ought to connect to the proxy server.
Let's look at definition of an HTTPS proxy. Stealing from the man page for curl >= v7.53:
An HTTPS proxy receives all transactions over an SSL/TLS connection.
Once a secure connection with the proxy is established, the user agent
uses the proxy as usual, including sending CONNECT requests to instruct
the proxy to establish a [usually secure] TCP tunnel with an origin
server. HTTPS proxies protect nearly all aspects of user-proxy
communications as opposed to HTTP proxies that receive all requests
(including CONNECT requests) in vulnerable clear text.
With HTTPS proxies, it is possible to have two concurrent _nested_
SSL/TLS sessions: the "outer" one between the user agent and the proxy
and the "inner" one between the user agent and the origin server
(through the proxy). This change adds supports for such nested sessions
as well.
Let's try and see with examples (curl >= v7.53):
Here, I'll use a proxy which does not support client-proxy connection over SSL/TLS.
Make sure no proxy environment variables are set beforehand:
((curl-7_53_1))$ env | grep -i proxy
((curl-7_53_1))$
env: http_proxy, outer_scheme: http, inner_scheme: http
((curl-7_53_1))$ http_proxy="http://proxy.mydomain.com:3128" ./src/curl -s -vvv http://stackoverflow.com -o /dev/null
* Rebuilt URL to: http://stackoverflow.com/
* Trying 10.1.1.7...
* TCP_NODELAY set
* Connected to proxy.mydomain.com (10.1.1.7) port 3128 (#0)
> GET http://stackoverflow.com/ HTTP/1.1
> Host: stackoverflow.com
> User-Agent: curl/7.53.1-DEV
> Accept: */*
> Proxy-Connection: Keep-Alive
>
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Cache-Control: private
< Content-Type: text/html; charset=utf-8
< X-Frame-Options: SAMEORIGIN
< X-Request-Guid: 539728ee-a91d-4964-bc7e-1d21d91a6f1d
< Content-Length: 228257
< Accept-Ranges: bytes
< Date: Thu, 16 Mar 2017 05:19:31 GMT
< X-Served-By: cache-jfk8137-JFK
< X-Cache: MISS
< X-Cache-Hits: 0
< X-Timer: S1489641571.098286,VS0,VE7
< Vary: Fastly-SSL
< X-DNS-Prefetch-Control: off
< Set-Cookie: prov=b2e2dcb8-c5ff-21d9-5712-a0e012573aa6; domain=.stackoverflow.com; expires=Fri, 01-Jan-2055 00:00:00 GMT; path=/; HttpOnly
< X-Cache: MISS from proxy.mydomain.com
< X-Cache-Lookup: MISS from proxy.mydomain.com:3128
< Via: 1.1 varnish, 1.0 proxy.mydomain.com (squid)
* HTTP/1.0 connection set to keep alive!
< Connection: keep-alive
<
{ [2816 bytes data]
* Connection #0 to host proxy.mydomain.com left intact
env: http_proxy, outer_scheme: https, inner_scheme: http
((curl-7_53_1))$ http_proxy="https://proxy.mydomain.com:3128" ./src/curl -s -vvv http://stackoverflow.com -o /dev/null
* Rebuilt URL to: http://stackoverflow.com/
* Trying 10.1.1.7...
* TCP_NODELAY set
* Connected to proxy.mydomain.com (10.1.1.7) port 3128 (#0)
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:#STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
} [5 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol
* Closing connection 0
env: https_proxy, outer_scheme: http, inner_scheme: https
((curl-7_53_1))$ https_proxy="http://proxy.mydomain.com:3128" ./src/curl -s -vvv https://stackoverflow.com -o /dev/null
* Rebuilt URL to: https://stackoverflow.com/
* Trying 10.1.1.7...
* TCP_NODELAY set
* Connected to proxy.mydomain.com (10.1.1.7) port 3128 (#0)
* Establish HTTP proxy tunnel to stackoverflow.com:443
> CONNECT stackoverflow.com:443 HTTP/1.1
> Host: stackoverflow.com:443
> User-Agent: curl/7.53.1-DEV
> Proxy-Connection: Keep-Alive
>
< HTTP/1.0 200 Connection established
<
* Proxy replied OK to CONNECT request
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:#STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
} [5 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.2 (IN), TLS handshake, Server hello (2):
{ [108 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [3044 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [333 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [70 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
{ [1 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use http/1.1
* Server certificate:
* subject: C=US; ST=NY; L=New York; O=Stack Exchange, Inc.; CN=*.stackexchange.com
* start date: May 21 00:00:00 2016 GMT
* expire date: Aug 14 12:00:00 2019 GMT
* subjectAltName: host "stackoverflow.com" matched cert's "stackoverflow.com"
* issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=DigiCert SHA2 High Assurance Server CA
* SSL certificate verify ok.
} [5 bytes data]
> GET / HTTP/1.1
> Host: stackoverflow.com
> User-Agent: curl/7.53.1-DEV
> Accept: */*
>
{ [5 bytes data]
< HTTP/1.1 200 OK
< Cache-Control: private
< Content-Type: text/html; charset=utf-8
< X-Frame-Options: SAMEORIGIN
< X-Request-Guid: 96f8fe3c-058b-479e-8ef2-db6d09f485d3
< Content-Length: 226580
< Accept-Ranges: bytes
< Date: Thu, 16 Mar 2017 05:20:39 GMT
< Via: 1.1 varnish
< Connection: keep-alive
< X-Served-By: cache-jfk8135-JFK
< X-Cache: MISS
< X-Cache-Hits: 0
< X-Timer: S1489641639.425108,VS0,VE9
< Vary: Fastly-SSL
< X-DNS-Prefetch-Control: off
< Set-Cookie: prov=f1a401f1-f1a0-5f09-66ca-9a792543ee82; domain=.stackoverflow.com; expires=Fri, 01-Jan-2055 00:00:00 GMT; path=/; HttpOnly
<
{ [2181 bytes data]
* Connection #0 to host proxy.mydomain.com left intact
env: https_proxy, outer_scheme: https, inner_scheme: https
((curl-7_53_1))$ https_proxy="https://proxy.mydomain.com:3128" ./src/curl -s -vvv https://stackoverflow.com -o /dev/null
* Rebuilt URL to: https://stackoverflow.com/
* Trying 10.1.1.7...
* TCP_NODELAY set
* Connected to proxy.mydomain.com (10.1.1.7) port 3128 (#0)
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:#STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
} [5 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol
* Closing connection 0
Now, I'll show the same outputs for a proxy which supports connection over SSL/TLS. To run a local https proxy, I have installed squid version 4.0.17. I have pointed proxy.mydomain.com to localhost by overriding it in /etc/hosts. And the relevant squid config line is:
https_port 3127 cert=/etc/squid/ssl_cert/myCA.pem
Please note that I am not using any explicitly specified (complicated?) modes right now (sslbump/intercept/accel/tproxy)
I have added the certificate to the trust store too:
sudo cp /etc/squid/ssl_cert/myCA.pem /etc/pki/ca-trust/source/anchors/mySquidCA.pem
sudo update-ca-trust
Now, for the real test:
env: http_proxy, outer_scheme: https, inner_scheme: http
/t/curl-curl-7_53_1 ❯❯❯ http_proxy=https://proxy.mydomain.com:3127 ./src/curl -s -vvv http://google.com -o /dev/null
* Rebuilt URL to: http://google.com/
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to proxy.mydomain.com (127.0.0.1) port 3127 (#0)
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:#STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
} [5 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.2 (IN), TLS handshake, Server hello (2):
{ [86 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [1027 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [262 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
{ [1 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / AES256-GCM-SHA384
* Proxy certificate:
* subject: C=IN; ST=SomeState; L=SomeLocation; O=Default Company Ltd; CN=proxy.mydomain.com; emailAddress=no-reply#gmail.com
* start date: Mar 16 06:43:35 2017 GMT
* expire date: Mar 16 06:43:35 2018 GMT
* common name: proxy.mydomain.com (matched)
* issuer: C=IN; ST=SomeState; L=SomeLocation; O=Default Company Ltd; CN=proxy.mydomain.com; emailAddress=no-reply#gmail.com
* SSL certificate verify ok.
} [5 bytes data]
> GET http://google.com/ HTTP/1.1
> Host: google.com
> User-Agent: curl/7.53.1-DEV
> Accept: */*
> Proxy-Connection: Keep-Alive
>
{ [5 bytes data]
< HTTP/1.1 302 Found
< Cache-Control: private
< Content-Type: text/html; charset=UTF-8
< Location: http://www.google.co.in/?gfe_rd=cr&ei=ejTKWLGzM-Ts8AepwJyQCg
< Content-Length: 261
< Date: Thu, 16 Mar 2017 06:45:14 GMT
< X-Cache: MISS from lenovo
< X-Cache-Lookup: MISS from lenovo:3128
< Via: 1.1 lenovo (squid/4.0.17)
< Connection: keep-alive
<
{ [5 bytes data]
* Connection #0 to host proxy.mydomain.com left intact
env: https_proxy, outer_scheme: https, inner_scheme: https
/t/curl-curl-7_53_1 ❯❯❯ https_proxy=https://proxy.mydomain.com:3127 ./src/curl -s -vvv https://google.com -o /dev/null
* Rebuilt URL to: https://google.com/
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to proxy.mydomain.com (127.0.0.1) port 3127 (#0)
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:#STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
} [5 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.2 (IN), TLS handshake, Server hello (2):
{ [86 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [1027 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [262 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
{ [1 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Proxy certificate:
* subject: C=IN; ST=SomeState; L=SomeLocation; O=Default Company Ltd; CN=proxy.mydomain.com; emailAddress=no-reply#gmail.com
* start date: Mar 16 06:43:35 2017 GMT
* expire date: Mar 16 06:43:35 2018 GMT
* common name: proxy.mydomain.com (matched)
* issuer: C=IN; ST=SomeState; L=SomeLocation; O=Default Company Ltd; CN=proxy.mydomain.com; emailAddress=no-reply#gmail.com
* SSL certificate verify ok.
* Establish HTTP proxy tunnel to google.com:443
} [5 bytes data]
> CONNECT google.com:443 HTTP/1.1
> Host: google.com:443
> User-Agent: curl/7.53.1-DEV
> Proxy-Connection: Keep-Alive
>
{ [5 bytes data]
< HTTP/1.1 200 Connection established
<
* Proxy replied OK to CONNECT request
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:#STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
} [5 bytes data]
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
} [5 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.2 (IN), TLS handshake, Server hello (2):
{ [102 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [3757 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [148 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [70 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
{ [1 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-ECDSA-AES128-GCM-SHA256
* ALPN, server accepted to use http/1.1
* Server certificate:
* subject: C=US; ST=California; L=Mountain View; O=Google Inc; CN=*.google.com
* start date: Mar 9 02:43:31 2017 GMT
* expire date: Jun 1 02:20:00 2017 GMT
* subjectAltName: host "google.com" matched cert's "google.com"
* issuer: C=US; O=Google Inc; CN=Google Internet Authority G2
* SSL certificate verify ok.
} [5 bytes data]
> GET / HTTP/1.1
> Host: google.com
> User-Agent: curl/7.53.1-DEV
> Accept: */*
>
{ [5 bytes data]
< HTTP/1.1 302 Found
< Cache-Control: private
< Content-Type: text/html; charset=UTF-8
< Location: https://www.google.co.in/?gfe_rd=cr&ei=hDTKWJXlMubs8Aek-6WQAg
< Content-Length: 262
< Date: Thu, 16 Mar 2017 06:45:24 GMT
< Alt-Svc: quic=":443"; ma=2592000; v="36,35,34"
<
{ [262 bytes data]
* Connection #0 to host proxy.mydomain.com left intact
As is evident from the outputs, there is an SSL handshake with the proxy server first in both cases.
Now, I'll rant a little.
Many clients (e.g: curl = 7.51.0), don't support SSL/TLS connection with the proxy itself and throw an error of the sort:
$ https_proxy=https://proxy.mydomain.com:3128 curl -vvvv https://google.com
* Rebuilt URL to: https://google.com/
* Unsupported proxy scheme for 'https://proxy.mydomain.com:3128'
* Closing connection -1
curl: (7) Unsupported proxy scheme for 'https://proxy.mydomain.com:3128'
Then, there are clients (e.g. curl=7.47.0), which would just ignore non-supported schemes in the proxy URL and that would mislead people into believing things about what they accomplished. In general, they would never connect to proxy server over SSL/TLS, even if the variable explicitly specifies the scheme as 'https' and fallback to using unencrypted connection with the proxy server.
Then there are other clients (e.g wget v1.18), which would confuse us further:
In the following case, the error message is misleading, because the
scheme can hold the value https:// even for a HTTP request to the
outside world (as shown in the example above, using squid), since we want the connection to the proxy server to be over SSL/TLS.
http_proxy=https://proxy.mydomain.com:3128 wget http://google.com
Error in proxy URL https://proxy.mydomain.com:3128: Must be HTTP.
Not only this, but the confusion increases, when it falls back, make
us believe that it is probably connecting to the proxy server over
SSL/TLS, when actually it is not, and also making us think that
https:// in the scheme should only work when the inner protocol is
also https://
https_proxy=https://proxy.mydomain-research.com:3128 wget https://google.com
--2017-03-16 11:21:06-- https://google.com/
Resolving proxy.mydomain-research.com (proxy.mydomain-research.com)... 10.1.1.7
Connecting to proxy.mydomain-research.com (proxy.mydomain-research.com)|10.1.1.7|:3128... connected.
Proxy request sent, awaiting response... 301 Moved Permanently
Location: https://www.google.com/ [following]
--2017-03-16 11:21:07-- https://www.google.com/
Connecting to proxy.mydomain-research.com (proxy.mydomain-research.com)|10.1.1.7|:3128... connected.
Proxy request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘index.html’
For reading more about the security aspects of connecting(and not connecting) with the proxy server over TLS/SSL, visit: https://security.stackexchange.com/a/61336/114965

Resources