For Tensorflow serving 1.7.0, What is the version of SSL/TLS being used. Is it boringssl or OpenSSL? Please let me know the version number
TensorFlow Serving uses gRPC which in turn uses/can use boringssl. As to the concrete version, are you referring to the TLS protocol versions (i.e. 1.1 to 1.3) or the actual source code revision of the boringssl implementation ?
The latter is somewhat difficult, as there are no named releases of boringssl in the traditional sense.
TensorFlow Serving 1.7.0 seems to use a grpc version that includes the following version:
https://boringssl.googlesource.com/boringssl/+archive/886e7d75368e3f4fab3f4d0d3584e4abfc557755.tar.gz
I have not found a way to map back from the SHA256 hash (886e7d75368e3f4fab3f4d0d3584e4abfc557755) to the commit ID in the boringssl/master-with-google branch.
There is some TLS1.3 code in that version of boringssl, but checking against my tensorflow_model_server with SSL enabled (note that the stock Google server does not use SSL at all), it looks like TLS1.2 is in use (which makes sense to the newness of 1.3):
openssl s_client -connect my-ssl-enabled-tensorflowserver:myport -tls1_2
yields the appropriate "let's use this" response.
Related
Having an issue running airflow on my M1 Mac. Keeps erroring out with error in Flask-OpenID setup command: use_2to3 is invalid. I have setuptools < 58 and still having issues.
ERROR: Could not find a version that satisfies the requirement flask-openid==1.2.5 (from versions: 0.9, 0.9.1, 1.0, 1.0.1, 1.1, 1.1.1, 1.2, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.3.0)
ERROR: No matching distribution found for flask-openid==1.2.5
Yes. It's been fixed in flask_openid 1.2.6 (It's not a problem with Airflow but with FlaskOpenID).
Looks like for some reason your setuptools is not what you think it is. See:
https://github.com/pallets-eco/flask-openid/issues/59
You have not explained a crucial things - how you are installing airflow, neither which version of Airflow you try to install - which does not help in trying to help you unfortunately, so I have to make some guesses. Here is what you can do if you cannot - for any reason - downgrade to setuptools < 0.58.
If you are using Airlfow 2 and using constraints (as you should - this is the only supported way https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-pypi.html) for some older version of Airflow, then possibly flask-openid is 1.2.5 in those old constraint versions. Please check it and if you REALLY want to stay with an older version, then you can download the constraint file locally, modify flask-openid version to 1.2.6 and point to the file instead of the github URL as you should do normally (If you don't use constraints - start using them immediately).
However, better option than installing an old version of Airflow, will be to update to the latest version of Airflow (currently 2.2.2 but we are about to start voting on 2.2.3), where this problem is for sure fixed (also in few other versions). Airflow folows SemVer so you should be generally safe to migrate to 2.2.2 if you've used an earlier version of Airflow 2.
If you are trying to install Airflow 1.10.* - then don't do it. Move Airflow 2 immediately. Airflow 1.10 has reached end of life in June 2021 and it's almost half a year as it did not receive any fixes - it won't receive any fixes for the Flask OpenID problem, so you are pretty much on your own here.
Also you make yourself vulnerable to unpatched security issues (Airlfow 1.10 stopped receiving also critical security fixes as of June 2021).
I am having issues with the Dexador library (and same issue with the Drakma library) when attempting to webscrape. They work fine with HTTP requests, but I receive an error when working with HTTPS requests. Here is an example of the basic get request that I sent:
(defvar *url* "https://www.amazon.com/")
(defvar *request* (dex:get *url*))
Then I receive this backtrace:
No OpenSSL version number could be determined, both SSLeay and OpenSSL_version_num failed.
So i'm assuming something may be wrong with the OpenSSL library, but i'm not 100% sure what's going on. I achieved the request successfully on my windows machine from the command line, and OpenSSL works fine on my Windows 10 machine, but not in Common Lisp.
One solution that I saw was to set the keyword argument insecure to true:
(defvar *request* (dex:get *url* :insecure t))
But this does not work and a receive the same error. When I attempted to scrape an http website the request was successful:
(defvar *request* (dex:get "http://paulgraham.com"))
I am wondering if anyone else has had this problem and if there are any solutions that anyone has found. I do not know if this is an issue with Windows 10, a dependency library, or something else.
This works fine from the command line with:
curl -v "https://www.amazon.com"
The issue here is that my version of OpenSSL was too new. CL+SSL is a dependency of Dexador, and CL+SSL wants the 1.0.1 version, and it wants the 32-bit version of OpenSSL specifically. After playing around with various versions, that seemed to fix the problem.
ETA: Per https://github.com/hiratake55/RForcecom/issues/42, it looks like the author of the rforcecom package has updated rforcecom to use httr instead of RCurl (as of today, to be uploaded to CRAN tomorrow, 7/1/16), so my particular issue will be solved at that point. However, the general case (implementing TLS 1.1 / 1.2 in RCurl) may still be worth pursuing for other packages. Or everyone may just switch to the more recent curl package instead of RCurl.
Background: I've been using the rforcecom package to communicate with Salesforce for several months. Salesforce recently disabled support for TLS v1.0 and is requiring TLS v1.1 or higher in their Sandboxes; this update will take place for Production environments in March 2017.
rforcecom uses RCurl to communicate with salesforce.com servers. Generally the curlPerform method is used, which is implemented something like this (this example is from rforcecom.login.R):
h <- basicHeaderGatherer()
t <- basicTextGatherer()
URL <- paste(loginURL, rforcecom.api.getSoapEndpoint(apiVersion), sep="")
httpHeader <- c("SOAPAction"="login","Content-Type"="text/xml")
curlPerform(url=URL, httpheader=httpHeader, postfields=soapBody, headerfunction = h$update, writefunction = t$update, ssl.verifypeer=F)
This has been working for me for a while, as I mentioned. Now that Salesforce has disabled TLS v1.0 on Sandboxes, though, it fails with the following error:
UNSUPPORTED_CLIENT: TLS 1.0 has been disabled in this organization. Please use TLS 1.1 or higher when connecting to Salesforce using https.
I've modified and sourced (without implementing it in the local copy of the package, as I'm not experienced enough to do this) a change to my local copy of the login module for RForcecom, and I've discovered through experimentation that I can specify any of the existing enumerated values for SSLVERSION successfully by adding sslversion=SSLVERSION_TLSv1, sslversion=SSLVERSION_SSLv3, etc. to the curlPerform options where it is called. However, all of these give me the same error as above. When I attempt to use one of the options that is implemented in libcurl but not in RCurl (SSLVERSION_TLSv1.1, SSLVERSION_TLSv1.2), I get the following error:
Error in merge(list(...), .opts) : object 'SSLVERSION_TLSv1.1' not found
or:
Error in merge(list(...), .opts) : object 'SSLVERSION_TLSv1.2' not found
I've verified with curlVersion() that my libcurl version is 7.40.0, which according to https://curl.haxx.se/libcurl/c/CURLOPT_SSLVERSION.html does support those options. However, I'm unable to get RCurl to recognize them.
At this point what I'm looking for is a way to get RCurl to use TLS v1.1 or TLS v1.2, and I would very much appreciate any assistance I can get with that. I apologize for any problems/issues with my question as this is my first time asking one myself, I've previously always been able to muddle through by reading other people's questions and answers.
RCurl is an interface to libcurl and what it supports depends on the latter. It is possible that your libcurl was built with an older version of OpenSSL which does not have support for TLS v1.1 or v.1.2. Your can determine your SSL version from R like this:
RCurl::curlVersion()$ssl_version
I think that by default (e.g. ssl option CURL_SSLVERSION_DEFAULT), during the SSL handshake, the server and client would agree on the newest version that they both support. To make it work you would have to update OpenSSL to a newer version, recompile libcurl with it and rebuild RCurl so that it registers the updates.
That said, you can enforce a particular ssl version not defined in RCurl by passing the integer value of the required option yourself. The numbers you are looking for can be deduced from the C enum defined in the curl header file on GitHub:
enum {
CURL_SSLVERSION_DEFAULT, // 0
CURL_SSLVERSION_TLSv1, /* TLS 1.x */ // 1
CURL_SSLVERSION_SSLv2, // 2
CURL_SSLVERSION_SSLv3, // 3
CURL_SSLVERSION_TLSv1_0, // 4
CURL_SSLVERSION_TLSv1_1, // 5
CURL_SSLVERSION_TLSv1_2, // 6
CURL_SSLVERSION_TLSv1_3, // 7
CURL_SSLVERSION_LAST /* never use, keep last */ // 8
};
For example:
# the data of you post request
nameValueList = list(data1 = "data1", data2 = "data2")
CURL_SSLVERSION_TLSv1_1 <- 5L
CURL_SSLVERSION_TLSv1_2 <- 6L
# TLS 1.1
opts <- RCurl::curlOptions(verbose = TRUE,
sslversion = CURL_SSLVERSION_TLSv1_1, ...)
# TLS 1.2
opts <- RCurl::curlOptions(verbose = TRUE,
sslversion = CURL_SSLVERSION_TLSv1_2, ...)
# finally, POST the data
RCurl::postForm(URL, .params = nameValueList, .opts = opts)
In general, this may not be a good practice because the authors of cURL could decide to change the values (although I believe chances are low) those options have.
I am scraping the following site using requests and certifi: https://ecf.ared.uscourts.gov/
When I scrape it on my local machine, it works fine, but when I run automated tests in Travis, it fails with:
SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)
The code I'm using for this is pretty straightforward:
url = 'https://ecf.ared.uscourts.gov/cgi-bin/login.pl'
r = requests.post(
url,
verify=certifi.where(),
timeout=60,
files={
'login': ('', username),
'key': ('', password)
},
)
Any ideas what's different between Travis and my local machine that would cause a certificate failure? I thought the idea of certifi was to standardize all this. (I've verified I have updated versions on both computers.)
Got it! Travis uses an old version of OpenSSL which doesn't support cross-signed certificates, but my local dev machine uses a modern version of OpenSSL.
The solution here is to use certifi.old_where() which provides older, less secure certificate support. I've resigned myself to this until I can upgrade the version of OpenSSL that we have.
ETA: Per https://github.com/hiratake55/RForcecom/issues/42, it looks like the author of the rforcecom package has updated rforcecom to use httr instead of RCurl (as of today, to be uploaded to CRAN tomorrow, 7/1/16), so my particular issue will be solved at that point. However, the general case (implementing TLS 1.1 / 1.2 in RCurl) may still be worth pursuing for other packages. Or everyone may just switch to the more recent curl package instead of RCurl.
Background: I've been using the rforcecom package to communicate with Salesforce for several months. Salesforce recently disabled support for TLS v1.0 and is requiring TLS v1.1 or higher in their Sandboxes; this update will take place for Production environments in March 2017.
rforcecom uses RCurl to communicate with salesforce.com servers. Generally the curlPerform method is used, which is implemented something like this (this example is from rforcecom.login.R):
h <- basicHeaderGatherer()
t <- basicTextGatherer()
URL <- paste(loginURL, rforcecom.api.getSoapEndpoint(apiVersion), sep="")
httpHeader <- c("SOAPAction"="login","Content-Type"="text/xml")
curlPerform(url=URL, httpheader=httpHeader, postfields=soapBody, headerfunction = h$update, writefunction = t$update, ssl.verifypeer=F)
This has been working for me for a while, as I mentioned. Now that Salesforce has disabled TLS v1.0 on Sandboxes, though, it fails with the following error:
UNSUPPORTED_CLIENT: TLS 1.0 has been disabled in this organization. Please use TLS 1.1 or higher when connecting to Salesforce using https.
I've modified and sourced (without implementing it in the local copy of the package, as I'm not experienced enough to do this) a change to my local copy of the login module for RForcecom, and I've discovered through experimentation that I can specify any of the existing enumerated values for SSLVERSION successfully by adding sslversion=SSLVERSION_TLSv1, sslversion=SSLVERSION_SSLv3, etc. to the curlPerform options where it is called. However, all of these give me the same error as above. When I attempt to use one of the options that is implemented in libcurl but not in RCurl (SSLVERSION_TLSv1.1, SSLVERSION_TLSv1.2), I get the following error:
Error in merge(list(...), .opts) : object 'SSLVERSION_TLSv1.1' not found
or:
Error in merge(list(...), .opts) : object 'SSLVERSION_TLSv1.2' not found
I've verified with curlVersion() that my libcurl version is 7.40.0, which according to https://curl.haxx.se/libcurl/c/CURLOPT_SSLVERSION.html does support those options. However, I'm unable to get RCurl to recognize them.
At this point what I'm looking for is a way to get RCurl to use TLS v1.1 or TLS v1.2, and I would very much appreciate any assistance I can get with that. I apologize for any problems/issues with my question as this is my first time asking one myself, I've previously always been able to muddle through by reading other people's questions and answers.
RCurl is an interface to libcurl and what it supports depends on the latter. It is possible that your libcurl was built with an older version of OpenSSL which does not have support for TLS v1.1 or v.1.2. Your can determine your SSL version from R like this:
RCurl::curlVersion()$ssl_version
I think that by default (e.g. ssl option CURL_SSLVERSION_DEFAULT), during the SSL handshake, the server and client would agree on the newest version that they both support. To make it work you would have to update OpenSSL to a newer version, recompile libcurl with it and rebuild RCurl so that it registers the updates.
That said, you can enforce a particular ssl version not defined in RCurl by passing the integer value of the required option yourself. The numbers you are looking for can be deduced from the C enum defined in the curl header file on GitHub:
enum {
CURL_SSLVERSION_DEFAULT, // 0
CURL_SSLVERSION_TLSv1, /* TLS 1.x */ // 1
CURL_SSLVERSION_SSLv2, // 2
CURL_SSLVERSION_SSLv3, // 3
CURL_SSLVERSION_TLSv1_0, // 4
CURL_SSLVERSION_TLSv1_1, // 5
CURL_SSLVERSION_TLSv1_2, // 6
CURL_SSLVERSION_TLSv1_3, // 7
CURL_SSLVERSION_LAST /* never use, keep last */ // 8
};
For example:
# the data of you post request
nameValueList = list(data1 = "data1", data2 = "data2")
CURL_SSLVERSION_TLSv1_1 <- 5L
CURL_SSLVERSION_TLSv1_2 <- 6L
# TLS 1.1
opts <- RCurl::curlOptions(verbose = TRUE,
sslversion = CURL_SSLVERSION_TLSv1_1, ...)
# TLS 1.2
opts <- RCurl::curlOptions(verbose = TRUE,
sslversion = CURL_SSLVERSION_TLSv1_2, ...)
# finally, POST the data
RCurl::postForm(URL, .params = nameValueList, .opts = opts)
In general, this may not be a good practice because the authors of cURL could decide to change the values (although I believe chances are low) those options have.