Help with ACME HTTP01 Let's Encrypt challenge please.
i'm trying to get certificate for domain behind proxy.
my config:
upstream kube-ingress-https {
# The least number of active connections
least_conn;
server 172.22.16.3:443 max_fails=1 fail_timeout=300;
}
server {
listen 443;
proxy_protocol on;
proxy_pass kube-ingress-https;
}
my scheme: WORLD ====> NGINX (reverse proxy) ==============> K8S ingress(cert manager)
cert-manager shows: Waiting for HTTP-01 challenge propagation: wrong status code '502', expected '200'
curl -I -k https://k8s-dash.domain.kz/.well-known/acme-challenge/pXy1kENDkrXpGNKLeyCc8huIfgUILzjErxiYhj80mxI
HTTP/2 502
date: Fri, 21 Oct 2022 06:55:51 GMT
content-type: text/html
content-length: 150
strict-transport-security: max-age=15724800; includeSubDomains
access-control-allow-origin: *
access-control-allow-credentials: true
curl -I -k https://k8s-dash.mydomain.kz
HTTP/2 200
date: Fri, 21 Oct 2022 06:55:58 GMT
content-type: text/html; charset=utf-8
content-length: 1338
accept-ranges: bytes
cache-control: no-cache, no-store, must-revalidate
last-modified: Fri, 15 Oct 2021 07:41:12 GMT
strict-transport-security: max-age=15724800; includeSubDomains
access-control-allow-origin: *
access-control-allow-credentials: true
i've found a solution:
i've delete ingress annotation: acme.cert-manager.io/http01-edit-in-place: "true"
I want to use nginx as internet out proxy.
So, I try to connect to http://mirror01.org/google.com, but cache files were not created.
If I delete rewrite rule and replace proxy_pass to http://google.come cache files will be created.
Where did I go wrong?
Trace using curl
HTTP/1.1 302 Moved Temporarily
Server: nginx/1.20.2
Date: Tue, 08 Mar 2022 15:44:42 GMT
Content-Type: text/html
Content-Length: 145
Connection: keep-alive
Location: http://google.com
HTTP/1.1 301 Moved Permanently
Location: http://www.google.com/
Content-Type: text/html; charset=UTF-8
Date: Tue, 08 Mar 2022 15:44:42 GMT
Expires: Thu, 07 Apr 2022 15:44:42 GMT
Cache-Control: public, max-age=2592000
Server: gws
Content-Length: 219
X-XSS-Protection: 0
X-Frame-Options: SAMEORIGIN
HTTP/1.1 200 OK
Content-Type: text/html; charset=ISO-8859-1
P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
Date: Tue, 08 Mar 2022 15:44:43 GMT
Server: gws
X-XSS-Protection: 0
X-Frame-Options: SAMEORIGIN
Transfer-Encoding: chunked
Expires: Tue, 08 Mar 2022 15:44:43 GMT
Cache-Control: private
Set-Cookie: 1P_JAR=2022-03-08-15; expires=Thu, 07-Apr-2022 15:44:43 GMT; path=/; domain=.google.com; Secure
Set-Cookie: NID=511=21PQ3ziwDTFTppHDMUoAVReBsFB6oGkVHqT38eqJO25UQkP3SJSEleOsHhefVDR_TgKSK6DpcTmBewjKC-kazv8mWRrfW10NUJevh16H5MZtKrKCJxllfI4r; expires=Wed, 07-Sep-2022 15:44:43 GMT; path=/; domain=.google.com; HttpOnly
Nginx conf:
access_log /opt/nginx/log/access.log main;
error_log /opt/nginx/log/error.log crit;
proxy_cache_path /opt/nginx/cache levels=1:2 keys_zone=default_cache:10m max_size=2g
inactive=120m use_temp_path=off;
proxy_cache_key "$scheme$request_method$host$request_uri";
proxy_cache_valid any 60m;
server {
listen 80;
server_name mirror01.org;
location / {
proxy_cache default_cache;
proxy_buffering on;
proxy_ignore_headers Expires;
proxy_ignore_headers X-Accel-Expires;
proxy_ignore_headers Cache-Control;
proxy_ignore_headers Set-Cookie;
proxy_hide_header X-Accel-Expires;
proxy_hide_header Expires;
proxy_hide_header Cache-Control;
proxy_hide_header Pragma;
add_header X-Proxy-Cache $upstream_cache_status;
rewrite ^/(.*)$ http://$request_uri? break;
proxy_pass $request_uri;
}
}
The following URL was posted in another question.
Using wget you get the csv file as expected, but curl ends up redirected you to something different. I was wondering what the differences are between the two commands or how to get the same result in curl.
wget
wget --output-document=test.csv --no-check-certificate 'https://docs.google.com/spreadsheet/ccc?key=0At2sqNEgxTf3dEt5SXBTemZZM1gzQy1vLVFNRnludHc&output=csv'
curl
curl --location --insecure --output test.csv 'https://docs.google.com/spreadsheet/ccc?key=0At2sqNEgxTf3dEt5SXBTemZZM1gzQy1vLVFNRnludHc&output=csv'
Updated with header information
header comparison
wget 1
--2014-07-03 09:54:30-- https://docs.google.com/spreadsheet/ccc?key=0At2sqNEgxTf3dEt5SXBTemZZM1gzQy1vLVFNRnludHc&output=csv
Resolving docs.google.com... 74.125.226.98, 74.125.226.100, 74.125.226.102, ...
Connecting to docs.google.com|74.125.226.98|:443... connected.
HTTP request sent, awaiting response...
HTTP/1.1 302 Moved Temporarily
Content-Type: text/html; charset=UTF-8
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: Fri, 01 Jan 1990 00:00:00 GMT
Date: Thu, 03 Jul 2014 13:54:30 GMT
X-Robots-Tag: noindex, nofollow, nosnippet
Location: https://www.google.com/url?q=https://docs.google.com/spreadsheet/ccc?key%3D0At2sqNEgxTf3dEt5SXBTemZZM1gzQy1vLVFNRnludHc%26output%3Dcsv%26pref%3D2&sa=p
Set-Cookie: NID=67=D4vu38cFuNFB-qdFSdaVBpLKJ94VcnpcVDfEpoyECGG-EesJlxBW4Rwb-AA-XAF7ztGOAIzx3u2YYqwRlt516cv3i6jSa9Pazf3uK-hyR5p5QoEYaZ-MqRpj9H_utLwU;Domain=.google.com;Path=/;Expires=Fri, 02-Jan-2015 13:54:30 GMT;HttpOnly
P3P: CP="This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info."
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Server: GSE
Alternate-Protocol: 443:quic
Transfer-Encoding: chunked
Location: https://www.google.com/url?q=https://docs.google.com/spreadsheet/ccc?key%3D0At2sqNEgxTf3dEt5SXBTemZZM1gzQy1vLVFNRnludHc%26output%3Dcsv%26pref%3D2&sa=p [following]
curl 1
HTTP/1.1 302 Moved Temporarily
Content-Type: text/html; charset=UTF-8
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: Fri, 01 Jan 1990 00:00:00 GMT
Date: Thu, 03 Jul 2014 13:59:48 GMT
X-Robots-Tag: noindex, nofollow, nosnippet
Location: https://www.google.com/url?q=https://docs.google.com/spreadsheet/ccc?key%3D0At2sqNEgxTf3dEt5SXBTemZZM1gzQy1vLVFNRnludHc%26output%3Dcsv%26pref%3D2&sa=p
Set-Cookie: NID=67=QTFWWFkySepW985crZ2dZk1JfQ8gGj_H59HwYp-SMcOvYl0J4JU3VfDGCqppxFcEPt-e48qr0yJOx2ImUKH65LlgvuLyF3Ec842bPFq-BFg9a7YWEP_5Uq8YJrJ58taL;Domain=.google.com;Path=/;Expires=Fri, 02-Jan-2015 13:59:48 GMT;HttpOnly
P3P: CP="This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info."
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Server: GSE
Transfer-Encoding: chunked
wget 2
--2014-07-03 09:54:30-- https://www.google.com/url?q=https://docs.google.com/spreadsheet/ccc?key%3D0At2sqNEgxTf3dEt5SXBTemZZM1gzQy1vLVFNRnludHc%26output%3Dcsv%26pref%3D2&sa=p
Resolving www.google.com... 74.125.225.144, 74.125.225.145, 74.125.225.148, ...
Connecting to www.google.com|74.125.225.144|:443... connected.
HTTP request sent, awaiting response...
HTTP/1.1 302 Found
X-Frame-Options: ALLOWALL
Location: https://docs.google.com/spreadsheet/ccc?key=0At2sqNEgxTf3dEt5SXBTemZZM1gzQy1vLVFNRnludHc&output=csv&pref=2
Cache-Control: private
Content-Type: text/html; charset=UTF-8
Set-Cookie: PREF=ID=1f6208c8ba0c71f9:FF=0:TM=1404395670:LM=1404395670:S=HaS679Z5xbmJBKs7; expires=Sat, 02-Jul-2016 13:54:30 GMT; path=/; domain=.google.com
Date: Thu, 03 Jul 2014 13:54:30 GMT
Server: gws
Content-Length: 311
X-XSS-Protection: 1; mode=block
Alternate-Protocol: 443:quic
Location: https://docs.google.com/spreadsheet/ccc?key=0At2sqNEgxTf3dEt5SXBTemZZM1gzQy1vLVFNRnludHc&output=csv&pref=2 [following]
curl 2
HTTP/1.1 302 Found
X-Frame-Options: ALLOWALL
Location: https://docs.google.com/spreadsheet/ccc?key=0At2sqNEgxTf3dEt5SXBTemZZM1gzQy1vLVFNRnludHc&output=csv&pref=2
Cache-Control: private
Content-Type: text/html; charset=UTF-8
Set-Cookie: PREF=ID=432f03534cff2fd2:FF=0:TM=1404395989:LM=1404395989:S=1NwOiUYJQYKfn6qF; expires=Sat, 02-Jul-2016 13:59:49 GMT; path=/; domain=.google.com
Set-Cookie: NID=67=EjeYW1PP63Nxk5upQVhEVreT_prZXQYQy4WVKZCHkY3cXffcTWyvXIJkt4Tg07LUoHo3GSkEg6qDh5ff5ESGhksbjT50ytYRd0SyKp7quyorpbT4GMhnbORlkFfTNdRc; expires=Fri, 02-Jan-2015 13:59:49 GMT; path=/; domain=.google.com; HttpOnly
P3P: CP="This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info."
Date: Thu, 03 Jul 2014 13:59:49 GMT
Server: gws
Content-Length: 311
X-XSS-Protection: 1; mode=block
Alternate-Protocol: 443:quic
wget 3
--2014-07-03 09:54:31-- https://docs.google.com/spreadsheet/ccc?key=0At2sqNEgxTf3dEt5SXBTemZZM1gzQy1vLVFNRnludHc&output=csv&pref=2
Connecting to docs.google.com|74.125.226.98|:443... connected.
HTTP request sent, awaiting response...
HTTP/1.1 302 Moved Temporarily
Content-Type: text/html; charset=UTF-8
Location: https://docs.google.com/spreadsheet/ccc?key=0At2sqNEgxTf3dEt5SXBTemZZM1gzQy1vLVFNRnludHc&output=csv
Date: Thu, 03 Jul 2014 13:54:31 GMT
Expires: Thu, 03 Jul 2014 13:54:31 GMT
Cache-Control: private, max-age=0
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Server: GSE
Alternate-Protocol: 443:quic
Transfer-Encoding: chunked
Location: https://docs.google.com/spreadsheet/ccc?key=0At2sqNEgxTf3dEt5SXBTemZZM1gzQy1vLVFNRnludHc&output=csv [following]
curl 3
HTTP/1.1 302 Moved Temporarily
Content-Type: text/html; charset=utf-8
Location: https://www.google.com/accounts/ServiceLogin?service=wise&passive=1209600&continue=https://docs.google.com/spreadsheet/ccc?key%3D0At2sqNEgxTf3dEt5SXBTemZZM1gzQy1vLVFNRnludHc%26output%3Dcsv%26pref%3D2&followup=https://docs.google.com/spreadsheet/ccc?key%3D0At2sqNEgxTf3dEt5SXBTemZZM1gzQy1vLVFNRnludHc%26output%3Dcsv%26pref%3D2<mpl=sheets
Content-Length: 2270
Set-Cookie: NID=67=NdTD41weGlHPUtsUMwF0a7ugZ5Hfof3Q8CFsy2gQcJuBaH8ugZIYppe2PWWhP5fEMtdToEi76-lQH_lAJUeLEkNo1nObesgzEnSSg3HEJeb-5vYrAs4fwR7bM7Ourxeh;Domain=.google.com;Path=/;Expires=Fri, 02-Jan-2015 13:59:49 GMT;HttpOnly
P3P: CP="This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info."
Date: Thu, 03 Jul 2014 13:59:49 GMT
Expires: Thu, 03 Jul 2014 13:59:49 GMT
Cache-Control: private, max-age=0
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Server: GSE
wget 4 (final)
--2014-07-03 09:54:31-- https://docs.google.com/spreadsheet/ccc?key=0At2sqNEgxTf3dEt5SXBTemZZM1gzQy1vLVFNRnludHc&output=csv
Reusing existing connection to docs.google.com:443.
HTTP request sent, awaiting response...
HTTP/1.1 200 OK
Content-Type: text/csv; charset=utf-8
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: Fri, 01 Jan 1990 00:00:00 GMT
Date: Thu, 03 Jul 2014 13:54:31 GMT
X-Robots-Tag: noindex, nofollow, nosnippet
Content-Disposition: attachment; filename="Download Test Spreadsheet.csv"
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Server: GSE
Alternate-Protocol: 443:quic
Transfer-Encoding: chunked
curl 4
HTTP/1.1 302 Moved Temporarily
Content-Type: text/html; charset=UTF-8
Location: https://accounts.google.com/ServiceLogin?service=wise&passive=1209600&continue=https%3A%2F%2Fdocs.google.com%2Fspreadsheet%2Fccc%3Fkey%3D0At2sqNEgxTf3dEt5SXBTemZZM1gzQy1vLVFNRnludHc%26output%3Dcsv%26pref%3D2&followup=https%3A%2F%2Fdocs.google.com%2Fspreadsheet%2Fccc%3Fkey%3D0At2sqNEgxTf3dEt5SXBTemZZM1gzQy1vLVFNRnludHc%26output%3Dcsv%26pref%3D2<mpl=sheets
Content-Length: 556
Date: Thu, 03 Jul 2014 13:59:49 GMT
Expires: Thu, 03 Jul 2014 13:59:49 GMT
Cache-Control: private, max-age=0
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Server: GSE
Alternate-Protocol: 443:quic
curl 5 (final)
HTTP/1.1 200 OK
Content-Type: text/html; charset=UTF-8
Strict-Transport-Security: max-age=10893354; includeSubDomains
Set-Cookie: GAPS=1:v3eXsN1lqmN5ryz1eyf2iMBP2uoIGg:wiYHYyLrGeoRHUfk;Path=/;Expires=Sat, 02-Jul-2016 13:59:49 GMT;Secure;HttpOnly;Priority=HIGH
X-Frame-Options: DENY
Date: Thu, 03 Jul 2014 13:59:49 GMT
Expires: Thu, 03 Jul 2014 13:59:49 GMT
Cache-Control: private, max-age=0
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Content-Length: 0
Server: GSE
Alternate-Protocol: 443:quic
a great debugging-technique is to open that link while having the developer toolbar open in chrome and look at the network-tab. all requests in that tab can be right-clicked to show the cURL command to download that information.
in your case, the issue seems to be that wget is handling cookies for you, while cURL does not. this should be easy to fix:
curl 'https://docs.google.com/spreadsheet/ccc?key=0At2sqNEgxTf3dEt5SXBTemZZM1gzQy1vLVFNRnludHc&output=csv' --location --cookie tmp.cookie
# Foo,Bar,Baz
# 1,2,3
# 4,5,6
I am in doubt if I have my cache enabled. It thought I did, but Google Speed Test tells me it's not and so does the 'seositecheckup'-test. These are the HTTP Headers:
HTTP/1.1 200 OK
Date: Tue, 13 May 2014 15:33:37 GMT
Server: Apache/2.2.26 (Unix) mod_ssl/2.2.26 OpenSSL/1.0.1e-fips mod_auth_passthrough/2.1 mod_bwlimited/1.4 FrontPage/5.0.2.2635
X-Powered-By: PHP/5.4.24
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: TCT_SESS=412bfe220023a4bf9b1233f24748796b; path=/; domain=.asecret.nl
Cache-Control: max-age=86400, public
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 14540
Connection: close
Content-Type: text/html
Both the expires_module and the headers_module are loaded and in WHM in my pre-main-include (should put the text at the top of the httpd.conf) I have this:
# 1 DAY
<FilesMatch "\.(html|htm|php|cgi|pl|css|js)$">
Header set Cache-Control "max-age=86400, public"
</FilesMatch>
# 1 MONTH
<FilesMatch "\.(jpg|jpeg|png|gif|swf)$">
Header set Cache-Control "max-age=2630000, public"
</FilesMatch>
I'm confused, is it enabled, or is it not?
I am testing one application, its containing a header HG-ACL-ID,HG-ACL,HG-USER , i didn't know what are these and use of these. can anybody help me out?
here is the http request and response headers
GET /nanotech-001/human_grading/api/assessments/4/peerGradingSets/latest HTTP/1.1
Host:
User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:27.0) Gecko/20100101 Firefox/27.0
Accept: application/json, text/javascript, */*; q=0.01
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
HG-ACL-ID: 1
X-CSRF-Token: LBiyM9kSAA8YHlBLUXQM
X-Requested-With: XMLHttpRequest
Referer:
Cookie: csrf_token=LBiyM9kSAA8YHlBLUXQM; __204u=4654485718-1388476789706; __204r=; __utma=158142248.881358618.1388476794.1388827712.1388830229.25; __utmc=158142248; __utmz=158142248.1388830229.25.13.utmcsr=class.coursera.org|utmccn=(referral)|utmcmd=referral|utmcct=/ml-004/quiz/attempt; __qca=P0-349638149-1388488065337; user_segment=Prospect; __utmb=158142248.500.10.1388830229; maestro_login_flag=1; CAUTH=Tb7QI7IfVe7siBPLxl1gjyymwc5qZXAKuH8R2DwINdUKnEqDRZn7C8pf2UUbdhmjeQH18RinYNkf5Em6b7ivzg.fAwaIfFYBb4fhYyga1K5Yw.ph7vJk6hG4RnPeLaJZGnfE0x8hdx_Rr0s3g_cvFDCOJl9kQRVpORQqSuq2PXL5nCNksPrU4m0fClcInHonBoi-P7v3JzWku5Y6_pkV4UWsEZKrx309RwtJGwhDUizW0iw2QVrl8aP9Mik-uSdgE948m2UsS4A-BOoUE-MLnel9YoAMSgtekYmtkT1ucWjNqR
Connection: keep-alive
Response:
HTTP/1.1 403 Forbidden
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Content-Type: text/html; charset=utf-8
Date: Sat, 04 Jan 2014 11:22:45 GMT
Expires: Thu, 19 Nov 1981 08:52:00 GMT
HG-ACL: {"submitting":"ri","training":"id","peerGrading":"i","selfGrading":"i","results":"i","phase":"grading"}
HG-User: {"inSignatureTrack":0}
Pragma: no-cache
Server: nginx
Set-Cookie: csrf_token=LBiyM9kSAA8YHlBLUXQM; expires=Mon, 03-Feb-2014 11:22:45 GMT; path=/nanotech-001
Vary: Accept-Encoding
Vary: Accept-Encoding
X-Powered-By: PHP/5.3.10-1ubuntu3.9
Content-Length: 61
Connection: keep-alive