I have written lua code and integrated in nginx to upload files into s3 bucket. If the upload fails or signature is malformed, i get an 403 Forbidden which is send to the client along with very verbose Error XML.
I wanted to intercept/ return default error msg instead of the verbose error msg from s3 which reveals Access Key ID.
I tried adding snippet below but it has no effect what so ever:
error_page 403 /forbidden;
location = /forbidden {
internal;
content_by_lua_block {
local cjson = require "cjson"
ngx.header.content_type = "application/json; charset=utf-8"
ngx.status = 403
ngx.say(cjson.encode({ status = "Forbidden. Check the Request" }))
return ngx.exit(ngx.status)
}
}
When i see the reponse of curl cmd to upload, There are two status codes, 100 and then 403.
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 403 Forbidden
< Date: Thu, 28 Sep 2017 04:26:31 GMT
< Content-Type: application/xml
< Transfer-Encoding: chunked
< Connection: keep-alive >
How do i handle this to return default response for all 403 status code response.
Related
I have a strange situation. I want to return the content type application/json; charset=utf-8 from an http handler.
func handleTest() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get("Accept") != "application/json" {
w.WriteHeader(http.StatusNotAcceptable)
return
}
w.WriteHeader(http.StatusOK)
w.Header().Set("Content-Type", "application/json; charset=utf-8")
json.NewEncoder(w).Encode(map[string]string{"foo": "bar"})
}
}
When I check for this in my unit tests it is correct. This test does not fail.
func TestTestHandler(t *testing.T) {
request, _ := http.NewRequest(http.MethodGet, "/test", nil)
request.Header.Set("Accept", "application/json")
response := httptest.NewRecorder()
handleTest().ServeHTTP(response, request)
contentType := response.Header().Get("Content-Type")
if contentType != "application/json; charset=utf-8" {
t.Errorf("Expected Content-Type to be application/json; charset=utf-8, got %s", contentType)
return
}
}
But when I try with curl (and other clients) it comes out as text/plain; charset=utf-8.
$ curl -H 'Accept: application/json' localhost:8080/test -v
* Trying 127.0.0.1:8080...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET /test HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.68.0
> Accept: application/json
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Tue, 28 Dec 2021 13:02:27 GMT
< Content-Length: 14
< Content-Type: text/plain; charset=utf-8
<
{"foo":"bar"}
* Connection #0 to host localhost left intact
I have tried this with curl, insomnia and python. In all 3 cases the content type came out as text/plain; charset=utf-8.
What is causing this problem and how can I fix it?
From the http package docs:
WriteHeader sends an HTTP response header with the provided status code.
and
Changing the header map after a call to WriteHeader (or Write) has no effect unless the modified headers are trailers.
So you are setting the "Content-Type" header after the header has already been sent out to the client. While mocking this likely works because the buffer where the headers are stored can be modified after the WriteHeader call. But when actually using a TCP connection you can't do this.
So simply move your w.WriteHeader(http.StatusOK) so it happens after the w.Header().Set(...)
How to collect the HTTP response status for a script?
Below is a sample code which will start a server and allow two routes for interaction.
using Genie
import Genie.Router: route
import Genie.Renderer.Json: json
Genie.config.run_as_server = true
route("/try/", method=GET) do
(:message => "Welcome") |> json
end
route("/test/", method=POST) do
data = jsonpayload()
<body>
end
Genie.startup()
How to collect the response status like 200, 500 or others as a string variable?
Open connection to your server using HTTP and look for the status field:
julia> using HTTP
julia> response = HTTP.get("http://127.0.0.1:8000/try")
HTTP.Messages.Response:
"""
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Server: Genie/1.18.1/Julia/1.6.1
Transfer-Encoding: chunked
{"message":"Welcome"}"""
julia> response.status
200
If you rather want to control the status yourself you can add on the server side:
route("/tryerror/", method=GET) do
Genie.Responses.setstatus(503)
end
And now let us test it for 503:
julia> response = HTTP.get("http://127.0.0.1:8000/tryerror")
ERROR: HTTP.ExceptionRequest.StatusError(503, "GET", "/tryerror", HTTP.Messages.Response:
"""
HTTP/1.1 503 Service Unavailable
Content-Type:
Server: Genie/1.18.1/Julia/1.6.1
Transfer-Encoding: chunked
""")
I have an app to create server certificate requests, just as if one were using java keytool or something. I'm trying to return the created certificate request and the key in a zip file, but for the life of me, I can't get my REST controller to respond to the http request. CORRECTION: The controller responds, but the code within the method is never executed.
The server does receive the request, because my CORS filter is executed. But I have a debug set in the controller method, and it's never triggered. Is the signature of the method correct? I need another set of eyes, please?
Here is my controller code:
#RequestMapping(method = RequestMethod.POST, value = "/generateCert/")
public ResponseEntity<InputStreamResource> generateCert(#RequestBody CertInfo certInfo) {
System.out.println("Received request to generate CSR...");
byte[] responseBytes = commonDataService.generateCsr(certInfo);
InputStreamResource resource = new InputStreamResource(new ByteArrayInputStream(responseBytes));
System.out.println("Generated CSR with length of " + responseBytes.length);
return ResponseEntity.ok()
.header(HttpHeaders.CONTENT_DISPOSITION, "attachment;filename=certificate.zip")
.contentType(MediaType.parseMediaType("application/zip"))
.contentLength(responseBytes.length)
.body(resource);
}
And here is the Angular request:
generateCertificate(reqBody: GenerateCert) {
let headers = new Headers();
headers.append('Content-Type', 'application/json');
this.http.post(this.urlGenerateCert, JSON.stringify(reqBody), {headers: headers}).subscribe(
(data) => {
let dataType = data.type;
let binaryData = [];
binaryData.push(data);
this.certBlob = new Blob(binaryData);
});
return this.certBlob;
}
And finally, the request and response headers I copied from the Network Panel:
Response
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: Content-Type, Authorization, Accept, X-Requested-With, remember-me
Access-Control-Allow-Methods: POST, GET, OPTIONS
Access-Control-Allow-Origin: *
Access-Control-Max-Age: 3600
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Length: 0
Date: Thu, 27 Dec 2018 22:48:00 GMT
Expires: 0
Location: http://localhost:8102/login
Pragma: no-cache
Set-Cookie: JSESSIONID=EDACE17328628D579670AD0FB53A6F35; Path=/; HttpOnly
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-XSS-Protection: 1; mode=block
Request
Accept: application/json, text/plain, */*
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Connection: keep-alive
Content-Length: 205
Content-Type: application/json
Host: localhost:8102
Origin: http://localhost:4200
Referer: http://localhost:4200/generateCerts
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.80 Safari/537.36
I really struggled with getting CORS working, so maybe that's interfering with the request? I hate to post all that code unless absolutely necessary. Anybody got any ideas?
Listing of request/response headers lack information on URL, method and most important response status code.
Seeing Location: http://localhost:8102/login among response headers I can guess that it could be 401 Unauthorized or anything else that redirects to the login page. Hence, if there is an auth filter in the filter chain, it may be a culprit.
The following request headers
Host: localhost:8102
Origin: http://localhost:4200
suggests that you are doing CORS and the CORS filter may be involved indeed and fulfill response before the request gets routed to the controller. I suggest setting a breakpoint into the CORS filter (and into others if any) and debug it to the point where the response is returned.
define a proxy.conf.json
{
"/login*": {
"target":"http://localhost:8080",
"secure":false,
"logLevel":"debug"
}
}
now in your package.json
"scripts": {
"start":"ng serve --proxy-config proxy.config.json"
}
I think there is issue while getting connection in both webapp.please try .
When Angular encounters this statement
this.http.post(url,body).subscribe(data => # some code
);
It comes back immediately to run rest of the code while service continues to execute. Just like Future in Java.
Here if you
return this.cert;
You will not get the value that may eventually get populated by the this.http service. Since the page has already rendered and the code executed. You can verify this by including this within and outside the Observable.
console.log(“Inside/outside observable” + new Date().toLocalTimeString());
Thanks to everyone who contributed. I discovered the error was due to the headers of my controller method. After changing them, the method was invoked properly. This is what worked:
#RequestMapping(method = RequestMethod.POST, path = "/generateCert",
produces = {MediaType.APPLICATION_OCTET_STREAM_VALUE}, consumes = {MediaType.APPLICATION_JSON_VALUE})
public ResponseEntity<byte[]> generateCert(#RequestBody CertInfo certInfo) {
byte[] responseBytes = commonDataService.generateCsr(certInfo);
return ResponseEntity.ok()
.header(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_OCTET_STREAM_VALUE)
.contentLength(responseBytes.length)
.body(responseBytes);
}
In Nginx I want to send a pre-request to another endpoint which checks for rate limiting violations. Basically exactly what auth_request does, but where auth_request only accepts authentication status codes (200, 401, 403), I want it to only allow rate limiting codes (200 or 429).
Is there a more general version of auth_request which could be used for this?
For now we're using auth_request, but the downside is it turns 429 status codes into 500s.
Thanks!
Below config works for me and returns a 429 instead of 500
events {
worker_connections 1024;
}
http {
server {
listen 80;
location /api {
auth_request /rate_limit;
error_page 500 = #rate_limit_error;
echo "You were allowed to access the API";
}
location #rate_limit_error {
return 429 "Limit has been exceeded\n";
}
location = /rate_limit {
internal;
return 400 "Access is not allowed";
}
}
}
The test shows the correct response
$ curl -v localhost/api?count=2
* Trying ::1...
* Connected to localhost (::1) port 80 (#0)
> GET /api?count=2 HTTP/1.1
> Host: localhost
> User-Agent: curl/7.47.0
> Accept: */*
>
< HTTP/1.1 429
< Server: openresty/1.11.2.2
< Date: Sat, 30 Sep 2017 18:50:56 GMT
< Content-Type: text/plain
< Content-Length: 24
< Connection: close
<
Limit has been exceeded
* Closing connection 0
If you don't want to return a message or something else. You can also use error_page 500 = 429;
I'm just trying to get the response body when the post request fails using RCurl. When I'm running it through R, I only get the response error and it fails out.
Error: Unprocessable Entity
when I post the same request with the UI interface for testing, it gives
{
"reason": [
"Can not create Data with Name: DataTest. Data Name should be unique."
],
"singleReason": "Can not create Data with Name: DataTest. Data Name should be unique."
}
Any help on how to get response bodies on 4XX errors is appreciated. TIA.
Post request
postdata.json <- '{"name":"DataTest","description":"Test Payload","algorithm":{"name":"DataTest","version":"0.1.0"}}'
post.result <- httpPOST(url=SERVER, postfields = postdata.json, verbose = T,
httpheader=c(Authorization=access.token, 'Content-Type'='application/json', Accept='application/json'))
RStudio output
* About to connect() to SERVER port 80 (#0)
* Trying SERVER... * connected
* Connected to SERVER port 80 (#0)
> POST /api/test HTTP/1.1
Host: SERVER
Authorization: AUTHENTICATION
Content-Type: application/json
Accept: application/json
Content-Length: 171
< HTTP/1.1 422 Unprocessable Entity
< Content-Type: application/json;charset=UTF-8
< Date: Thu, 13 Nov 2014 16:31:42 GMT
< Server: Apache-Coyote/1.1
< Content-Length: 215
< Connection: keep-alive
<
* Connection #0 to host SERVER left intact
Show Traceback
Rerun with Debug
Error: Unprocessable Entity
n.b. I'm okay with it failing and returning an Error. That's expected. I'm just trying to get the response body associated with the error code.
Here's one approach with httr:
library(httr)
postdata.json <- '{"name":"DataTest","description":"Test Payload","algorithm":{"name":"DataTest","version":"0.1.0"}}'
res <- POST(SERVER,
body = postdata.json,
add_headers(
Authorization = access.token,
'Content-Type' = 'application/json',
Accept = 'application/json'
)
)
http_status(res)
content(res)
I was able to get the response body using RCurl by adding a basicTextGatherer() and a writefunction.
reader <- basicTextGatherer()
post.result <- httpPOST(url=SERVER, postfields = postdata.json, verbose = T,
writefunction = reader$update,
httpheader=c(Authorization=access.token,
'Content-Type'='application/json',
Accept='application/json')
)
return(body=reader$value())
but Hadley's solution is simpler to work with and doesn't error out.