Might be a silly question, but I haven't found any clear answer yet.
My server handles ETag caching for some quite big JSON responses we have, returning 304 NOT MODIFIED with an empty body if the If-None-Match header contains the same hash as the one newly generated (shallow ETags).
Are browsers supposed to handle this automagically, or do the in-browser client apps consuming the API asynchronously need to implement some logic to handle such responses (i.e. use the cached version if 304 is responded, create/update the cached version otherwise)?
Because so far, I've manually implemented this logic client-side, but I'm wondering whether I just reinvented a square wheel...
In other words, with the Cache-Control header for example, the in-browser client apps don't need to parse the value, check for max-age for instance, stores it somehow, setup a timeout, etc.: everything is handled ahead by the browsers directly. The question is: are browsers supposed to behave the same way when they receive a 304?
Here is how I wrote my client so far (built with AngularJS, running in browsers):
myModule
.factory("MyRepository", ($http) => {
return {
fetch: (etag) => {
return $http.get(
"/api/endpoint",
etag ? { headers: { "If-None-Match": etag } } : undefined
);
}
};
})
.factory("MyService", (MyRepository, $q) => {
let latestEtag = null;
let latestVersion = null;
return {
fetch: () => {
return MyRepository
.fetch(latestEtag)
.then((response) => {
latestEtag = response.headers("ETag");
latestVersion = response.data;
return angular.copy(latestVersion);
})
.catch((response) => {
return 304 === error.status
? angular.copy(latestVersion)
: $q.reject(response)
});
}
};
});
So basically, is the above logic effectively needed, or am I supposed to be able to simply use $http.get("/api/endpoint") directly?
This code above is working fine, which seems to mean that it needs to be handled programmatically, although I've never seen such "custom" implementations on the articles I read.
The 304 responses are automagically handled by browser as such
So I created a simple page
<html>
<head>
<script src="./axios.min.js"></script>
<script src="./jquery-3.3.1.js"></script>
</head>
<body>
<h1>this is a test</page>
</body>
</html>
and the added a test.json file
root#vagrant:/var/www/html# cat test.json
{
"name": "tarun"
}
And then in nginx added below
location ~* \.(jpg|jpeg|png|gif|ico|css|js|json)$ {
expires 365d;
}
Now the results
AXIOS
As you can see the first request is 200 and second one 304 but there is no impact on the JS code
jQuery
Same thing with jQuery as well
From the curl you can see that server didn't send anything on the 2nd 304 request
$ curl -v 'http://vm/test.json' -H 'If-None-Match: "5ad71064-17"' -H 'DNT: 1' -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: en-US,en;q=0.9' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36' -H 'Accept: */*' -H 'Referer: http://vm/' -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive' -H 'If-Modified-Since: Wed, 18 Apr 2018 09:31:16 GMT' --compressed
* Trying 192.168.33.100...
* TCP_NODELAY set
* Connected to vm (192.168.33.100) port 80 (#0)
> GET /test.json HTTP/1.1
> Host: vm
> If-None-Match: "5ad71064-17"
> DNT: 1
> Accept-Encoding: gzip, deflate
> Accept-Language: en-US,en;q=0.9
> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36
> Accept: */*
> Referer: http://vm/
> X-Requested-With: XMLHttpRequest
> Connection: keep-alive
> If-Modified-Since: Wed, 18 Apr 2018 09:31:16 GMT
>
< HTTP/1.1 304 Not Modified
< Server: nginx
< Date: Wed, 18 Apr 2018 09:42:45 GMT
< Last-Modified: Wed, 18 Apr 2018 09:31:16 GMT
< Connection: keep-alive
< ETag: "5ad71064-17"
<
* Connection #0 to host vm left intact
So you don't need to handle a 304, browser will do that work for you.
Yes, probably all modern major browsers handle response validation using conditional requests well. Relevant excerpt from The State of Browser Caching, Revisited article by Mark Nottingham:
Validation allows a cache to check with the server to see if a stale stored response can be reused.
All of the tested browsers support validation based upon ETag and Last-Modified. The tricky part is making sure that the 304 Not Modified response is correctly combined with the stored response; specifically, the headers in the 304 update the stored response headers.
All of the tested browsers do update stored headers upon a 304, both in the immediate response and subsequent ones served from cache.
This is good news; updating headers with a 304 is an important mechanism, and when they get out of sync it can cause problems.
For more information check HTTP Caching article by Ilya Grigorik.
Related
I have an app to create server certificate requests, just as if one were using java keytool or something. I'm trying to return the created certificate request and the key in a zip file, but for the life of me, I can't get my REST controller to respond to the http request. CORRECTION: The controller responds, but the code within the method is never executed.
The server does receive the request, because my CORS filter is executed. But I have a debug set in the controller method, and it's never triggered. Is the signature of the method correct? I need another set of eyes, please?
Here is my controller code:
#RequestMapping(method = RequestMethod.POST, value = "/generateCert/")
public ResponseEntity<InputStreamResource> generateCert(#RequestBody CertInfo certInfo) {
System.out.println("Received request to generate CSR...");
byte[] responseBytes = commonDataService.generateCsr(certInfo);
InputStreamResource resource = new InputStreamResource(new ByteArrayInputStream(responseBytes));
System.out.println("Generated CSR with length of " + responseBytes.length);
return ResponseEntity.ok()
.header(HttpHeaders.CONTENT_DISPOSITION, "attachment;filename=certificate.zip")
.contentType(MediaType.parseMediaType("application/zip"))
.contentLength(responseBytes.length)
.body(resource);
}
And here is the Angular request:
generateCertificate(reqBody: GenerateCert) {
let headers = new Headers();
headers.append('Content-Type', 'application/json');
this.http.post(this.urlGenerateCert, JSON.stringify(reqBody), {headers: headers}).subscribe(
(data) => {
let dataType = data.type;
let binaryData = [];
binaryData.push(data);
this.certBlob = new Blob(binaryData);
});
return this.certBlob;
}
And finally, the request and response headers I copied from the Network Panel:
Response
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: Content-Type, Authorization, Accept, X-Requested-With, remember-me
Access-Control-Allow-Methods: POST, GET, OPTIONS
Access-Control-Allow-Origin: *
Access-Control-Max-Age: 3600
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Length: 0
Date: Thu, 27 Dec 2018 22:48:00 GMT
Expires: 0
Location: http://localhost:8102/login
Pragma: no-cache
Set-Cookie: JSESSIONID=EDACE17328628D579670AD0FB53A6F35; Path=/; HttpOnly
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-XSS-Protection: 1; mode=block
Request
Accept: application/json, text/plain, */*
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Connection: keep-alive
Content-Length: 205
Content-Type: application/json
Host: localhost:8102
Origin: http://localhost:4200
Referer: http://localhost:4200/generateCerts
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.80 Safari/537.36
I really struggled with getting CORS working, so maybe that's interfering with the request? I hate to post all that code unless absolutely necessary. Anybody got any ideas?
Listing of request/response headers lack information on URL, method and most important response status code.
Seeing Location: http://localhost:8102/login among response headers I can guess that it could be 401 Unauthorized or anything else that redirects to the login page. Hence, if there is an auth filter in the filter chain, it may be a culprit.
The following request headers
Host: localhost:8102
Origin: http://localhost:4200
suggests that you are doing CORS and the CORS filter may be involved indeed and fulfill response before the request gets routed to the controller. I suggest setting a breakpoint into the CORS filter (and into others if any) and debug it to the point where the response is returned.
define a proxy.conf.json
{
"/login*": {
"target":"http://localhost:8080",
"secure":false,
"logLevel":"debug"
}
}
now in your package.json
"scripts": {
"start":"ng serve --proxy-config proxy.config.json"
}
I think there is issue while getting connection in both webapp.please try .
When Angular encounters this statement
this.http.post(url,body).subscribe(data => # some code
);
It comes back immediately to run rest of the code while service continues to execute. Just like Future in Java.
Here if you
return this.cert;
You will not get the value that may eventually get populated by the this.http service. Since the page has already rendered and the code executed. You can verify this by including this within and outside the Observable.
console.log(“Inside/outside observable” + new Date().toLocalTimeString());
Thanks to everyone who contributed. I discovered the error was due to the headers of my controller method. After changing them, the method was invoked properly. This is what worked:
#RequestMapping(method = RequestMethod.POST, path = "/generateCert",
produces = {MediaType.APPLICATION_OCTET_STREAM_VALUE}, consumes = {MediaType.APPLICATION_JSON_VALUE})
public ResponseEntity<byte[]> generateCert(#RequestBody CertInfo certInfo) {
byte[] responseBytes = commonDataService.generateCsr(certInfo);
return ResponseEntity.ok()
.header(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_OCTET_STREAM_VALUE)
.contentLength(responseBytes.length)
.body(responseBytes);
}
I have a service written in Jolie, where I want to extract the http headers on request. In the same way the request.id can be printed out, I would like to print the headers. There is a try on the bold letter down in the code. Here the code:
execution { concurrent }
inputPort UserDB_Service {
Location: "socket://localhost:8002/"
Protocol: http { .format = "json"}
Interfaces: Users, ShutdownInterface, ConnectionPool
}
outputPort DB_Connector {
Location: "socket://localhost:1000/"
Protocol: sodep
Interfaces: ConnectionPool
}
init
{
connectionConfigInfo#DB_Connector()(connectionInfo);
connect#Database(connectionInfo)()
}
main
{
//Example: http://localhost:8002/retrieve?id=1
[ retrieve(request)(response) {
query#Database(
"select * from users where user_id=:id" {
.id = request.id
}
)(sqlResponse);
println#Console( "You have requested the user_id: " + request.id)();
**println#Console( "Request Headers: " + response.format)();**
if (#sqlResponse.row == 1) {
response -> sqlResponse.row[0]
}
} ]
}
Thanks for the help.
I did not understand if you know which headers you want to have in the inbound request or if you just want to print the whole http message for debugging purposes. It is quick in both cases, I report both solutions :)
In the first case you can set the headers parameter of the http protocol for the inputPort to include in the request message also the content of a specific header, e.g.,
http {
.headers.format = "format";
}
and then you can inspect the value in the usual way
println#Console( request.format )()
In the second case, you can use
http {
.debug = true;
.debug.showContent = true
}
to see the log of all http requests and responses and their bodies.
These and further info on protocols and in particular the http protocol is in the documentation of the Jolie site.
I put the output here again. I wonder if it is possible to extract the "iv-user: g47257" header, which I have injected by using Fiddler. Thanks again for the help.
The headers are like this (better format).
INFO: [UserDB_crud.ol] [HTTP debug] Receiving:
HTTP Code: 0
Resource: /retrieve?id=1
--> Header properties
iv-user: g47257
accept-language: en-US,en;q=0.8,da;q=0.6,es;q=0.4
host: localhost:8002
upgrade-insecure-requests: 1
connection: keep-alive
cache-control: max-age=0
accept-encoding: gzip, deflate, sdch
accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp
,*/*;q=0.8
user-agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTM
L, like Gecko) Chrome/48.0.2564.116 Safari/537.36
You have requested the user_id: 1
mar. 10, 2016 2:30:44 PM jolie.Interpreter logInfo
INFO: [UserDB_crud.ol] [HTTP debug] Sending:
HTTP/1.1 200 OK
Server: Jolie
X-Jolie-MessageID: 0
Content-Type: application/json; charset=utf-8
Content-Encoding: gzip
Content-Length: 72
?V*H,..?/JQ?R*I-.Q?Q*-N-??♦
↑?(?%?"dRs‼3s?\►?????T♂ %??WE
mar. 10, 2016 2:30:44 PM jolie.Interpreter logInfo
INFO: [UserDB_crud.ol] [HTTP debug] Receiving:
HTTP Code: 0
Resource: /favicon.ico
--> Header properties
iv-user: g47257
referer: http://localhost:8002/retrieve?id=1
accept-language: en-US,en;q=0.8,da;q=0.6,es;q=0.4
host: localhost:8002
connection: keep-alive
cache-control: no-cache
pragma: no-cache
accept-encoding: gzip, deflate, sdch
user-agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTM
L, like Gecko) Chrome/48.0.2564.116 Safari/537.36
accept: */*
mar. 10, 2016 2:30:44 PM jolie.Interpreter logWarning
WARNING: [UserDB_crud.ol] Received a message for operation favicon.ico, not specified in the input port at the receiving service. Sending IOException to the caller.
mar. 10, 2016 2:30:44 PM jolie.Interpreter logInfo
INFO: [UserDB_crud.ol] [HTTP debug] Sending:
HTTP/1.1 200 OK
Server: Jolie
X-Jolie-MessageID: 0
Content-Type: application/json; charset=utf-8
Content-Encoding: gzip
Content-Length: 102
?VJ-*?/R??V?M-.NLOU?R??w?HN-(???S?QJ?O☺?→←↓↑↑?(?$?$???%?d?(?↨?▬%?¶Z)?%?e&???☺ ??Z ?yd?Y
I re-post my last comment here since other people faced the same difficulties found by Efrin but might miss the solution I posted as a comment.
You can inspect the headers of a HTTP request as shown in the code below
include "console.iol"
inputPort Me {
Location: "socket://localhost:8000"
Protocol: http { .headers.iv_user = "ivUser" }
RequestResponse: myRequest
}
main {
myRequest( request )(){ println#Console( request.ivUser )() }
}
Remember that, as reported in the documentation, Jolie http.headers parameters map - in header names with _, e.g., in your case, header iv-user becomes iv_user in the Jolie HTTP protocol parameters.
Besides the description and code found in the Jolie documentation, you can find further examples and a more thorough explanation on how the HTTP protocol works in Jolie in its presentation paper wrote by Montesi https://doi.org/10.1016/j.scico.2016.05.002.
I have an existing MVC application which I am integrating a hub into, now I have setup the hub like so:
routeTable.MapHubs("myapp/chat/room", new HubConfiguration { EnableCrossDomain = true, EnableDetailedErrors = true, EnableJavaScriptProxies = true });
Then in the clientside I am connecting like so:
var connection = $.hubConnection(SystemConfiguration.ServiceUrl + "/myapp/chat/room", { useDefaultPath: false });
var hub = this.Connection.createHubProxy("ChatHub"); // Same name as on the hub attribute
connection.start().done(function(){ /* do stuff */});
Then I see the HTTP Request like so:
http://localhost:23456/myapp/chat/room/negotiate?_=1374187915970
Response Headers
Access-Control-Allow-Cred... true, true
Access-Control-Allow-Head... content-type, x-requested-with, *
Access-Control-Allow-Meth... GET, POST, PUT, DELETE, OPTIONS
Access-Control-Allow-Orig... http://localhost:34567, http://localhost:34567
Access-Control-Max-Age 10000
Cache-Control no-cache
Content-Length 420
Content-Type application/json; charset=UTF-8
Date Thu, 18 Jul 2013 22:52:18 GMT
Expires -1
Pragma no-cache
Server Microsoft-IIS/8.0
X-AspNet-Version 4.0.30319
X-Content-Type-Options nosniff
Request Headers
Accept application/json, text/javascript, */*; q=0.01
Accept-Encoding gzip, deflate
Accept-Language en-US,en;q=0.5
Content-Type application/x-www-form-urlencoded; charset=UTF-8
Host localhost:23456
Origin http://localhost:34567
Referer http://localhost:34567/myapp/chat?chatId=1764a2e3-ff6f-4a17-9c5f-d99642301dbf
User-Agent Mozilla/5.0 (Windows NT 6.2; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0
The response though contains no body, its got a 200 status though... I am debugging on the server and the hub methods are never hit. The only non standard thing in this scenario is that I have a custom CORS HttpModule which intercepts traffic and appends the CORS required headers, as you can see in the response, so not sure if this confuses SignalR's CORS support in some way. Anyway I can see the HttpModule being hit so it goes past there fine, but is somehow lost between there and the hub.
Tried googling but not much info on this topic...
The issue seems to be down to my CORS handling at HttpModule level, it must somehow conflict with SignalR... if I put a check in the module to see if the URL contains "chat/room" and just ignore the request if needed it then works fine, however it feels like a hack, but at least it works now.
This is the Google Apps Script that makes the request:
UrlFetchApp.fetch('https://user:password#sitename.com/api', {
method: 'put',
headers: { 'Accept': '*/*' },
payload: 'foo=bar&baz=qux'
});
Which can be successfully posted to http://requestb.in/, for examination:
PUT /1bfk94g1 HTTP/1.1
User-Agent: Mozilla/5.0 (compatible; GoogleDocs; script; +http://docs.google.com)
Host: requestb.in
Content-Type: application/x-www-form-urlencoded
Content-Length: 43
Connection: keep-alive
Accept-Encoding: gzip
Accept: */*
foo=bar&baz=qux
But fails when the request URL is https://user:password#sitename.com/api. The only error information shown is Bad request: https://user:password#sitename.com/api.
I've constructed a curl command which yields the exact same HTTP request:
curl -XPUT \
-H 'Accept-Encoding: gzip' \
--user-agent 'Mozilla/5.0 (compatible; GoogleDocs; script; +http://docs.google.com)' \
-d foo='bar' \
-d baz='qux'\
https://user:password#sitename.com/api
Which successfully posts to https://user:password#sitename.com/api. How can two identical requests have different outcomes? Am I missing something with regards to my Google Apps Script? I've tried using the debugger, but it didn't help.
It appears that UrlFetchApp doesn't yet support requests with credentials in the URL. The Authorization HTTP header needs to be built manually. The following works:
var response = UrlFetchApp.fetch('https://sitename.com/api', {
headers: {
'Authorization': 'Basic ' + Utilities.base64Encode(user + ':' + password)
}
});
For some reason the script below is not working.
This is the code I am using to generate the json data:
Protected Sub Page_Load(ByVal sender As Object, ByVal e As System.EventArgs)
Response.Clear()
Response.Write(generate_json_data())
Response.End()
End Sub
This produces the following output on screen:
[ {id:0,value:"c++"}, {id:1,value:"java"}, {id:2,value:"php"}, {id:3,value:"coldfusion"}, {id:4,value:"javascript"}, {id:5,value:"asp"}, {id:6,value:"ruby"} ];
This is the jquery I have so far, which does not seem to work. It does not give an error, when I type into the input field, nothing happens, when it should be displaying some of the data from the json data.
$("input").autocomplete({
source: "serverside_array.aspx",
dataType: "json",
select: function (event, ui) {
$("#txtAllowSearch").val(ui.item.value); // display the selected text
$("#txtAllowSearchID").val(ui.item.id); // save selected id to hidden input
}
});
EDIT 1: The header from the developer tools in chromium 10
Request URL:http://intranet/test_array.aspx?term=j
Request Method:GET
Status Code:200 OK
Request Headers
Accept:application/json, text/javascript, */*
Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.3
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-US,en;q=0.8
Connection:keep-alive
Content-Type:application/x-www-form-urlencoded
Host:intranet
Referer:http://intranet/rights_stage_three.aspx
User-Agent:Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.17 (KHTML, like Gecko) Chrome/10.0.652.0 Safari/534.17
X-Requested-With:XMLHttpRequest
Query String Parameters
term:j
Response Headers
Cache-Control:private
Content-Length:204
Content-Type:application/json; charset=utf-8
Date:Thu, 27 Jan 2011 16:11:14 GMT
Server:Microsoft-IIS/6.0
X-AspNet-Version:2.0.50727
X-Powered-By:ASP.NET
EDIT 2: looking at the reponse data XHR, I get the following
name test_array.aspx
method get
status 200 ok
type application/json
size 204b
time pending
EDIT 3:
Now I am totally confused. I got it working by simply changing the contenttype generated by the aspx page to:
text/xml
instead of
applicaiton/json
why does it work with text/xml when I am returning json?
Use a webdev tool like Firebug or Operas and Chromes built-in dev-tools which can list HTTP requests and responses to you.
Check if a JavaScript error is logged.
If not, check for the response, and that the responses Content-Type is text/javascript if it’s a callback, or application/json if only JSON data.
Without the content type the ajax request may fail because of security against XSS.
[ {id:0,value:"c++"}, {id:1,value:"java"}, {id:2,value:"php"}, {id:3,value:"coldfusion"}, {id:4,value:"javascript"}, {id:5,value:"asp"}, {id:6,value:"ruby"} ];
The semicolon at the end of the string should not be there. Also the Content-Type of the response should be application/json. And consider to put all values in double quotes. May be it will work without it, but it is not the proper JSON format without it. This is the standard compliant JSON string:
[{"id":0,"value":"c++"}, {"id":1,"value":"java"}, {"id":2,"value":"php"}]