I enabled http pipelining support in google chrome and observerd some problems in how data is received even when using big sites like amazon.com. What is the current support for pipelining from major servers ? I wonder if issues can be caused also because of our transparent proxy (microsoft TMG) although http://technet.microsoft.com/en-us/library/cc302548.aspx mentions that
"ISA Server does not implement pipelining. Client request pipelining is supported, allowing a client to make multiple requests without waiting for each response. However, pipelining when sending requests to the Web server is not supported." I think this should not cause data to be received wrongfully from the pipeline aware web server.
Related
To implement HTTP/2 support on nginx/1.11.1, I'm going to redirect all HTTP Request to HTTPS.
In this case, how will Bot and Browsers, that don't support HTTP/2 protocol, behave and render the page?
Is there a way for me to simulate HTTP/1.1 browser behavior on Chrome Developer Tools?
You are mixing two concepts here that are somehow related, but they are largely different: HTTP to HTTPs redirect, and HTTP 1.1 vs HTTP/2 negotiation.
Redirecting HTTP to HTTPS requests is fine. Virtually every client (browser, bot, etc) available these days is capable of understanding HTTPS requests.
As for HTTP 1.1 vs HTTP/2, Nginx will fallback to HTTP 1.1 if the HTTP/2 connection fails because the client doesn't support it.
Last but not least, this question has very little to do with StackOverflow. It is more appropriate in ServerFault or SuperUser.
Potentially interesting
TCP retransmission will increase. This could lead on poorly configured devices for connection abort.
Question
If 2 HTTP requests are made to the same server at the same time from fresh, e.g. GET /image1.png HTTP/1.1 & GET /image2.png HTTP/1.1 with no previous connection to the server. Then should 1 TCP connection be made or 2?
Info
Persistent connections supported by default in HTTP 1.1. HTTP 1.0 uses the Connection: Keep-Alive.
It seems pretty clear from reading the RFC that if the above requests are made one after each other then the second request should reuse the connection.
HTTP Pipelining is sending multiple requests down the same connection without first waiting for a response. I am not sure where this fit into the answer tho.
If 2 HTTP requests are made to the same server at the same time from fresh, e.g. GET /image1.png
If the requests are made by the browser simultaneously (and there is no HTTP proxy server), then there will be two connections made to the server (unless http pipelining is enabled). Per the wikipedia article on pipelining,
Out of all the major browsers, only Opera based on Presto layout engine had a fully working implementation that was enabled by default. In all other browsers HTTP pipelining is disabled or not implemented.
Internet Explorer 8 does not pipeline requests, due to concerns regarding buggy proxies and head-of-line blocking.
Mozilla browsers (such as Mozilla Firefox, SeaMonkey and Camino) support pipelining, however it is disabled by default. Pipelining is disabled by default to avoid issues with misbehaving servers. When pipelining is enabled, Mozilla browsers use some heuristics, especially to turn pipelining off for older IIS servers.
Konqueror 2.0 supports pipelining, but it's disabled by default.
Google Chrome supports pipelining for HTTP in the stable release as a non-default option (starting with version 18). There is no support for pipelining HTTPS yet.[11] As of version 26, the flag to enable HTTP pipelining in Chrome has been disabled.
So, probably two connections.
Web-servers work in response to incoming HTTP requests... process the request and return an HTTP response. Are there any common ways that a server can PUSH data to clients in this architecture... e.g a request comes in from client1 and the server wants to notify client2? It can obviously be done by a non-web server, using sockets, but what about a web-server app which has to support page requests AND allow PUSHing data..?
what about a web-server app which has to support page requests AND allow PUSHing data..?
Servlet 3.0 introduces Async support allowing to write Comet style applications (i.e. applications using Long-lived HTTP connections and either long polling or streaming).
If you can't wait for Servlet 3.0 Async support and don't want to use proprietary Comet or WebSocket support from containers (like GlassFish, Jetty), then have a look at Atmosphere.
See also
JavaOne 2008: Comet (AJAX, Grizzly and Cometd)
Asynchronous processing support in Servlet 3.0
Servlet 3.0 Async API or Atmosphere? A Simple Comparison
You can use web app containers like Jetty which support Web Sockets if you don't mind waiting for the web world to catch up to this up-and-coming standard. Then you'll have real bi-directional communications instead of HTTP + Polling or special plug-ins or the like.
No, not without some client side tech (Flash, Silverlight, Applets, etc.)
You could have the page poll the server with AJAX though.
Another possibility would be to abuse HTTP Keep Alive to achieve this. See http://en.wikipedia.org/wiki/HTTP_persistent_connection for some background. In your scenario you would have client2 initiate a connection to the server that then would stay open listening for notifications.
This is not a great solution, first off you need to keep lots of long lived TCP connections around, and if a connection is lost there is no way for the server to reconnect. It must wait for the client to come back.
I'm working on Comet support for CppCMS framework via long XMLHttpRequest polls. In many cases, such request is closed by client before any response from server was given -- for example the page is closed, user moves to other page or it is just refeshed.
At the server side I expect that I would recieve the notification that connection is dropped. I tested the application via 3 connectors: FastCGI, SCGI and simple HTTP Proxy.
From 3 major UNIX web servers, Apache2, lighttpd and Nginx, only the last one had closed
connection as expected allowing my application to remove the request from wait queue -- this worked for both FastCGI and HTTP Proxy connectors. (Nginx does not have scgi module by default).
Others, Apache and Lighttpd do not close connection or inform the backend about disconnected
clients, the proceed as if the client is still on line. This happens for all 3 supported APIs: FastCGI, SCGI and HTTP Proxy.
I had opened an issue for Lighttpd, but what
more conserns me is the fact that Apache -- mature and well supported web server as lighttpd
and does not discloses the server backend that client had gone.
Questions:
Is this a bug or this is a feature? Is there any reason not to close the connection between web server and application backend?
Are there real life Comet application working behind these servers via FastCGI/SCGI/HTTP-Proxy backends?
If the above true, how do they deal with this issue? I understand that I can timeout all connections every 10 seconds, but I would like to keep them idle as far as client listens -- because this allows easier scale up -- each connection is very cheep -- the cost is only the opended socket.
Thanks!
(1) Feature. Or, more specifically, fallout from an implementation detail.
A TCP/IP connection does not involve a constant flow of traffic back and forth. Thus, there is no way to know that a client is gone without (a) the client telling you it is closing the connection or (b) a timeout.
(2) I'm not specifically familiar with Comet or CppCMS. But, yes, there are all kinds of CMS servers running behind the mentioned web servers and they all have to deal with this issue (and, yes, it is a pain).
(3) Timeouts are the only way, but you can mitigate the pain, so to speak. Have the client ping the server across the connection every N seconds when there is otherwise no activity. Doesn't have to do anything and you can tack stuff on the reply; notifications of concurrent edits or whatever you need.
You are correct in that it is surprising that mod_fastcgi doesn't support telling the backend that Apache has detected the disconnect or the connection timed out. And you aren't the first to be dismayed.
The second patch on this page should fix that particular issue:
http://osdir.com/ml/web.fastcgi.devel/2006-02/msg00015.html
http://ncannasse.fr/blog/tora_comet
I don't have any concrete information for you, but this article does mention that they can detect when the client has disconnected from Apache. See tora.Queue. And it sounds like the source is available in the neko CVS, so you might be able to find some clues there. Good luck.
Could somebody give me a brief overview of the differences between HTTP 1.0 and HTTP 1.1? I've spent some time with both of the RFCs, but haven't been able to pull out a lot of difference between them. Wikipedia says this:
HTTP/1.1 (1997-1999)
Current version; persistent connections enabled by default and works well with proxies. Also supports request pipelining, allowing multiple requests to be sent at the same time, allowing the server to prepare for the workload and potentially transfer the requested resources more quickly to the client.
But that doesn't mean a lot to me. I realize this is a somewhat complicated subject, so I'm not expecting a full answer, but can someone give me a brief overview of the differences at a bit lower level?
By this I mean that I'm looking for the info I would need to know to implement either an HTTP server or application. I'm mostly looking for a nudge in the right direction so that I can figure it out on my own.
Proxy support and the Host field:
HTTP 1.1 has a required Host header by spec.
HTTP 1.0 does not officially require a Host header, but it doesn't hurt to add one, and many applications (proxies) expect to see the Host header regardless of the protocol version.
Example:
GET / HTTP/1.1
Host: www.blahblahblahblah.com
This header is useful because it allows you to route a message through proxy servers, and also because your web server can distinguish between different sites on the same server.
So this means if you have blahblahlbah.com and helohelohelo.com both pointing to the same IP. Your web server can use the Host field to distinguish which site the client machine wants.
Persistent connections:
HTTP 1.1 also allows you to have persistent connections which means that you can have more than one request/response on the same HTTP connection.
In HTTP 1.0 you had to open a new connection for each request/response pair. And after each response the connection would be closed. This lead to some big efficiency problems because of TCP Slow Start.
OPTIONS method:
HTTP/1.1 introduces the OPTIONS method. An HTTP client can use this method to determine the abilities of the HTTP server. It's mostly used for Cross Origin Resource Sharing in web applications.
Caching:
HTTP 1.0 had support for caching via the header: If-Modified-Since.
HTTP 1.1 expands on the caching support a lot by using something called 'entity tag'.
If 2 resources are the same, then they will have the same entity tags.
HTTP 1.1 also adds the If-Unmodified-Since, If-Match, If-None-Match conditional headers.
There are also further additions relating to caching like the Cache-Control header.
100 Continue status:
There is a new return code in HTTP/1.1 100 Continue. This is to prevent a client from sending a large request when that client is not even sure if the server can process the request, or is authorized to process the request. In this case the client sends only the headers, and the server will tell the client 100 Continue, go ahead with the body.
Much more:
Digest authentication and proxy authentication
Extra new status codes
Chunked transfer encoding
Connection header
Enhanced compression support
Much much more.
HTTP 1.0 (1994)
It is still in use
Can be used by a client that cannot deal with chunked
(or compressed) server replies
HTTP 1.1 (1996- 2015)
Formalizes many extensions to version 1.0
Supports persistent and pipelined connections
Supports chunked transfers, compression/decompression
Supports virtual hosting (a server with a single IP Address hosting multiple domains)
Supports multiple languages
Supports byte-range transfers; useful for resuming interrupted data
transfers
HTTP 1.1 is an enhancement of HTTP 1.0. The following lists the
four major improvements:
Efficient use of IP addresses, by allowing multiple domains to be
served from a single IP address.
Faster response, by allowing a web browser to send multiple
requests over a single persistent connection.
Faster response for dynamically-generated pages, by support for
chunked encoding, which allows a response to be sent before its
total length is known.
Faster response and great bandwidth savings, by adding cache
support.
For trivial applications (e.g. sporadically retrieving a temperature value from a web-enabled thermometer) HTTP 1.0 is fine for both a client and a server. You can write a bare-bones socket-based HTTP 1.0 client or server in about 20 lines of code.
For more complicated scenarios HTTP 1.1 is the way to go. Expect a 3 to 5-fold increase in code size for dealing with the intricacies of the more complex HTTP 1.1 protocol. The complexity mainly comes, because in HTTP 1.1 you will need to create, parse, and respond to various headers. You can shield your application from this complexity by having a client use an HTTP library, or server use a web application server.
A key compatibility issue is support for persistent connections. I recently worked on a server that "supported" HTTP/1.1, yet failed to close the connection when a client sent an HTTP/1.0 request. When writing a server that supports HTTP/1.1, be sure it also works well with HTTP/1.0-only clients.
One of the first differences that I can recall from top of my head are multiple domains running in the same server, partial resource retrieval, this allows you to retrieve and speed up the download of a resource (it's what almost every download accelerator does).
If you want to develop an application like a website or similar, you don't need to worry too much about the differences but you should know the difference between GET and POST verbs at least.
Now if you want to develop a browser then yes, you will have to know the complete protocol as well as if you are trying to develop a HTTP server.
If you are only interested in knowing the HTTP protocol I would recommend you starting with HTTP/1.1 instead of 1.0.
HTTP 1.1 is the latest version of Hypertext Transfer Protocol, the World Wide Web application protocol that runs on top of the Internet's TCP/IP suite of protocols. compare to HTTP 1.0 , HTTP 1.1 provides faster delivery of Web pages than the original HTTP and reduces Web traffic.
Web traffic Example: For example, if you are accessing a server. At the same time so many users are accessing the server for the data, Then there is a chance for hanging the Server. This is Web traffic.
HTTP 1.1 comes with the host header in its specification while the HTTP 1.0 doesn't officially have a host header, but it doesn't refuse to add one.
The host header is useful because it allows the client to route a message throughout the proxy server, and the major difference between 1.0 and 1.1 versions HTTP are:
HTTP 1.1 comes with persistent connections which define that we can have more than one request or response on the same HTTP connection.
while in HTTP 1.0 you have to open a new connection for each request and response
In HTTP 1.0 it has a pragma while in HTTP 1.1 it has Cache-Control
this is similar to pragma