I do understand the basic idea of it. But most of the implementations I've seen have done nothing but confused me - I find myself incapable of fully understanding the concept of Comet and long polling... Simply put, I ask for simple explanation of these ideas. I am especially interested in an explanation of the hidden iframe polling technique. What gets executed, what gets requested etc.
Also, what are the advantages of it over the classic ajax approach? (besides the reduced traffic and more real-time feeling).
Thanks.
The technique is very nicely explained in the following article. The core idea resides on the chunked transfer encoding HTTP technique. A hidden iframe is included in the page which points to a server side script which uses chunked encoding. In chunked encoding the response is not sent entirely in one go and the stream closed. The server doesn't say in advance how much data is going to send so the browser keeps the channel open. Then when the server wants to push some data to the client it simply sends a chunk of response which represents a javascript function. The browser receives and executes this function on the client. This way the server can successfully PUSH information when some events occur like for example some data changes on the server, ...
Also, what are the real advantages of it over the classic ajax
approach? (besides the reduced traffic and more real-time feeling).
Aren't those advantages sufficient? Reduced traffic means more responsive application. Did you know that large sites like Google and Amazon conducted studies and explicitly throttled down their servers in order to increase the response time with a couple of milliseconds. I can't remember the exact but they were flagrant: they lost like 70% of their customers after doing that. Remember: the most important feature of a web application (and not only by the way) is its responsiveness.
So it's basically PULL (Ajax) vs PUSH (Comet). PUSH techniques scale better when the number of clients starts to increase.
Related
I have this scenario where there are multiple Help buttons which may or may not be clicked. Now, I am very confused to send all information(including the text after user click help buttons) at once, or every time user clicks on one of help button make the API call and send the Response.
What I see is there can be case of Network Overload when you make Multiple calls but at the same time my seniors suggest its for -
1) API Best Practices
2) Design Practices and
3) API splitted into Granular Level.
I couldnt find any convincing answer for this anywhere and found only Tutorials for "How to make an API ?"
It depends on the expected network latency (ping time) between your clients and your server. In a high-latency situation with otherwise good bandwidth (e.g. mobile apps), many small requests will perform significantly worse than one large one.
Also, having one big request can help with
better efficiency for compressing the response;
avoiding the overhead of the extra HTTP requests and response headers.
I know I should probably use WebSockets or Server-Side Events, but what happened to the Comet Streaming techniques that involved writing chunked data from a server to either an iFrame or as a response to a xmlHttpRequest? I have stumbled upon multiple demos but none of them work as intended any more and since most of the material on this type of streaming is quite old, I'm wondering whether or not it is still doable in the year of 2015?
Just to be clear, I'm referring to the Comet techniques where the server keeps a connection open using chunked transfer encoding flushing new data on the fly. Incremental rendering in browsers should presumably make this data available, either as it comes in in the case of an iFrame (e.g. "Forever iFrame") or by reading the responseText property of the xmlHttpRequest object when its readyState returns 3 (e.g. "XHR streaming"). However, all browsers seem to buffer the data until the connection is closed, no matter how much bogus preamble I add before starting to send real data. Also, I'm not referring to the special case with Content-Type set to multipart/x-mixed-replace which works in Firefox only, but an approach which seems to have worked on most of the browsers a few years back.
Does anyone know if current browser behaviour has obsoleted these Comet streaming techniques?
Example demos:
Polling responseText on load through xmlHttpRequest object:
http://ajaxify.com/run/streaming/xmlHttpRequest/countdown/
http://ajaxify.com/run/streaming/xmlHttpRequest/
Slow loading of regular page, incremental rendering does NOT kick in for me:
http://ajaxify.com/run/streaming/
Polling iFrame content on slow iFrame load:
http://ajaxify.com/run/streaming/xmlHttpRequest/iframe/
These demos does not use chunked transfer encoding but resulting behaviour is the same, that is, incremental rendering isn't happening.
Article on Comet streaming:
http://cometdaily.com/2007/11/05/the-forever-frame-technique/
Short version conclusion:
All the Comet techniques still work, however, in my case the antivirus got in the way of things in an unforeseen manner so make sure to test things on different computers and in safe mode as well to be sure if it doesn't work!
I am starting to work on a project where I need to stream Twitter data using PowerTrack/GNIP and I have to be honest when I say I am very very inexperienced when it comes to networks and I have absolutely no knowledge when it comes to Data Stream (HTTP), how they work etc.
Are there any resources out there that go through all of this in simple terms? I would love to be able to map Data streaming process in my head before I start looking at APIs etc.
Thanks
Take a look at the following two resources which give a good overview of video streaming. Video streaming has probably more background available and should help you understand the concepts:
https://developer.apple.com/library/ios/documentation/NetworkingInternet/Conceptual/StreamingMediaGuide/Introduction/Introduction.html
http://www.jwplayer.com/blog/what-is-video-streaming/
In very simple terms, streaming breaks a large file or live stream into chunks, and sends those chunks one after another to a client (e.g. browser). The client can generally request a start point for content which is not a live stream. In the background this generally works by the client sending requests for each individual chunk (rather than just one request with multiple responses).
The advantage of the multiple request approach is that you know the client is actually still interested (e.g. the user has not browser to another page etc) and for video and audio etc the client can dynamically request different bandwidth files depending on the current network connection - see: http://en.wikipedia.org/wiki/Adaptive_bitrate_streaming.
Twitter do have a streaming page also, but you have probably already seen this:
https://dev.twitter.com/streaming/overview
I have multiple AJAX requests going out of my browser.
My UI is comprised of multiple views and the AJAX requests are trying to populate those views simultaneously. In some cases I require more than 10 simultaneous requests to be sent from client and processed concurrently at the server.
But due to browser limitations on max concurrent requests to a single domain and because of HTTP's "A server MUST send its responses to requests in the same order that the requests were received" constraint, I am not deriving as much concurrency in request processing as I would want.
From my application's standpoint, I dont need responses to come in the order in which I sent the request. I am ok if view8 gets populated before view1, for example.
Async processing using Servlet 3.0 constructs seems to address only one-side of the problem (the Server-side) and hence cannot be fully exploited for maximizing application concurrency.
My question is:
Am I missing out on some proper constructs ? ('proper' in contrast to workarounds like "host your images from a different sub domain") that can yield me more concurrency ?
This seems like something many web UIs would need ! If not, then I am designing my UI the wrong way. In either case, I would appreciate your inputs.
Edit1: To my advantage, I dont have to support a huge number of concurrent clients. The maximum number of concurrent clients accessing the app would be < 100. Given that fact, basically am trying to enhance the experience of these clients when I have the processing power available aplenty on my server-side.
Edit2: Our application/API is not for 'public' consumption. For ex: It is like my company's webmail app. It is hosted on the internet but it is not meant for everyone's consumption. Only meant for consumption by the relevant few.
The reason why am giving that info, is to differentiate my app from SO/Twitter, which seem to differentiate their (REST) API users from their normal website users. In our case, we think we should not differentiate that way and want to provide single-set of REST endpoints for both.
The reason behind the limitation in the spec (RFC2616) seems to be : "These guidelines are intended to improve HTTP response and avoid congestion.". However, intranet web apps have more luxuries and should not have to be so constrained !?
The server is exposing REST API and hence the UI makes specific GETs
for various resource catogories (ex: blogs, videos, news, articles).
Since each resource catogory has its exclusive view it all fits in
nicely. It feels wrong to collate requests to get blogs and videos
together in one request. Isnt it ?
Well, IMHO being pragmatic is more important. Sure, it makes sense for a service to expose RESTful API but it's not always necessary to expose the entire API to the browser. Your API can be separate from your server side web app. You can always make those multiple API requests on the server side, collate the results and send them back to the client. For e.g. look at the SO home page. The StackOverflow API does expose a RESTful API but when loading the home page the browser doesn't send across multiple requests just to populate the tags, thread listing etc.
Thanks Sanjay for the suggestion. But we wanted to have a single-API
for both REST clients and Browser clients. Interestingly, the root URI
"stackoverflow.com" is not mentioned in SO's REST API, but the browser
client uses it. I suppose if they had exposed the root URI, their
response would be difficult to process (as it would be a mixture of
data). Their REST API is granular (as is in my application), but their
javascript code uses some other doors(APIs) to decrease no. of
round-trips to the server! Somehow that doesnt feel right (Am a novice
in this field though). Feel free to correct me
SO doesn't use any "other doors". It's just that they simply don't send across 10 concurrent requests for populating something on the page. They make XHR request when you vote, mark thread as favorite, comment etc. For loading the page itself, there are no multiple requests. If you want to directly hit your RESTful API from the browser, you'll have to honor the limitations. Either that or go the desktop way which allows you virtually unlimited connections to your server but I guess you don't want to go that route...
It is quite easy to update the interface by sending jQuery ajax request and updating with new content. But I need something more specific.
I want to send the response to client without their having requested it and update the content when they have found something new on the server. No need to send an ajax request every time. When the server has new data it sends a response to every client.
Is there any way to do this using HTTP or some specific functionality inside the browser?
Websockets, Comet, HTTP long polling.
It has name server push (you can also find it under name Comet technology). Do search using these keywords and you will find bunch examples, tools and so on. No special protocol is required for that.
Aaah! You are trying to break the principles of the web :) You see if the web was pure MVC (model-view-controller) the 'server' could actually send messages to the client(s) and ask them to update. The issue is that the server could be load balanced and the same request could be sent to different servers. Now if you were to send a message back to the client you'll have to know who all are connected to the server. Let's say the site is quite popular and you have about 100,000 people connecting to it every day. You'll actually have to store the IPs of each of them to know where on the internet they are located and to be able to "push" them a message.
Caveats:
What if they are no longer browsing your website? You see currently there is no way to log out automatically if you close your browser. The server needs to check after a fixed timeout if you have logged out (or you send a new nonce with every response to prevent the server from doing that check)
What about a system restart/crash etc? You'd lose all the IPs that you were keeping track of and you are back to square one - you have people connected to you but until you receive new requests you can't really "send" them data when they may be expecting it as per your model.
Let's take an example of facebook's news feeds or "Most recent" link close to the top right - sometimes while you are browsing your wall you see the number next to most recent has gone up or a new 'feed' has come to the top of your wall post! It's the client sending periodic requests to the server to find out what was updated rather than the other way round
You see, it keeps it simple and restful. You may feel it's inefficient for the client to "poll" the server to pull the data and you'd prefer push, but the design of the server gets simplified :)
I suggest ajax-pulling is the best way to go - you are distributing computation to the client and keeping it simple (KIS principle :)
Of course you can get around it, the question is, is it worth it?
Hope this helps :)
RFC 6202 might be a good read.