I would like to stream data from the client to the server.
My application streams audio data to the server. I do not know how long the audio will be when I begin streaming it. I want to reduce latency by transmitting the data as it is being recorded. Once all the data has been uploaded, then I will process it.
So, what I would like is a HTTP POST where the body is streamed. At the client, the POST would be sent as the data is available. At the server end, I would like it to arrive like a normal POST with a complete body of collected data.
I am currently using Restlet, and implementing my stream as a Chunked POST to the Restlet framework.
However, I can not find a client API that allows me to begin the POST, then start streaming the data.
I haven't found anything useful searching the archive. Can anyone give me a pointer here. Can HTTP POST be used this way? Can I use Restlet for this? Is there another standard pattern/API for streaming from the client to the server?
Many thanks in advance
Peter
Going to answer my own question for those that come after.
Here is a nice tutorial on streaming requests with Apache HttpClient
Search for "Request Streaming:"
http://hc.apache.org/httpclient-3.x/performance.html#Request_Response_entity_streaming
I can not find a client API that allows me to begin the POST
You could always use a browser. Serverside it's just the same a a file upload.
Related
I have to give seminar on Requests and Requests-html. I am searching that but can't find any website. Both Requests and Requests-html has same methods but what is the difference
Requests-HTML helps you to parse contents of a webpage (aka web-scraping). You can connect to a webpage and parse its contents like links, raw data, search for specific terms. Generally, it is used for data analytical purpose and requires less technical expertise than requests.
Requests helps you to make HTTP calls programatically. You can send GET/POST et al requests just like curl commands and receive response to be processed by certain logic. Generally backend API developers use it and requires technical knowledge of how HTTP works.
I am starting to work on a project where I need to stream Twitter data using PowerTrack/GNIP and I have to be honest when I say I am very very inexperienced when it comes to networks and I have absolutely no knowledge when it comes to Data Stream (HTTP), how they work etc.
Are there any resources out there that go through all of this in simple terms? I would love to be able to map Data streaming process in my head before I start looking at APIs etc.
Thanks
Take a look at the following two resources which give a good overview of video streaming. Video streaming has probably more background available and should help you understand the concepts:
https://developer.apple.com/library/ios/documentation/NetworkingInternet/Conceptual/StreamingMediaGuide/Introduction/Introduction.html
http://www.jwplayer.com/blog/what-is-video-streaming/
In very simple terms, streaming breaks a large file or live stream into chunks, and sends those chunks one after another to a client (e.g. browser). The client can generally request a start point for content which is not a live stream. In the background this generally works by the client sending requests for each individual chunk (rather than just one request with multiple responses).
The advantage of the multiple request approach is that you know the client is actually still interested (e.g. the user has not browser to another page etc) and for video and audio etc the client can dynamically request different bandwidth files depending on the current network connection - see: http://en.wikipedia.org/wiki/Adaptive_bitrate_streaming.
Twitter do have a streaming page also, but you have probably already seen this:
https://dev.twitter.com/streaming/overview
It is quite easy to update the interface by sending jQuery ajax request and updating with new content. But I need something more specific.
I want to send the response to client without their having requested it and update the content when they have found something new on the server. No need to send an ajax request every time. When the server has new data it sends a response to every client.
Is there any way to do this using HTTP or some specific functionality inside the browser?
Websockets, Comet, HTTP long polling.
It has name server push (you can also find it under name Comet technology). Do search using these keywords and you will find bunch examples, tools and so on. No special protocol is required for that.
Aaah! You are trying to break the principles of the web :) You see if the web was pure MVC (model-view-controller) the 'server' could actually send messages to the client(s) and ask them to update. The issue is that the server could be load balanced and the same request could be sent to different servers. Now if you were to send a message back to the client you'll have to know who all are connected to the server. Let's say the site is quite popular and you have about 100,000 people connecting to it every day. You'll actually have to store the IPs of each of them to know where on the internet they are located and to be able to "push" them a message.
Caveats:
What if they are no longer browsing your website? You see currently there is no way to log out automatically if you close your browser. The server needs to check after a fixed timeout if you have logged out (or you send a new nonce with every response to prevent the server from doing that check)
What about a system restart/crash etc? You'd lose all the IPs that you were keeping track of and you are back to square one - you have people connected to you but until you receive new requests you can't really "send" them data when they may be expecting it as per your model.
Let's take an example of facebook's news feeds or "Most recent" link close to the top right - sometimes while you are browsing your wall you see the number next to most recent has gone up or a new 'feed' has come to the top of your wall post! It's the client sending periodic requests to the server to find out what was updated rather than the other way round
You see, it keeps it simple and restful. You may feel it's inefficient for the client to "poll" the server to pull the data and you'd prefer push, but the design of the server gets simplified :)
I suggest ajax-pulling is the best way to go - you are distributing computation to the client and keeping it simple (KIS principle :)
Of course you can get around it, the question is, is it worth it?
Hope this helps :)
RFC 6202 might be a good read.
In general chat application, client's browser always poll to server to check for new messages.
// the function to check new messages in server
function check(){
// but this question is less about jQuery.
$.ajax({
type: "POST",
url: "check.aspx",
data: "someparam=123",
success: function(msg){
// process msg here
// CHECK IT AGAIN, but sometimes we need to make delay here
check();
}
});
}
Then I read Nicholas Allen's blog about Keeping Connections Open in IIS.
It makes me think if it is possible to push data from my server to client's browser by transfer chunked HTTP (it means like streaming, right?) and keep the connection open.
while keeping the connection open, in server, I have idea to keep something run to check new messages. something like this, maybe
while(connectionStillOpen) {
// any new message?
if( AnyMessage() )
{
// send chunked data, can I?
SendMessageToBrowser();
// may be we need to make delay here
Sleep(forSomeTime);
}
}
that's a raw idea.
My Chat App created in ASP.net. With my less understanding of WCF and advanced IIS streaming module, I need your advice about to implement this idea.
yea, Impossible is probably the answer. But I need to know why if its still impossible.
UPDATE (3 years later):
This is exactly what I was looking for:
Microsoft ASP.NET SignalR
Yes, it's impossible to push data from server directly to your browser client.
But you can to check server for new messages every, let's say, 3 seconds and refresh your client interface.
Maybe you want to take a look on some Comet implementations
A server cannot initiate communication with the client. So the server cannot push data to the client. But you can achieve the push mechanism using "Reverse AJAX". The following article should shed more light.
Reverse AJAX
One method is there which is called Reverse AJAX. By using which server can transfer data to client without any request from the Client.
The current generation of JavaScript / Ajax libraries don't provide access to partial responses; you only receive notification when the entire request is complete.
If you're open to using Silverlight, you can use a raw TCP connection.
Comet is another option -- but that's basically just long polling that still originates from client-side script.
Its not possible to push the data from the server. Because HTTP respond only to the requests and cannot contact the client directly. But we have a workaround here called COMET or ReverseAJAX, by using this technique we can simulate the duplex calls.
Its nothing but the long living AJAX
calls, and will respond to the client
if there is a expected event happening
in server side, otherwise it stays
calm. This Comet (programming)
wikipedia article explains more about
the approach
I have answerd similar question here asp-net-chat-with-wcf. pls check out
Scenario:
localhost receives the current HttpRequest with 3 hidden inputs and a posted file. I must then forward this form data to an external image host and get the response.
See the System.Net.WebClient and related classes. You can use them to create a request to the remote server and handle the response. Also get Fiddler to help you replicate what the browser sends.
I hate doing this. It wastes my server's bandwidth and ties up IIS threads as well as using my server's CPU. It sucks and it's worth avoiding at all cost. Many services like, one that comes to mind is fliqz, provide a mechanism such that the files are uploaded directly from the client to their server (bypassing yours) and then they make a request to your server passing it various info on the query string.