I'm trying to write a WP8 app that needs to upload a large amount of data back to my server. My server runs on ASP.net and implements REST using WebAPI.
I've gotten to a point where I can upload small amount of data, say 2-5MB using a POST and transfer them over to Azure blobs. Now, I'm thinking about moving a decent amount of data, say ~40-50MB from the phone using the background transfer API defined here http://msdn.microsoft.com/en-us/library/windowsphone/develop/hh202955(v=vs.105).aspx
The phone API supports -
Over cellular connection - 5 MB,
Over Wi-Fi connection with battery power - 20 MB
Over Wi-Fi connection with external power - 100 MB
The part that I'm struggling to understand is -
The MSDN kb article recommends that the server implement range requests, which is fair. However, it doesn't say how much could be chunked at a time. Can my upload server config be unbounded for request size?
I would prefer to keep the client as 'dumb' as possible and use the existing transfer APIs on the phone. My concerns are around performance of my server and how much memory would be available on the server if I start seeing considerable traffic on the server. Can someone give me pointers for server best practices to accept large amounts of data?
BackgroundTranferRequest does not support the Range header. see here
Implement your own upload and download client and you can use Range and also bypass the various size limitations.
Related
After knowing about some great features of WebRTC, I thought of using WebRTC one to one audio/video calls in my web application. The web application is for many organizations/entities of a category who can register and keep recording several records daily for their internal working and about their clients. The clients of these individual organizations/entities also have access to the web application to access their details.
The purpose of using WebRTC is for communication between clients and organizations. Also for daily inquires by new people to these organizations about products and services.
While going through articles on google etc. I found broadcasting or one to many calls requires very high bandwidth to users if we don't make use of Media Server.
So what is the case for one to one calls?
Will it affect the performance of web application or bring any critical situation if several users are making audio/video calls(one to one) to each other simultaneously as a routine?
The number of users will be very large and users will be recording daily several entries as their routine work. But still it is manageable and application will be running smoothly but I am not sure about the new concept WebRTC. Will it require a very high hosting plan? Is using WebRTC for current scenario suitable or advisable?
WebRTC by its nature is Peer-to-Peer. Meaning that the streaming data is handled CLIENT side. All decoding, encoding, ICE candidate gathering/negotiation, and media encrypting/transmitting will happen on the client side and not on server side. So, you will be providing the pages, client side JS, and some data exchange(session negotiation signalling) but all in all, it is not a huge amount of work. It should be easily handled without having to worry about your host machine being over worked.
All that said, here are the only a performance concerns that would POSSIBLY affect your hosting server.
Signalling session startup, negotiations, and tare down. This is very minimal(only some json data at the beginning of a session). This should not be too much of a burden but you should be aware that if 1000 sessions start at the same time, you will have a queue of messages to direct to the needed parties. How you determine the parties, forward the messages, and what work you do server side could all affect performance. If written smartly(how to store sessions, how to forward messages, etc.) should not be a terrible burden.This could easily done with SignalR since you are on ASP.NET or you could use a separate one running Node.js(or the same box, does not matter) if you so desired.
RTP TURN relay if needed. This will probably be through a different server(or the same one as your hosting server if you want). For SOME connections, a TURN server is needed and any production ready WebRTC solution should take this into account. Here is a good open source turn server. Bandwidth usage here could be very high as RTP packets are sent to this server and the forwarded to the peer in the connection.
If you are recording the streams, you may have increased hosting traffic depending on how you implement it. Firefox supports client side recording of the streams but Chrome does not(they say it is in the works currently). You could use existing JS libraries to record the feeds client side and then push them anywhere you want. You could also push all the data through a MediaServer that will mux, demux, and forward the data to be recorded anywhere you like. Janus-Gateway videoroom is a good lightweight example of a mediaserver.
Client side is a different story.
There are higher level concerns in the Javascript. If you use one of the recording JS libraries, this is especially evident as they do canvas captures numerous times a second which are a heavy hit and would degrade the user experience.
CPU utilization by the browser will increase as the quality of the video being streamed increases. This is rather obvious as HD video frames take more CPU power to encode/decode than SD frames.
Client side bandwidth usage can also be an issue. Chrome and Firefox try to modify the bitrate of each video/audio feed dynamically but the video Bitrate can go all the way up to 2 Mbps. You can cap this in Chrome( by adding an attribute in the SDP) but not in Firefox(last I checked) as of yet.
I've a scenario where there're two servers(server 1 & server 2). There's a web cam/CCTV cam(or of any kind) that feeds video to server 1. I assume the video feeding to the server 1 is easily done by the camera setup. This live video is now somehow sent to Server 2 which is eventually broadcast to the clients.
Now the thing is I actually need a lead to follow how to start with the whole thing and where does media server's role kick in(if it's actually needed). I don't have any idea whatsoever regarding the whole process and am having trouble making relevant searches. Any advise or help would be much appreciated. Thanks in advance
is there any specific reason why 2 servers are involved ? You could easily stream video from server 1, where you get the camera feed.
You could either use a streaming media server (Like adobe media server) or use a standalone application like Windows Media Encoder to give out the live stream for users to view.
Does your server 1 have enough bandwidth to stream the video to multiple users ? If 100 people view your stream at 1 Mbps, you will be needing around 20Mbps bandwidth in your server at a minimum, else the video streams may suffer. If you cannot arrange that much bandwidth in your server, you will have to use a CDN hosted streaming server (There are lot of service providers).
If only a few users will be viewing your stream simultaneously, it may be fine with your existing setup.
If you are following the two server setup as you mentioned, follow these steps to broadcast.
Set up Adobe media server (trial will do for upto 10 simultaneous connection streams) on server 2.
Install Adobe media encoder on server 1, where the video stream is available.
From server 1, push the video stream via adobe media encoder to server 2, (set up a publishing point first for live).
get the streaming link from AMS installed in server 2, which can be embedded into any compatible player (flowplayer or jwplayer), and put it in a webpage for public access.
Hope this helps.
I was designing a video management system for multiple clients where the recording server is supposed to record video clips to its local drive and then whenever the client wants to playback these recorded videos, it should be shared with the client. There seem to be two approaches to this:
Use windows file sharing protocol like SMB/CIFS to share the drive to all the clients - windows permits up-to 20 clients, once the drive is shared it will be mapped by every client as a local drive and client can view whichever file is to be played back
Use a streaming server and based on client requests, stream (HTTP/RTSP streaming) a particular file that is required by a client (separate thread to listen and stream required files to each client), in this case server management would become tedious as the number of clients increase
Can anyone let me know the pros and cons of each of these approaches, so that I can design an efficient architecture for the same? Most video management software I have seen, seem to be using the second approach, any specific advantages in doing that?
Regards,
Saurabh Gandhi
Consider a web application such as Google Chat, where the servers serve hundreds of millions of clients simultaneously. In such application, the servers have to push notifications to clients at near real time (in the chat example - incoming messages, presence notification etc.).
How do they implement it? Significant part of the clients are browser based. I suppose polling would overload even Google's servers. So, are they using something like Comet? If so - do they need to allocate a server for every 65536 clients (maximum TCP connections per machine)? I understand that there is a way to circumstance this limitation but I don't know how it's implemented.
Chat is not handled by single application / hardware / instance.
They definitely using many instances with load balancing that allows to scale chat system horizontally. It might be dedicated for regions or just single clustered system (I believe it is dedicated within regions but still clustered within region).
As well you can have as many connections as hardware and network will handle but not 64k.
Because 64k (actually less then that) is regarding Binding sockets (server sockets, but not client ones).
In case with google and based on supported browsers, they definitely use mixed technologies to communicate selecting the most powerful based on browser support. That can be long-polling, sockets and even oldest one: simple ajax.
As well for example facebook chat is based on erlang. And using erlang there is many examples having more then million connections.
I don't know how Google handle this, ans they probably won't tell us. But, today you can deal with http streaming, websockets or long polling to build such application. To give you an example Atmosphere framework
is a tool to build "real-time", efficient and scalable web application.
I would like to create a browser-based network game. To ensure it can be used by as many people as possible, I'd like to embed all the traffic in standard HTTP packages.
Assuming I use IIS as my back end, how should I code this to minimize latency?
Is it reasonable to start with an ASP application of some kind (ASP.NET MVC perhaps) using shared state in memory? Or should I plan from the outset on writing some kind of IIS plugin of my own? Or should I abandon IIS and write a custom server instead?
Is it reasonable to start with each client repeatedly querying, say ten times per second, or should I make the data more stream-like somehow (and if so how)?
This can work just fine, but you're going to have to do something less "conventional."
To make this work, don't do individual requests, keep the request open and then transmit data with the open connection.
You could try using a protocol like Bayeux, but there are no rules here.
Here's an example with ASP.NET using COMET.
Designing an app to hit a web server 10 times a second is not a good idea. Web servers are designed for less frequent client requests and probably larger processing times and reponse sizes than your game will be using. That's not to say a web server wouldn't be able to cope just that it would not be an efficient client-server match.
For any type of app that requires multiple packets per second you really should think about a lighter protocol than HTTP which is fairly verbose. For example if your game needs to send 4 bytes to register a user's battleship co-ordinates you don't really want to transmit an extra few hundred bytes of HTTP headers.
I'd recommend a browser plugin technology like Siverlight of Flash. Both of those support TCP socket connections. You would need to write your own server to sit at the other end of the TCP socket. With that approach you'd also have the advantage of being able to push data out to the clients without having to rely on client polling mechanisms which are required with HTTP, e.g. AJAX.
One problem you will face with standard HTTP messages is the size (and lack of control) over the header data.
I was involved in the design of a flash game which was played by several million people. We needed to communicate with the server every few seconds and ended up using ultra-lightweight JSON messages to save on bandwidth and keep it snappy.
Size of JSON message: 16 bytes
Size of HTTP Header: 200+ bytes
HTTP is not really a good protocol for high traffic, low latency requirements. It was designed for larger, less frequent request/response models and has status codes like 304 Not Modified for this very reason.
You'll probably want to drop down to a custom TCP/IP implementation.