Currently I am looking at different CDN networks to bring our site, VOD and the VOD content to a CDN based solution.
Requirements:
a) A Webserver (Apache, Mysql) (not necessarily) and
b) A Streaming Server (preferably Wowza), which has to have the VOD Content cloud-based,
meaning edge-oriented. (necessary!)
The reason I need Wowza is because it covers a very big spectrum of streams for end-devices,
and security, which we need. (and please don't suggest Amazon streaming, because it's flash)
The problem in general we had with amazon etc. is, that they do offer a good CDN network, but a) I didn't find their price strategic quite easy to see through, and b) - which is more important - they do not (!) offer to use Wowza on their cloud / edge oriented. Therefore it doesn't really make sense to me to use Wowza with Amazon.
My question is: Are there other good CDN networks (like edgecast etc.) that you can recommend? Akamai seems quite expensive. Edgecast I asked already (they'll come back to me yet).
I am happy for all answers. Thank you!
Edgecast doesn't offer Wowza streaming, but you may want to check with some of the folks listed at http://www.wowza.com/partners.
With Amazon, you're essentially rolling your own CDN by creating instances in different regions.
try Amazon CloudFront. They say anything that serve with HTTP will work.
I'm running 3 Wowza boxes on AP region, and now running FMS + CLoudFront for my scalable livestreaming project.
Yet to test Wowza + CLoudFront
Edgecast is FMS based, not wowza.
ScaleEngine is a wowza CDN (disclaimer, I work for ScaleEngine) - all the origins, and all the edges, are wowza. They can set anything in Application.xml you need, or install your custom module, or run a mid-origin setup from your single wowza server.
Related
I have a client with a 1-2 thousand viewer audience, with everyday streams, same concurrent number of viewers.
Ive got a server set up for their website etc, but am in the process of figuring out the best way to stream with OBS onto that server, and than re-distribute that stream to clients (as an embed on the website).
Now from the calculations i did, running that kind of concurrent viewers is very problematic, as it forces you into a 10gbit link - which is very expensive, and i would ideally like to fit within 1-2gbps, if possible.
A friend of mine recommended to look into "Multicast" which supossedly uses MUCH less bandwith than regular live streaming options. Is multicast doable? Ive had a NGINX live stream set up on my server by a friend before, but never looked into the config and if multicast is supported within that. Are there any other options? What would you recommend?
Also, the service of that live stream isnt a high profit / organisation type of deal, so any pre-made services just dont make sense, as it would easily cost 40+ dollars per stream, which is just too much for my client.
Thank you for any help!
Tom
Rather than Multicast, P2P is more practical solution on Internet, to save money not bandwidth.
Especially for H5 browser, it's possible to use WebRTC DataChannel to transport P2P data.
But Multicast does not work on internet routers.
Multicast works by sending a single stream across the network to edge points where clients can 'join' the multicast to get an individual stream for them.
It requires that the network supports multicast protocols and the edges align with your users.
It is typically used when an operator has their own IP network for service like IPTV, rather than for services over the internet.
For your scenario, you would usually use an organ server and a CDN - this will usually reduce the load on your own server as the video will be cached on the network and multiple user can access the same 'chunks' of the video.
You can see and AWS example for on demand video here - other vendor and cloud providers have solutions too so this is just an example:
https://docs.aws.amazon.com/AmazonS3/latest/userguide/tutorial-s3-cloudfront-route53-video-streaming.html
You can find more complex On Demand and Live tutorial also but they are likley more that you need: https://aws.amazon.com/cloudfront/streaming/
Exploring P2P may be an option also as Winton suggests - some CDN may also leverage P2P technology internally.
I have a system that will generate image optimization and resizing for a client who has a news portal with lots of pageviews. We will provide only the images to this portal, but users are all on the same country as the our server. The question is, whats the best strategy thinking about cost-benefit:
Route all(most) image traffic via some paid CDN
Setup an internal image server using nginx and a loadbalancer
Monthly we estimate a bandwidth of 11TB, with millions of requests. (images only)
It is not a questions if it is possible or what is more cost efficient.
You need to calculate the costs based on many factors: Actual sizing of your servers. Amount of servers. Bandwith. Where are the servers located and much more.
It will be a lot of work to setup and maintain / monitor your own CDN probaly but sure you can do it.
I dont think that anybody can create this calculation for you. See the comment fro Rob. It is not realy a question for SO.
I'm having to learn Squid on the fly for a project and have tried to Google this, but not sure how to ask the question so Google can find it. I have a customer that wants to distribute larger amounts of video to multiple clients, but they have a slow WAN. The idea is to have Squid as a reverse proxy and cache to hold the videos on the local networks for faster transfer to the end users and to avoid saturating the WAN.
My question is this. If multiple users try to download a video that Squid has not yet cached, or fully cached, at the same time, what happens? In my particular scenario (insert unnecessary explanation here), this will happen frequently. Is a separate download of the video from the host opened for each request to Squid, or is Squid smart enough to manage it and only have one connection back to the host actually downloading the video?
This problem is called thundering herd or cache miss storm. It happens with new resources and also on higly used resources's cache invalidation.
For squid used as a reverse Proxy there are some options available to handle this, like the Collapsed Forwarding. The idea is to temporize parallel queries to the same resources, and use only one query fetching the resource on the backend.
Check the link for details, because depending on the squid version you may have to try different configuration tokens and values.
I'm making a network game (1v1) where in-game its p2p - no need for a game server.
However, for players to be able to "find each other", without the need to coordinate in another medium and enter IP addresses (similar to the modem days of network games), I need to have a coordination/matching server.
I can't use regular web hosting because:
The clients will communicate in UDP.
Therefore I'll need to do UDP Hole Punching to be able to go through the NAT
That would require the server to talk in UDP and know the client's IP and port
afaik with regular web hosting (php/etc) I can only get the client's IP address and can only communicate in TCP (HTTP).
Options I am currently considering:
Use a hosting solution where my program can accept UDP connection. (any recommendations?)
UDPonNAT seems to do this but uses GTalk and requires each client to have a GTalk account for this (which probably makes it an unsuitable solution)
Any ideas? Thanks :)
First, let me say that this is well out of my realm of expertise, but I found myself very interested, so I've been doing some searching and reading.
It seems that the most commonly prescribed solution for UDP NAT traversal is to use a STUN server. I did some quick searches to see if there are any companies that will just straight-up provide you with a STUN hosting solution, but if there even were any, they were buried in piles of ads for simple web hosting.
Fortunately, it seems there are several STUN servers that are already up and running and free for public use. There is a list of public STUN servers at voip-info.org.
In addition, there is plenty more information to be had if you explore SO questions tagged "nat".
I don't see any other choice than to have a dedicated server running your code. The other solutions you propose are, shall we say, less than optimal.
If you start small, virtual hosting will be fine. Costs are pretty minimal.
Rather than a full-blown dedicated server, you could just get a cheap shared hosting service and have the application interface with a PHP page, which in turn interfaces with a MySQL database backend.
For example, Lunarpages has a $3/month starter package that includes 5gb of space and 50gb of bandwidth. For something this simple, that's all you should need.
Then you just have your application poll the web page for the list of games, and submit a POST request in order to add their own game to the list.
Of course, this method requires learning PHP and MySQL if you don't already know them. And if you do it right, you can have the PHP page enter a sort of infinite loop to keep the connection open and just feed updates to the client, rather than polling the page every few seconds and wasting a lot of bandwidth. That's way outside the scope of this answer though.
Oh, and if you're looking for something absolutely free, search for a free PHP host. Those exist too! Even with an ad-supported host, your app could just grab the page and ignore the ads when you parse the list of games. I know that T35 used to be one of my favorites because their free plan doesn't track space or bandwidth (it limits the per-file size, to eliminate their service being used as a media share, but it shouldn't be a problem for PHP files). But of course, I think in the long run you'll be better off going with a paid host.
Edit: T35 also says "Free hosting allows 1 domain to be hosted, while paid offers unlimited domain hosting." So you can even just pay for a domain name and link it to them! I think in the short term, that's your best (cheapest) bet. Of course, this is all assuming you either know or are willing to learn PHP in order to make this happen. :)
There's nothing that every net connection will support. STUN is probably good, UPnP can work for this.
However, it's rumored that most firewalls can be enticed to pass almost anything through UDP port 53 (DNS). You might have to argue with the OS about your access to that port though.
Also, check out SIP, it's another protocol designed for this sort of thing. With the popularity of VOIP, there may be decent built-in support for this in more firewalls.
If you're really committed to UDP, you might also consider tunneling it over HTTP.
how about you break the problem into two parts - make a game matcher client (that is distinct from the game), which can communicate via http to your cheap/shared webhost. All gamers who wants to use the game matching function use this. THe game matcher client then launches the actual game with the correct parameters (IP, etc etc) after obtaining the info from your server.
The game will then use the standard way to UDP punch thru NAT, etc etc, as per your network code. The game dont actually need to know anything about the matcher client or matcher server - in the true sense of p2p (like torrents, once you can obtain your peer's IPs, you can even disconnect from the tracker).
That way, your problems become smaller.
An intermediate solution between hosting your own dedicated server and a strictly P2P networking environment is the gnutella model. In that model, there are superpeers that act like local servers, having known IP addresses and being connected to (and thus having knowledge of) more clients than a typical peer. This still requires you to run at least one superpeer yourself, but it gives you the option to let other people run their own superpeers.
I'm a Java coder and not very familiar with how networks work (other than basic UDP/TCP connections)
Say I have servers running on machines in the US, Asia, Latin America and Europe. When a user requests a service, I want their request to go to the server closest to them.
Is it possible for me to have one address: mycompany.com, and somehow get requests routed to the appropriate server? Apparently when someone goes to cnn.com, they receive the pictures, videos, etc. from a server close to them. Frankly, I don't see how that works.
By the way, my servers don't serve web pages, they serve other services such as stock market data....just in case that is relevant.
Since I'm a programmer, I'm interested to know how one would do it in software. Since this is little more than an idle curiosity, pointers to commercial products or services won't be very helpful in understanding this problem :)
One simple approach would be to look at the first byte (Class A) of the IP address coming into the UDP DNS request and then based off that you could deliver the right geo-located IP.
Another approach would be a little more complicated. Instead of using the server that is geographically closest to the user, you could use the server that has the lowest latency for that user.
The lower latency will provide faster transfer speeds while being easier to calculate than geographic location.
For a much more detailed look, check out this article on CDNs (pay attention to the Technology Section):
Content Delivery Network - Wikipedia
These are the kinds of networks that the large sites use to distribute their content over the net (Akamai is a popular example). As you can see, things can get pretty complicated pretty quickly with CDNs having their own proprietary protocols, etc...
Update: I didn't see the disclaimer about commercial solutions at the end of the original post. I'll leave this up for those who may find it of interest.
--
Take a look at http://ultradns.com/. A managed DNS service like that may be just what you need to accomplish what you are looking for.
Amazon.com, Forbes.com, Oracle, all use them...
Quote From http://ultradns.com/solutions/traffic.html:
UltraDNS Traffic Management solution provides a set of tools allowing IT administrators to define load balancing configurations for content servers residing in one or more geographic locations. The Traffic Management Solution manages traffic directed to the servers by dynamically changing the responses to DNS requests. Load balancing is performed based on dynamic metrics obtained from the host servers on a continual monitoring basis. The UltraDNS Traffic Management solution is not a single application, but combines the capabilities of several existing UltraDNS systems to control traffic, manage site failures, and optimize web content systems.
One approach is, as Jeff mentioned, using the IP address: http://en.wikipedia.org/wiki/Geolocation_software
In my experienced, this is precise to the nearest relatively large city (in the US at least). There are several open databases to aid in this (see the wiki link). Then you can generate image tags and download links and such based on this information.
As for locating the nearest server, I'm sure you can think of a few ways to do it. For instance, if the best return you can get is major city, you can lookup that city in a list of Latitude/Longitude and calculate the nearest server based on that.