I am interested in writing my own internet ad server.
I want to serve billions of impressions with as little hardware possible.
Which server-side technologies are best suited for this task? I am asking about the relative overhead of serving my ad pages as either pages rendered by PHP, or Java, or .net, or coding Http responses directly in C and writing some multi-socket IO monster to serve requests (I assume this one wins, but if my assumption is wrong, that would actually be most interesting).
Obviously all the most efficient optimizations are done at the algorithm level, but I figure there has got to be some speed differences at the end of the day that makes one method of serving ads better than another. How much overhead does something like apache or IIS introduce? There's got to be a ton of extra junk in there I don't need.
At some point I guess this is more a question of which platform/language combo is best suited - please excuse the in-adroitly posed question, hopefully you understand what I am trying to get at.
You're going to have a very difficult time finding an objective answer to a question like this. There are simply too many variables:
Does your app talk to a database? If so, which one? How is the data modeled? Which strategy is used to fetch the data?
Does your app talk across a network to serve a request (web service, caching server, etc)? If so, what does that machine look like? What does the network look like?
Are any of your machines load balanced? If so, how?
Is there caching? What kind? Where does it live? How is cached data persisted?
How is your app designed? Are you sure it's performance-optimal? If so, how are you sure?
When does the cost of development outweigh the cost of adding a new server? Programmers are expensive. If reduced cost is your goal with reducing hardware, you'll likely save more money by using a language in which your programmers feel productive.
Are you using 3rd party tools? Should you be? Are they fast? Won't some 3rd party tools reduce your cost?
If you want some kind of benchmark, Trustleap publishes challenge results between their G-Wan server using ANSI C scripts, IIS using C#, Apache with PHP, and Glassfish with Java. I include it only because it attempts to measure the exact technologies you mention. I would never settle on a technology without considering the variables above and more.
Errata:
G-Wan uses ANSI C scripts (rather than "compiled ANSI C" as explained above)
And it transparently turns synchronous (connect/recv/send/close) system calls into asynchronous calls (this is working even with shared libraries).
This can help a great deal to scale with database server requests, posts, etc.
Related
I would like to bring your attention to something that I re-think for days. The new features and impact of HTTP/2 protocol for web development. I would also like to ask some related questions, because my annual planning is getting less accurate because of HTTP/2.
Since HTTP/2 uses a single, multiplexed connection, instead of multiple connections in HTTP 1.x domain sharding techniques will not be needed any more.
With HTTP/1.x you may have already put files in different domains to increase parallelism in file transfer to the web browser; content domain networks (CDNs) do this automatically. But it doesn't help – and can hurt – performance under HTTP/2.
Q1: Will HTTP/2 minimize the need for CDNs?
Code files concatenating. Code chunks that would normally be maintained and transferred as separate files are combined into one. The browser then finds and runs the needed code within the concatenated file as needed.
Q2. Will HTTP/2 eliminate the need to concatenate files with similar extensions (css, javascript) and the usage of great Grunt and Gulp tools to do so?
Q. Also, in order to simplify and keep the question more compact, I would ask quite generally what may be other impacts of HTTP/2 on web development as you can foresee?
Q1: Will HTTP/2 minimize to need for CDNs?
It will certainly shift the balance a bit, provided that you use the right software. I talk about balance because CDNs cost money and management time.
If you are using CDNs to offload traffic you still will need them to offload traffic.
If you are a smallish website (and most websites are, in numerical terms), you will have less of a reason to use a CDN, as latency can be hidden quite effectively with HTTP/2 (provided that you deploy it correctly). HTTP/2 is even better than SPDY, and check this article for a use case regarding SPDY.
Also, most of the third-party content that we incorporate into our sites already uses CDNs.
Q2. Will HTTP/2 eliminate the need to concatenate files with similar extensions (css, javascript) and the usage of great Grunt and Gulp tools to do so?
Unfortunately not. Concatenating things won't be needed, unless the files you are delivering are extremely small, say a few hundred bytes. Everything else is still relevant, including minification and adding those ugly query strings for cache busting.
Q3 . Also, in order to simplify and keep the question more compact, I would ask quite general what may be other impacts of HTTP/2 on web development as you can foresee?
This is a tricky question. In one hand HTTP/2 arrives at a moment when the web is mature, and developers have whole stacks of things to take care of. HTTP/2 can be seen as a tiny piece to change in such a way that the entire stack doesn't crumble. Indeed, I can imagine many teams selling HTTP/2 to management this way ("It won't be a problem, we promise!").
But from a technical standpoint, HTTP/2 allows for better development workflows. For example, the multiplexing nature of HTTP/2 means that most of the contents of a site can be served over a single connection, allowing some servers to learn about interactions between assets by just observing browser behaviors. The information can be used together with other features of HTTP/2 and the modern web (specifically, HTTP/2 PUSH and the pre-open headers) to hide a lot of latency. Think about how much work that can save developers interested in performance.
Q1: Will HTTP/2 minimize to need for CDNs?
No. CDN's are primarily to co-locate content close to the user based on geographic location. Closer your are to the server, faster you will get the contet.
Q2. Will HTTP/2 eliminate the need to concatenate files with similar extensions (css, javascript) and the usage of great Grunt and Gulp tools to do so?
Concatenation is only a part of things a tool like is Grunt/Gulp does. Linting, conversions, runnings tests are other things you would still need a tool for. So they will stay. In terms of concat, you would ideally move away from creating a single large concat file per type and move to creating smaller concatenated files per module.
Q3. Also, in order to simplify and keep the question more compact, I would ask quite general what may be other impacts of HTTP/2 on web development as you can foresee?
General idea is HTTP/2 will not make a huge change to the way we develop things as its a protocol level change. Developers would ideally remove optimizations (like compacting, sharding) which are not optimization techniques with http/2
Can someone please explain this excerpt from golang's documentation on CGI:
"
Note that using CGI means starting a new process to handle each request, which is typically less efficient than using a long-running server. This package is intended primarily for compatibility with existing systems.
"
I use CGI to make database puts and gets.
Is this inefficient? Should I be using a 'long-running server'?
If so what does that mean, and how do I implement it?
... http://golang.org/pkg/net/http/cgi/
Yes, it is inefficient. The cost of starting a whole new process is generally much more than just connecting through to an already-existing process, or doing something on a thread within the current process.
In terms of whether it's necessary, that depends. If you're creating a search engine to rival Google, I would suggest CGI is not the way to go.
If it's a personal website accessed once an hour, I think you can probably get away with it.
In terms of a long running server, you can generally write something like a plug-in for a web server which is running all the time and the web server just passes off requests to it when needed (and possibly multiple threads of "it").
That way, it's ready all the time, you don't have to wait while the web server starts another process to handle the request.
In fact, Apache itself does CGI via a module (like a plug-in) which integrates itself into Apache at runtime - the actual calling of external processes is handled from that module. The source code, if you think it will help, can be found in mod_cgi.c if you do a web search for it.
Another example is mod_perl which is a Perl interpreter module, available at this link.
One option to look into is fastcgi which is a long running server program that doesn't continually restart each request. It used to be that fast cgi had its disadvantages due to memory leaks over time in languages like C, C++, FPC, etc. since they are not garbage collected. A small memory leak in one fastcgi program after millions of hits to the website could bring the server down, whereas regular old CGI was a garbage collector itself: the program restarted and therefore cleaned up each time someone requested the page and the cgi exited. In the case of Go lang memory leaks are not a concern, however fast cgi could have some hidden gotchyas such as: if golang has any memory leaks in its garbage collector itself... (unlikely, but gotchyas like this might pop up - also heap fragmentation .... over time..)
Generally fastcgi and "long running" is premature optimization. I've seen people with 5 visitors to their personal home page website a day yelling "hey maybe I should use fastcgi" when in fact they would need 5 million visitors a day - but they like to be hip and cool so they start thinking about fast cgi before their site is even known by 3 people.
You need to ask yourself: does the server you are using have a lot of traffic, and by a lot of traffic I don't mean 100 visitors a day... even 1000 unique visitors a day is not a lot.
It is unclear whether you are wanting to write Go lang cgi programs for apache server, or python ones for a go server, or whether you are writing a go server that has cgi capability for python and perl. Clarify what you are actually doing.
As for rivaling Google as a search engine which someone posted about in another answer: if you look at the history of Google they actually coded their programs in C++/C via some cgi system ... rather than using PHP, perl, or other hip and cool stuff that the kids use. Look up backrub project and its template system eons ago. It was called Ctemplate (C compiled programs called upon html templates.....)
https://www.google.com/search?safe=off&q=google+backrub+template+ctemplate
Fastcgi was maybe something that google figured out before there was a fastcgi, or they had their own proprietary solution similar to fastcgi, I don't know since I didn't work at google - but since they used C++/C programs to power google in the old days (and probably still today for some stuff) they must have been using some cgi technology, even if it was modified cgi technology for speed.
I'm profiling a asp(classic) web service. The web service makes database calls, reads/writes to files, and processes xml. On a windows server 2003 box(2.7ghz, 4 core, 4gb ram) how many requests per second should I be able to handle before things start to fail.
I'm building a tool to test this, but I'm looking for a number of requests per second to shoot for.
I know this is fairly vague, but please give the best estimate you can. If you need more information, please ask.
95% of the performance of any data-driven app is dependent on the database: 1) the way you do your calls, 2) the indexes, 3) the hardware under the database (disk subsystem in particular).
I have seen a machine, like you are describing, handle 40 requests per second (2500/minute), but numbers like 10 per second (600/minute) are more common. I would expect even lower if you are running your DB on the same machine, and even lower still if that DB is SQLExpress or MSAccess.
Also, at capacity, your app will probably not fail, but IIS will Queue requests, once it is saturated, and may timeout some of those requests if it can't service them before the timeout expires.
Btw, instead of building a tool to test your app, you may want to look into using a test tool such as Microsoft WCAT. It is pretty smooth and easy to use.
How fast should it be? Fast enough.
How fast is fast enough? That's a question that only you and your users can answer. If your service is horrifically inefficient and keeps up with demand, it's fast enough. If your service is assembly-optimized, lightning-fast, and overwhelmed with requests, it's not fast enough.
If the server is handling its actual workload, then don't worry about how fast it "should" be. When the server is having trouble, or when you anticipate that it soon will, then you should look at improving the code or upgrading the hardware. Remember Knuth's Law – premature optimization is the root of all evil. Any work you do now to make it faster may never pay off, and you may be forced to make compromises with flexivility or maintainability. Remember, too, an older adage – if it ain't broke, don't fix it.
Yes I would also say 10 per second is a good benchmark. For a high performance app you would want to get more than this, but if you have no specific goal you should generally be able to get at least 10 requests per sec for a general web page with a bunch of database queries.
From all the articles I've read so far about Mochiweb, I've heard this over and over again that Mochiweb provides very good scalability. My question is, how exactly does Mochiweb get its scalability property? Is it from Erlang's inherent scalability properties or does Mochiweb have any additional code that explicitly enables it to scale well? Put another way, if I were to write a simple HTTP server in Erlang myself, with a simple 'loop' (recursive function) to handle requests, would it have the same level of scalability as a simple web server built using the Mochiweb framework?
UPDATE: I'm not planning to implement a full blown web-server supporting every feature possible. My requirements are very specific - to handle POST data from a HTML form with fixed controls.
Probably. :-)
If you were to write a web server that handles each request in a separate process (light weight thread in Erlang) you could reach the same kind of "scalability" easily. Of course the feature set would be different, unless you implement everything Mochiweb has.
Erlang also has great built in support for distribution among many machines, this might be possible to use to gain even more scalability.
MochiWeb isn't scalable itself, as far as I understand it. It's a fast, tiny server library that can handle thousands of requests per second. The way in which it does that has nothing to do with "scalability" (aside from adjusting the number of mochiweb_acceptors that are listening at any given time).
What you get with MochiWeb is a solid web server library, and Erlang's scalability features. If you want to run a single MochiWeb server, when a request comes in, you can still offload the work of processing that request to any machine you want, thanks to Erlang's distributed node infrastructure and cheap message passing. If you want to run multiple MochiWeb servers, you can put them behind a load balancer and use mnesia's distributed features to sync session data between machines.
The point is, MochiWeb is small and fast (enough). Erlang is the scalability power tool.
If you roll your own server solution, you could probably meet or beat MochiWeb's efficiency and "scalability" out of the box. But then you'd have to rethink everything they've already thought of, and you'd have to battle test it yourself.
Our app need instant notification, so obvious I should use some some WCF duplex, or socket communication. Problem is the the app is partial trust XBAP, and thus I'm not allowd to use anything but BasicHttpBinding. Therefore I need to poll for changes.
No comes the question: My PM says the update interval should be araound 2 sec, and run on a intranet with 500 users on a single web server.
Does any of you have experience how polling woould influence the web server.
The service is farly simple, it takes a guid as an arg, and returns a list of guids. All data access are cached, so I guess the load on the server is minimal for one single call, but for 500...
Except from the polling, the webserver has little work.
So, based on this little info (assume a standard server HW, whatever that is), is it possible to make a qualified guess?
Is it possible or not to implement this, will it work?
Yes, I know estimating this is difficult, but I'd be really glad if some of you could share some thoughts on this
Regards
Larsi
Don't estimate, benchmark.
Polling.. bad, but if you have no other choice, then it's ideal :)
Bear in mind the fact that you will no doubt have keep-alives on, so you will have 500 permanently-connected users. The memory usage of that will probably be more significant than the processor usage. I can't imagine network access (even in a relatively bloaty web service) would use much network capacity, but your network latency might become an issue - especially as we've all seen web applications 'pause' for a little while.
In the end though, you'll probably be ok, but you'll have to check it yourself. There are plenty of web service stress testers, you can use Microsoft's WAS tool for one, here's a few links to others.
Try using soapui, a web service testing tool, to check the performance of your web service. There is a paid version and an open source version that is free.
cheers
I don't think it will be a particular problem. I'd imagine the response time for each request would be pretty low, unless you're pulling back a hell of a lot of data, so 500 connections spread over 2 seconds shouldn't hit it too hard.
You can use a stress testing tool to verify your webserver can handle the load though, before you commit to this design.
250 qps probably is doable with quite modest hardware and network bandwidth provided you do minimize the data sent back & forth. I assume you're caching these GUID lists on the client so you can just send a small "no updates" response in the normal case.
Should be pretty easy to measure with a simple prototype though to be more confident.