In lua resty https://github.com/pintsized/lua-resty-http, I saw we can use request_pipeline for requests. I am wandering whether this will increase the performance. While after reading the source code, I found the request_pipeline method is also implemented with the regular send_request, and a loop is used to send each request one at a time.
Seems it cannot help to improve performance, if it is the case, why bothering to have this method?
Thanks
Someone answered on github https://github.com/pintsized/lua-resty-http/issues/130
https://en.wikipedia.org/wiki/HTTP_pipelining
Pipelining is intended to reduce latency when sending many requests.
It's not commonly used though as many intermediaries do not support it
well. It's part of the HTTP spec which is why its included.
Problem solved
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 1 year ago.
Improve this question
We are developing a browser extension which would send all the URLs visited by a logged in user to backend APIs to be persisted.
Now as number of requests send to backend API would be huge and hence we are confused between if we create a persistent connection via websocket OR do it via TCP connection i.e. using HTTP rest API calls.
The data post to backend API doesn't need to be real time as we anyway would be using that data in our models which doesn't demand them to be real time.
We are inclined towards HTTP rest API calls as due to below reasons
Easy to implement
Easy to scale(using auto-scaling techniques)
Everyone in the team is already comfortable with the rest APIs
But at the same time cons would be
On the scale where we would have a lot of post requests going to server not sure it would be optimised
Feels like websockets can give us an optimised infrastructure :(
I would love if I can hear from community if we can have any pitfalls going with rest API calls option.
So first of all TCP is the transport layer. It is not possible to use raw TCP, you have to create some protocol on top of it. You have to give meaning to the stream of data.
REST or HTTP or even WebSockets will never be as efficient as customly designed protocol on top of raw TCP (or even UDP). However the gain may not be as spectacular as one may think. I've actually done such transition once and we've experienced only few percent of performance gain. And it was neither easy to do correctly nor easy to maintain. Of course YMMV.
Why is that? Well, the reason is that HTTP is already quite highly optimized. First of all you have "keep alive" header that keeps the connection open if it is used. And so the default HTTP mechanisms already persists connection when used. Secondly HTTP handles body compression by default, and with HTTP/2 it also handles headers compression. With HTTP/3 you even have more efficient TLS usage and better support in case of unstable network (e.g. mobile).
Another thing is that since you do not require real time data then you can do buffering. So you don't send data each time it is available, but you gather it for say few seconds, or minutes or maybe even hours, and send it all in one go. With such approach the difference between HTTP and custom protocol will be even less noticable.
All in all: I advice you start with the simplest solution there is, in your case it seems to be REST. Design your code so that transition to other protocol is as simple as possible. Optimize later if needed. Always measure.
Btw, there are lots of valid privacy and security concerns around your extension. For example I'm quite surprised that you didn't mention TLS at all. Which matters, not only because of security, but also because of performance: establishing TLS connections is not free (although once established, encryption does not affect performance much).
Putting my discomfort aside (privacy, anyone?)...
Assuming your extension collates the Information, you might consider "pushing" to the server every time the browser starts / quits and then once again every hour or so (users hardly ever quite their browsers these days)... this would make REST much more logical.
If you aren't collating the information on the client side, you might prefer a WebSocket implementation that pushes data in real time.
However, whatever you decide, you would also want to decouple the API from the transmission layer.
This means that (ignoring authentication paradigms) the WebSockets and REST implementations would look largely the same and be routed to the same function that contains the actual business logic... a function you could also call from a script or from the terminal. The network layer details should be irrelevant as far as the API implementation is concerned.
As a last note: I would never knowingly install an extension that collects so much data on me. Especially since URLs often contain private information (used for REST API routing). Please reconsider if you want to take part in creating such a product... they cannot violate our privacy if we don't build the tools that make it possible.
I want to send a newsletter to our site's registered users - more than 100000
Today, I send the mails in a simple for loop, which is very heavy and slow.
I want it to finish the process as fast as possible (even if the process would run in the background and didn't bother the users' UI)
I tried TPL, Async, smtpClient.SendAsyncMail() and more, but never could actually create a working example that proves that these systems work faster (or work at all).
I've read many posts and explanations using thread, but they seem to be old, deprecated or irrelevant, since there are new technologies I surely don't know of.
Could you please tell me what is the most efficient and fastest way to send multiple emails in parallel? Could you show me an example that works?
UPDATE 20/06/2018
Each email is different for each user. Therefore a BCC solution is irrelevant for me.
I will specify my question a little more:
I only focus on the high-level where I use a sending function, such as ASP.NET smtpClient.Send() to send the mails to the SMTP.
I ignore for now the part of the SMTP itself. I have a great SMTP server that knows how to handle the queuing.
The main pitfall right now is the Send function that uploads the bytes to the SMTP server in ASP.NET.
That's where I couldn't find a way to send faster. (It is needles to say that if I buy a stronger server, even an iterative method will work faster)
But I want a scalable solution - a smarter one, where I can send in parallel huge amounts of mails.
UPDATE 28/06/2018
Apparently a parallel solution will not solve my problem. As explained by #Terry Carmen and others, I may have to look at other bottle necks in the process. Thank you all for helping!
What a great community!!!
Add the users as BCC recipients to the message and let the mail server do the work.
The mail server probably has a BCC limit, so you may have to see what the server's limit is and batch the requests.
edit
Regarding "sending in parallel"
You're looking for the wrong answer. If your machine takes X seconds to produce Y emails, running them "in parallel" won't fix anything . The most you would do is split the generation between several cores, which might improve performance a little, but is almost certainly not the problem.
You need to see "what the actual slow part is". It might be your disk or CPU or network connection, or database server or possibly a rate limit setting on the mail server. The actual SMTPClient object is almost certainly not the issue.
another edit
You can try sending in parallel by running separate instances of SMTPClient in separate threads; However this is unlikely to actually help you.
As mentioned before by myself and others, you are assuming that parallel send is some sort of magic bullet. It's not. There is something that is rate-limiting you and it's almost certainly not that you're single threaded.
Just wondering if anyone can help me understand the advantages of talking to Arango via HTTP or the driver? In my case the GoLang driver.
Aside from the few milliseconds overhead per request,would I experience a big difference in performance?
I like the concurrency attributes of using HTTP Requests, since my main tools all make HTTP communication very easy.
Thanks for any insights on this subject.
Quick scanning through the 3 recommended drivers shows they use HTTP under the hood.
So using them will not give you big difference in DB interaction, but may be more comfortable for development.
What is the correct way (or best) way to implement Comet, HTTP Push, or Reverse AJAX?
What .NET implementations would you recommend?
I have hear about, WebSync and PokeIn, both are paid implementations, I have used PokeIn and its pretty straight forward. If you are looking forward to code your own COMET implementation, I just can say that its a complex task, because you need to modify the natural behaviour if IIS. Its a hacky way to get around the limitations of the HTTP protocol and you need to know really well what you doing so don't end up breaking things around =).
It's also known as long-lived
requests. This is also by far the most
complex method to implement.
Basically, a request is made by the
client, and the server very slowly
responds, which causes the connection
to be maintained. Periodically, when
the server has something to push,
it'll "burst" send the information, so
to speak. This approach gives you
real-time push, which is great. But,
it has a serious down-side: holding
connections open like that isn't how
the underlying protocols are meant to
work, and most servers aren't terribly
happy about it. If your traffic gets
too great, you'll chew up threads on
the server and wind up bringing your
site down.
ref: http://www.coderanch.com/t/121668/HTML-JavaScript/does-Reverse-Ajax-Works
JOBG is correct re: the complexities; it's probably not a task you want to undertake lightly. I'm one of the authors of WebSync, and I can attest that it's a difficult task.
There are a ton of examples in the download, and the community edition is free.
Microsoft is developing HTTP push in SignalR
https://github.com/SignalR/SignalR
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
I have an existing standalone application which is going to be extended by a 3rd-party, using a network protocol. The capabilities are already implemented, all I need is to expose them to the outside.
Assuming the transport protocol is already chosen (UDP), are there any resources that will help me to design my application protocol?
There seems to be a lot of information about software design, but not on protocol design.
I've already looked at Application Protocol Design.
See Jabber protocols design guidelines and RFC 4101. Although it is aimed at making RFCs more easy to understand to reviewers, this RFC provides some interesting advices.
Have you looked at Google Protocol Buffer? It seems like a good way to resolve this issue.
You can create an endpoint that communicates with your existing app and then responds from 'outside' using the protobuffer protocol. It's binary, so it's tiny and fast and you don't have to write your own protocol manager, 'cause you can use the Google ones. The downside is that it has to be implemented on both sides of the system (on your 'server' side and on the consumer/client side).
Another recommendation for protocol buffers - nice tight binary with little effort. Note, however, that while the binary protocol is well defined, there isn't yet an agreed RPC standard (several are in progress, tending to lean towards TCP or HTTP).
The spec makes it very easy to have the client and server in different architectures, which is good - plus it is extensible.
Caveat: I'm the author of one of the .NET versions, so I may well be biased ;-p
First off, UDP is primarily a one-way broadcast transport method. Also, it is potentially lossy, so you need to be able handle missing packets and out-of-order packets. If you need any level of reliability from UDP, or require two-way connections, you will end up needing just about everything from TCP, so you might as well go with that to start with and let the network stack take care of it.
Next up, if your data is potentially larger than a single IP packet then you will need some way of identifying the start and end of each packet, and a means of handling illegal or corrupt packets. I would recommend some kind of header with packet length, some kind of footer, and maybe a checksum.
Then you need some way of encoding the messages and responses. There are many RPC protocols around. You could look at SOAP, or design a custom XML-based protocol, or a binary one.
You should really think hard about whether you really want to design, document and maintain your own protocol or use something that is already existing. It is probable there is already a documented protocol that matches your needs. Depending on what you are doing it will probably look overkill at first and implementing all the spec will look tedious and a lot less fun than writing your own but if you intend for your application to still be actively developed in a few years it should save you a lot of time and money to use something that already exist and is known by third parties. Besides, if you can use an existing library for that protocol, the implementation part should be a lot faster.
Designing new protocol is more fun than implementing one but less than maintaining one as you have to live with all the defects. No protocol is perfect but if you have never designed one you can be assured you will make more mistake designing it than the people who designed the existing well known protocol you could use instead.
In short, leverage what already exists whenever possible.
If you're choosing XML keep in mind that you will have a giant overhead of markup.
A simple binary protocol will also be need not so much ressources to parse compared to xml.
If you do not want to build your protocol from ground up, you should take a look at SOAP. Support varies for different programming languages, but cross language communication is explicitly encouraged.
Unfortunately UDP and SOAP seem to have stuck in its infancy, HTTP is most commonly used.
I have an existing standalone application which is going to be extended by a 3rd-party, using a network protocol.
It would help to know a little more about what your program does and what the nature of these 3rd party extensions are. Maybe some rationale for using UDP?