First I'll describe my case.
I have to do HTTPS requests to several APIs from my application and they should be ran concurrently.
I want to know if I should use a separate HTTP client per goroutine or I can share one client across all goroutines. Of course I'd like to enjoy connection reusing/pooling offered by the HTTP client, but I am concerned about it being thread(aka goroutine)-safe and if the client will run requests concurrently or they'll in fact be sequenced?
Http clients are thread safe according to the docs (https://golang.org/src/net/http/client.go):
Clients are safe for concurrent use by multiple goroutines.
The other question was should you use one client or one per request. You should use one client as per
https://pkg.go.dev/net/http#pkg-overview
"Clients and Transports are safe for concurrent use by multiple goroutines and for efficiency should only be created once and re-used"
Related
While looking for zookeeper, the accepted answer says that concurrent writes are not allowed.
Explaining Apache ZooKeeper
Now my question is as Zookeeper has linear writes, that does not stop me to use Asynchronous APIs to create nodes and take the response in a callback ? Though internally it may not allow concurrent writes , or am I missing something ?
Even though zookeeper operates in an ensemble, writes are always served through the leader. Therefore, leader is capable of queuing write requests and completing them sequentially.
Using the asynchronous API will not do any harm to the above mentioned approach. Even though the write requests are asynchronous (from the client side), leader will always make sure that they are served sequentially. Once a asynchronous write request is served, client will be notified through the callback. It is simple as that. Remember, the requests are asynchronous as viewed by the client. But from the leader's point of view, they are served sequentially.
Consider we have an HTTP server and there are many concurrent clients making HTTP requests to this server.
The responses are adequately big to be considered.
User may send requests on the trot, with different request bodies. Once a new request has been made, former requests made by same user become obsolete.
In this type of situation, looking from client aspect, that may not be an issue. However, in server-side, it will make considerable amount of CPU time steal. If there is no way to do further processing relied upon former requests that came from same users, what to do? Here are some workarounds I thought:
to make a new HTTP request from client in order to cancel the process in server-side (via different port)
checking client if incoming request is a further request (over IPv4 or MAC address, is there a popular way to conduct this?)
That being said, please let the task being done by server upon HTTP requests is nonatomic (otherwise I think it would be a maze, because as I know a block-closure passed to some dispatch authority will be executed, ultimately).
This was actually a request-response pattern question because it is not limited to use HTTP. Sadly, there was no tag named request-response.
Thank you.
We have a requirement to to support 10k+ users, where every user initiate a request and waits for a response from the server (the response can take as long as 20-30 seconds to arrive). it is only one request from the client, and after a long processing by the server, a response will be transmitted and then the connection will disconnect.
in the background, the server will do some DB search and wait for other background processes to notify on completion before responding to the client.
after doing some research i figured out we will need to use something like the atmosphere framework to support websockets/sse event/long polling along with an asynchronous server like netty (=> nettosphere) or jetty.
As for my experience - mostly Java EE world and Tomcat server.
my questions are:
what will be easier to implement in regard to my experience and our requirement: atmosphere + netty or atmoshphere+jetty? which one can scale better, has an easier learning curve and easier to implement other java technologies?
how do u implement in atmosphere a response that is sent only to the originating client and not broadcast to the rest of the clients? (all the examples i found are broadcast).
how can i implement in netty (or jetty) when using the atmosphere framework our response? i.e., the client send a request, after it is received in the server some background processes are run, and when they finish i need to locate the connection and transmit the response. is that achievable?
Some thoughts:
At 10k+ users, with 20-30 second response latency, you likely hit file descriptor limits if using just 1 network interface. Consider a solution that uses multiple network interfaces.
Your description of your request/response can be handled entirely with standard Servlet 3.0, standard HTTP/1.1, Async request handling, and large timeouts.
If your clients are web browsers, and you don't start sending a response from the server until the 20-30 second window, you might hit browser idle timeouts.
Atmosphere and Cometd do the same things, supporting long duration connections, with connection technique fallbacks, and with logical channel APIs.
I believe the AKKA framework will handle this sort of need. I am looking at using it to handle scaling issues possibly with a RabbitMQ to help off load work to potentially other servers that may be added later to scale as needed.
What are the different approaches in developing a web-server?
So I guess there are (1) multi-thread (2) event-loop, is there anything else? What would be the pros/cons of each approach? when would you use each? can you list specific impl' for each approache
Different approach can be:
Single threaded: All connections are handled by a single thread that
"listens" for and awaits for connections and processes requests.It
is simple to implement but it is the most useless server as it can
only serve request at a time
Multithreaded:The server listens for requests and each incoming
request is allocated to a new thread to handle it.So each client
connection is handled by its dedicated thread. This approach(unlike
1) supports concurrent processing of client requests but does not
scale well since each new request creates a new thread at the server
and this takes a lot of resources.Eventually the server will hit a
limit
Multithreaded-Pools:Same idea as (2) but instead of creating a new
thread to handle each incoming request a thread from a thread-pool
is used.I.e. threads are created and placed on a pool for later
reuse.This scales very well supporting multiple client requests and
it is the standard approach.E.g. Tomcat works like this.
Event-Queue:Each incoming request is placed into a queue and is
processed by a background thread taking requests of the queue. It is
non-blocking and this type of asynchronous processing also scales
well.To be honest I am not sure if it is better than (3) in
performance.I think that tomcat can be configured for this using the
NIO architecture
You should add non-blocking I/O. Have a look at Netty.
Some servers like G-WAN mix Multithreaded-Pools and Event-Queues, letting the server saturate CPU Cores with each thread processing many connections.
Disclamer: I am involved in the development of this project.
I want to use a client-server protocol to push data to clients which will always remain connected, 24/7.
HTTP is a good general-purpose client-server protocol. I don't think the semantics possibly could be very different for any other protocol, and many good HTTP servers exist.
The critical factor is the number of connections: the application will gradually scale up to a very large number of clients, say 100,000. They cannot be servers because they have dynamic IP addresses and may be behind firewalls. So, a socket link must be established and preserved, which leads us to HTTP push. Only rarely will data actually be pushed to a given client, so we want to minimize the connection overhead too.
The server should handle this by accepting the connection, inserting the remote IP and port into a table, and leaving it idle. We don't want 100,000 threads running, just so many table entries and file descriptors.
Is there any way to achieve this using an off-the-shelf HTTP server, without writing at the socket layer?
Use Push Framework : http://www.pushframework.com.
It was designed for that goal of managing a large number of long-lived asynchronous full-duplex connections.
LightStreamer (http://www.lightstreamer.com/) is the tool that is made specifically for PUSH operations of HTTP.
It should solve this problem.
You could also look at Jetty + Continuations.