While reading through the Java documentation on the grpc website, I found the newBlockingStub and newStub generated methods being referenced.
My main interest in grpc is to use a single connection to handle multiple rpcs in parallel.
By making multiple calls to newBlockingStub/newStub, do I get all these stubs to use the same underlying connection, and thus still have my rpcs working in parallel?
The Channel manages sharing a single connection with multiple RPCs. As long as stubs use the same underlying ManagedChannel, they may share a single connection.
I say may share a connection, since the ManagedChannel can be configured with different policies, for things like load balancing. The point is that ManagedChannel handles those decisions and the stub isn't impacted.
Related
When we create a managedChannelBuilder and use this to call a grpc-java service call, how many clients can we serve with this? Doesn't this channel be shutdown after individual service call?
Say I have a REST interface which accepts REST calls from a browser
and from within these REST Service methods, I am making grpc client calls to an independent grpc server. Also I can expect client connections in the range of [4000-5000] concurrently.
How well can I make use of this managedChannelBuilder. Do I need just one? Or do I need to pool multiple channelbuilders?
Generally, I'd suggest using a single ManagedChannel per endpoint when your code can be easily structured to share it. ManagedChannel multiplexes RPCs and is thread-safe, so it can handle multiple RPCs concurrently.
In rarer cases of high very high throughput, it may make sense to use more than one ManagedChannel. Eventually ManagedChannel (or, maybe Channel) should have support for doing this natively.
I'd like to know if there's a way to communicate directly between two (or more) flask-socketio servers. I want to pass information between servers, and have clients connect a single web socket server, which would have all the combined logic and data from the other servers.
I found this example in JS Socket IO Server to Server where the solution was to use a socket.io-client to connect to another server.
I've looked through the Flask-SocketIO documentation, as well as other resources, however it doesn't appear that Flask-SocketIO has a client component to it.
Any suggestions or ideas?
Flask-SocketIO 2.0 can (maybe) do what you want. This is explained in the Using Multiple Workers section of the documentation.
Basically, the servers are configured to connect to a shared message queue service (redis, for example), and then a load balancer in front of them assigns clients to any of the servers in the pool using sticky sessions. Broadcasting operations are coordinated automatically among the servers by passing messages on the queue.
As an additional feature, if you use this set up, you can have any process connect to the message queue to post messages for clients, so for example, you can emit events to clients from a worker or other auxiliary process that is not a SocketIO server.
From your question it is unclear if you were looking to implement something like this, or if you wanted to have the servers communicate for a different reason. Sending of custom messages on the queue is currently not supported, but your question gave me the idea, this might be useful for some scenarios.
As far as using a SocketIO client as in the question you referenced, that shouud also work. You can use this Python package: https://pypi.python.org/pypi/socketIO-client. If you go this route, you can have a server be a client and receive events or join rooms.
SignalR documentation says that scaleout/backplane works well in case of server broadcast type of load/implementation. However I doubt that in case of pure server broadcast it will cause duplicate messages to be sent to the clients. Consider the following scenario:
I have two instances of my hub sitting on two web servers behind a load balancer on my web farm.
The hub on each server implements a timer for database polling to fetch some updates and broadcast to clients in groups, grouped on a topic id.
The clients for a group/topic might be divided between the two servers.
Both the hub instances will fetch the same or overlapping updates from the database.
Now as each hub sends the updates to clients via the backplane, will it not result in duplicate updates sent to the clients?
Please suggest.
The problem is not with SignalR, but with your database polling living inside your hubs. A backplane deals correctly with broadcast replication, but if you add another responsibility to your hubs then it's a different story. That's the part that is duplicating messages, not SignalR, because now you have N pollers doing broadcast across all server instances.
You could, for example, remove that logic from hubs into something else, and letting just one single instance of your server applications use this new piece in order to do the generation of messages by polling, using maybe a piece of configuration to decide which one. This way you would send messages only from there, and SignalR's backplane would take care of the replication. It's just a very basic suggestion and it could be done differently, but the key point is that your poller should not be replicated, and that's not directly related to SignalR.
It's also true that polling might not be the best way to deal with your scenario, but IMO that would be answering a different question.
I am new to SignalR and I try to understand the difference between consistent connection and long polling,Is there different use in methods ? is one better than the other? are there any diffrenet functions need to polling and other functions to use consistent connection?, I googled but didn't find a simple answer to this question, can someone help? need an explanation.
SignalR is a framework that allows us to build real-time web applications. Ideally, we would use web sockets for this. However, web sockets is a new protocol and requires support from both the browser and the server. Thus, web sockets are not generally available and SignalR tries to provide an abstract connection similar to web sockets but built upon existing technologies and techniques. This abstraction is called a Persistent Connection.
Persistent connection is the term used to describe SignalR's abstract connections.
Long polling is one of several techniques used to implement SignalR's persistent connections (the others are Forever Frame, Server-Sent Events and Web Sockets).
Suppose we were to implement a network application, such as a chat with a central server and several clients: we assume that all communication must go through the central server, then it should pick up messages from some clients and forward them to target clients, and so on.
Regardless of the technology used (sockets, web services, etc..), it is possible to think that there are some producer threads (that generate messages) and some consumer threads (that read messages).
For example, you could use a single queue for incoming and outgoing messages, but using a single queue, you couldn't receive and send messages simultaneously, because only one thread at a time can access the queue.
Perhaps it would be more appropriate to use two queues: for example, this article explains a way in which you can manage a double queue so that producers and consumers can work almost simultaneously. This scenario may be fine if there are only a producer and a consumer, but if there are many clients:
How to make so that the central server can receive data simultaneously from multiple input streams?
How to make so that the central server can send data simultaneously to multiple output streams?
To resolve this problem, my idea is to use a double queue for each client: on the central server, each client connection may be associated with two queues, one for incoming messages from that client and one for outgoing messages addressed to that client. In this way the central server may send and receive data simultaneously on almost all the connections with the clients...
There are probably other ways to manage the queues ... What are the parameters to determine how many queues are needed and how to organize them? There are cases that do not need any queue?
To me, this idea of using a queue per client or multiple queues per client seems to miss the point. First of all, it is absolutely possible to build a queue which can be accessed simultaneously by 2 threads (one can be enqueueing an item while a different one is dequeueing another item). If you want to know how, post a specific question about that.
Second, even if we assume that only 1 thread at a time can access a single queue, and even if we assume that the server will be receiving or sending data to/from all the clients simultaneously, it still doesn't follow that you need a different queue for each client. To avoid limiting system performance, you just need to allow enough concurrency to utilize all the server's CPUs. Even with a single, system-wide queue, if dequeueing/enqueueing messages is fast enough compared to the other work the server is doing, it might not be a bottleneck. (And with an efficient implementation, simply inserting an item or removing an item from a queue should be very fast. It's a very simple operation.) For that message queue to become the bottleneck limiting performance, either you would need a LOT of CPUs, or everything else the server was doing would have to be very fast. In that case, you could work out some scheme with 2 or 4 system-wide queues, to allow 2x or 4x more concurrency.
The whole idea of using work queues in a multi-threaded system is that they 1) allow multiple consumers to all grab work from a single location, so producers can "dump" whatever work they need done at that single location without worrying about which consumer will do it, and 2) function as a load-balancing mechanism for the consumers. (Additionally, a work queue can act as a "buffer" if producers temporarily generate work too fast for the consumers.) If you have a dedicated pair of producer-consumer threads for each client, it calls into question why you need to use queues at all. Why not just do a synchronous "pass off" from dedicated producer to corresponding dedicated consumer? Or, why not use a single thread per client which acts as both producer and consumer? Using queues in the way which you are proposing doesn't seem to really gain anything.