We use gRPC in python to connect to external metrics collector, and we found in the timer thread in gRPC, the clocktype set in the condition is realtime type, what could I do to make it changed to monotonic?
Thanks.
Related
We have a service which sends grpc requests under heavy load.
Normally, grpc send is considered to be an IO stage, which means it should use an IO thread pool.
We've noticed that the serialization time of the proto objects consumes rather a-lot of cpu, which is not suitable to use with an IO thread pool.
Is there some way to separate the serialization step from the IO step, e.g can we somehow do something like:
val bytes = myProto.toBytes()
myService.send(bytes)
We have no control over the .proto files and the server, but we can generate a client and a stub. We currently use scalapb but java might do fine as well.
Another option is to somehow translate the client calls to http2 calls, anyone ever tried that?
Any other suggestions on how to tackle this issue?
Thanks in advance.
gRPC itself separates protobuf encoding from transmission. Protobuf encoding and decoding is performed on application threads: the threads calling gRPC and the threads delivering callbacks (e.g., channelBuilder.executor()). Then the I/O is processed asynchronously using separate network threads (Netty event loops). Just interact with gRPC on threads that can handle the protobuf CPU cost.
I am working on an orchestration system using Node-RED and MQTT.
I have decided to dissociate the event acquisition from the treatment. The main objective is to quickly push events on a queue and treat them soon as possible in real time.
The system operates like this :
I receive an event on an HTTP Rest API,
Push this event on an MQTT Topic,
On an another flow, listen and read events from the MQTT Topic,
Launch several actions/process from this event (up to 5/10 seconds).
But I am facing an issue: If I receive too quickly 2 related events, the second event could change the processing of the first event. To solve this, I would like to synchronize my event consumption/processing in order to keep them ordered.
MQTT QoS 2 messages will be delivered in order. How can I simply implement a synchronization paradigm in Node-RED ? Is it possible to avoid MQTT Client listening while processing an event ?
No, you can't turn the MQTT client off.
And no there is no concept of synchronisation, mainly because all NodeJS apps are purely single threaded so 2 things can't actually happen at once, tasks just yield normally when they get to something IO bound.
I'm not sure you actually gain anything receiving it via HTTP and then re-consuming it via MQTT.
If you want to queue the incoming events up you could use the delay node to rate limit the input to something you are sure the processing can manage. The rate limit option has 2 modes, one that drops messages and one the queues them,.
Place a simple message Q on output of MQTT client and get it to release next message by sending it trigger=true (or might be able to get it to release a message at a set rate). I am still looking at these.
https://flows.nodered.org/node/node-red-contrib-simple-message-queue
Here is my proposal to solve the problem: https://flows.nodered.org/flow/3003f194750b0dec19502e31ca234847.
Sync req/res REST API with async workers based on MQTT
Example implements one REST API endpoint, one inject node for doing requests to that endpoint and two workers doing their job async.
It's still in progress so expect changes. Feel free to contact me and chat about this or other solutions for orchestration along with Node-RED.
I'm trying to understand the idea of non-blocking web server and it seems like there is something I miss.
I can understand there are several reasons for "block" web request(psuedocode):
CPU bound
string on_request(arg)
{
DO_SOME_HEAVY_CPU_CALC
return "done";
}
IO bound
string on_request(arg)
{
DO_A_CALL_TO_EXTERNAL_RESOURCE_SUCH_AS_WEB_IO
return "done";
}
sleep
string on_request(arg)
{
sleep(VERY_VERY_LONG_TIME);
return "done";
}
are all the three can benefit from non-blocking server?
how the situation that do benefit from the non-blocking web server really do that?
I mean, when looking at the Tornado server documentation, it seems
like it "free" the thread. I know that a thread can be put to sleep
and wait for a signal from the operation system (at least in Linux),
is this the meaning of "freeing" the thread? is this some higher
level implementation? something that actually create a new thread
that is waiting for new request instead of the "sleeping" one?
Am I missing something here?
Thanks
Basically the way the non-blocking sockets I/O work is by using polling and the state machine. So your scheme for many connections would be something like that:
Create many sockets and make them nonblocking
Switch the state of them to "connect"
Initiate the connect operation on each of them
Poll all of them until some events fire up
Process the fired up events (connection established or connection failed)
Switch the state those established to "sending"
Prepare the Web request in a buffer
Poll "sending" sockets for WRITE operation
send the data for those who got the WRITE event set
For those which have all the data sent, switch the state to "receiving"
Poll "receiving" sockets for READ operation
For those which have the READ event set, perform read and process the read data according to the protocol
Repeat if the protocol is bidirectional, or close the socket if it is not
Of course, at each stage you need to handle errors, and that the state of each socket is different (one may be connecting while another may be already reading).
Regarding polling I have posted an article about how different polling methods work here: http://www.ulduzsoft.com/2014/01/select-poll-epoll-practical-difference-for-system-architects/ - I suggest you check it.
To benefit from a non-blocking server, your code must also be non-blocking - you can't just run blocking code on a non-blocking server and expect better performance. For example, you must remove all calls to sleep() and replace them with non-blocking equivalents like IOLoop.add_timeout (which in turn involves restructuring your code to use callbacks or coroutines).
How To Use Linux epoll with Python http://scotdoyle.com/python-epoll-howto.html may give you some points about this topic.
I'm building an ASP.NET service (a simple aspx) that requires a REQ call to a ZeroMQ REP node.
So I've to use the REQ/REP pattern, but I can't figure out the proper way to initialize the ZeroMQ context in the ASP.NET pipeline.
Moreover, can I share a single connection among the different ASP.NET threads and if so how?
edit: After some study it looks to me that an inproc router in a dedicated thread should be the way to go, since it would handle sincronization.
But more questions arise:
The other end of such an inproc node should be a DEALER? If so, should it connect to the REQ node? Or it should bind to a tcp port and I should code the REP server node to connect to it (the latter would be a bit cumbersome, since I could have different servers exposing the service)?
As an alternative, is it correct to build an inproc node bound to a ROUTER socket at one end, and connecting with REQ on the other? If so, should I code the node so that it handles a manual envelop of each message just to be able to send responses back to the correct requesting thread?
Is Application_Start the correct pipeline point to initialize the thread handling such router?
At the moment a ROUTER/DEALER inproc node that connect to the REQ server looks like the best option, but I'm not sure that it's possibile to connect from a DEALER socket. But this is still just a speculation and could be entirely wrong.
The zmq_socket manual states:
ØMQ sockets are not thread safe. Applications MUST NOT use a socket
from multiple threads except after migrating a socket from one thread
to another with a "full fence" memory barrier.
What I am trying to solve: have an Erlang TCP server that listens on a specific port (the code should reside in some kind of external facing interface/API) and each incoming connection should be handled by a gen_server (that is even the gen_tcp:accept should be coded inside the gen_server), but I don't actually want to initially spawn a predefined number of processes that accepts an incoming connection). Is that somehow possible ?
Basic Procedure
You should have one static process (implemented as a gen_server or a custom process) that performs the following procedure:
Listens for incoming connections using gen_tcp:accept/1
Every time it returns a connection, tell a supervisor to spawn of a worker process (e.g. another gen_server process)
Get the pid for this process
Call gen_tcp:controlling_process/2 with the newly returned socket and that pid
Send the socket to that process
Note: You must do it in that order, otherwise the new process might use the socket before ownership has been handed over. If this is not done, the old process might get messages related to the socket when the new process has already taken over, resulting in dropped or mishandled packets.
The listening process should only have one responsibility, and that is spawning of workers for new connections. This process will block when calling gen_tcp:accept/1, which is fine because the started workers will handle ongoing connections concurrently. Blocking on accept ensure the quickest response time when new connections are initiated. If the process needs to do other things in-between, gen_tcp:accept/2 could be used with other actions interleaved between timeouts.
Scaling
You can have multiple processes waiting with gen_tcp:accept/1 on a single listening socket, further increasing concurrency and minimizing accept latency.
Another optimization would be to pre-start some socket workers to further minimize latency after accepting the new socket.
Third and final, would be to make your processes more lightweight by implementing the OTP design principles in your own custom processes using proc_lib (more info). However, this you should only do if you benchmark and come to the conclusion that it is the gen_server behavior that slows you down.
The issue with gen_tcp:accept is that it blocks, so if you call it within a gen_server, you block the server from receiving other messages. You can try to avoid this by passing a timeout but that ultimately amounts to a form of polling which is best avoided. Instead, you might try Kevin Smith's gen_nb_server instead; it uses an internal undocumented function prim_inet:async_accept and other prim_inet functions to avoid blocking.
You might want to check out http://github.com/oscarh/gen_tcpd and use the handle_connection function to convert the process you get to a gen_server.
You should use "prim_inet:async_accept(Listen_socket, -1)" as said by Steve.
Now the incoming connection would be accepted by your handle_info callback
(assuming you interface is also a gen_server) as you have used an asynchronous
accept call.
On accepting the connection you can spawn another ger_server(I would recommend
gen_fsm) and make that as the "controlling process" by calling
"gen_tcp:controlling_process(CliSocket, Pid of spwned process)".
After this all the data from socket would be received by that process
rather than by your interface code. Like that a new controlling process
would be spawned for another connection.