We are trying to apply gRPC bidirectional streaming. Proto:
message Request {
oneof requestTypes{
ConfigRequest configRequest = 1;
DataRequest dataRequest = 2;
}
}
message ConfigRequest {
request_quantity = 1;
...
}
message DataRequest {
string id = 1;
bytes data = 2;
...
}
service Service {
rpc FuncService(stream Request) returns (stream Response);
}
The client side is written in C# and contains asynchrony.
If necessary, I can clarify the client code.
The server side is in python. Python code:
def FuncServe(self, request_iterator, context):
i = 0
for request in request_iterator:
...
if (i==request_quantity):
break
i+=1
The problem is the server hangs in a loop on requests, so I introduced an if-break statement. So far, everything is working, but we would like to work out the option when not all the requests stated in the config have arrived. In this case, we cannot avoid suspension in the loop and catch it. Also, I was unable to move the loop into a separate thread to control the execution time, it ends due to an empty iterator.
I would be very grateful for help in this loop problem.
Related
I use aws-neptune.
And I try to implement my queries as transactional(with sessionClient like: https://docs.aws.amazon.com/neptune/latest/userguide/access-graph-gremlin-sessions.html). But when I try to implement it, closing client throws exception. There is similar issue like my case: https://groups.google.com/g/janusgraph-users/c/N1TPbUU7Szw
My code looks like:
#Bean
public Cluster gremlinCluster()
{
return Cluster.build()
.addContactPoint(GREMLIN_ENDPOINT)
.port(GREMLIN_PORT)
.enableSsl(GREMLIN_SSL_ENABLED)
.keyCertChainFile("classpath:SFSRootCAG2.pem")
.create();
}
private void runInTransaction()
{
String sessionId = UUID.randomUUID().toString();
Client.SessionedClient client = cluster.connect(sessionId);
try
{
client.submit("query...");
}
finally
{
if (client != null)
{
client.close();
}
}
}
And exception is:
INFO (ConnectionPool.java:225) - Signalled closing of connection pool on Host{address=...} with core size of 1
WARN (Connection.java:322) - Timeout while trying to close connection on ... - force closing - server will close session on shutdown or expiration.
java.util.concurrent.TimeoutException
at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1771)
Is there any suggestion?
This might be a connectivity problem with the server which you are not able to observe while sending the query because you are not waiting for the future to complete.
When you do a client.submit("query...");, you receive a future. You need to wait for that future to complete to observe any exceptions (or success).
I would suggest the following:
Try hitting the server with a health status call using curl to verify connectivity with the server.
Replace the client.submit("query..."); with client.submit("query...").all().join(); to get the error during connection with the server.
I am new to GRPC so please let me know if I am doing something wrong here. I am looking at the greeter_async_server.cc example code. This seems to work fine for normal requests but I wanted to simulate a request getting stuck on the server so I added a sleep in the processing loop. I added this right before Finish is called on the responder so that it was in the actual processing logic of the request. While the server thread is sleeping it will not accept any new requests until the thread is free. I attempted to create another client request while the original request on the server is sleeping but the grpc server would not process the request. The client seemed to be stuck until the server came out of the sleep.
I also broke this process into debugger as well but the only request I saw was the one that was sleeping. The other threads were waiting on the completion queue.
I am new to grpc so if I am doing this wrong please let me know what I need to do to handle request while another request is stuck.
void Proceed() {
if (status_ == CREATE) {
// Make this instance progress to the PROCESS state.
status_ = PROCESS;
// As part of the initial CREATE state, we *request* that the system
// start processing SayHello requests. In this request, "this" acts are
// the tag uniquely identifying the request (so that different CallData
// instances can serve different requests concurrently), in this case
// the memory address of this CallData instance.
service_->RequestSayHello(&ctx_, &request_, &responder_, cq_, cq_,
this);
} else if (status_ == PROCESS) {
// Spawn a new CallData instance to serve new clients while we process
// the one for this CallData. The instance will deallocate itself as
// part of its FINISH state.
new CallData(service_, cq_);
// The actual processing.
std::string prefix("Hello ");
reply_.set_message(prefix + request_.name());
Sleep((DWORD)-1);
// And we are done! Let the gRPC runtime know we've finished, using the
// memory address of this instance as the uniquely identifying tag for
// the event.
status_ = FINISH;
responder_.Finish(reply_, Status::OK, this);
} else {
GPR_ASSERT(status_ == FINISH);
// Once in the FINISH state, deallocate ourselves (CallData).
delete this;
}
}
I'm facing problem with kestrel server's performance. I have following scenario :
TestClient(JMeter) -> DemoAPI-1(Kestrel) -> DemoAPI-2(IIS)
I'm trying to create a sample application that could get the file content as and when requested.
TestClient(100 Threads) requests to DemoAPI-1 which in turn request to DemoAPI-2. DemoAPI-2 reads a fixed XML file(1 MB max) and returns it's content as a response(In production DemoAPI-2 is not going to be exposed to outside world).
When I tested direct access from TestClient -> DemoAPI-2 I got expected result(good) which is following :
Average : 368ms
Minimum : 40ms
Maximum : 1056ms
Throughput : 40.1/sec
But when I tried to access it through DemoAPI-1 I got following result :
Average : 48232ms
Minimum : 21095ms
Maximum : 49377ms
Throughput : 2.0/sec
As you can see there is a huge difference.I'm not getting even the 10% throughput of DemoAPI-2. I was told has kestrel is more efficient and fast compared to traditional IIS. Also because there is no problem in direct access, I think we can eliminate the possible of problem on DemoAPI-2.
※Code of DemoAPI-1 :
string base64Encoded = null;
var request = new HttpRequestMessage(HttpMethod.Get, url);
var response = await this.httpClient.SendAsync(request, HttpCompletionOption.ResponseContentRead).ConfigureAwait(false);
if (response.StatusCode.Equals(HttpStatusCode.OK))
{
var content = await response.Content.ReadAsByteArrayAsync().ConfigureAwait(false);
base64Encoded = Convert.ToBase64String(content);
}
return base64Encoded;
※Code of DemoAPI-2 :
[HttpGet("Demo2")]
public async Task<IActionResult> Demo2Async(int wait)
{
try
{
if (wait > 0)
{
await Task.Delay(wait);
}
var path = Path.Combine(Directory.GetCurrentDirectory(), "test.xml");
var file = System.IO.File.ReadAllText(path);
return Content(file);
}
catch (System.Exception ex)
{
return StatusCode(500, ex.Message);
}
}
Some additional information :
Both APIs are async.
Both APIs are hosted on different EC2 instances(C5.xlarge Windows Server 2016).
DemoAPI-1(kestrel) is a self-contained API(without reverse proxy)
TestClient(jMeter) is set to 100 thread for this testing.
No other configuration is done for kestrel server as of now.
There are no action filter, middleware or logging that could effect the performance as of now.
Communication is done using SSL on 5001 port.
Wait parameter for DemoAPI2 is set to 0 as of now.
The CPU usage of DEMOAPI-1 is not over 40%.
The problem was due to HttpClient's port exhaustion issue.
I was able to solve this problem by using IHttpClientFactory.
Following article might help someone who faces similar problem.
https://www.stevejgordon.co.uk/httpclient-creation-and-disposal-internals-should-i-dispose-of-httpclient
DEMOAPI-1 performs a non-asynchronous read of the streams:
var bytes = stream.Read(read, 0, DataChunkSize);
while (bytes > 0)
{
buffer += System.Text.Encoding.UTF8.GetString(read, 0, bytes);
// Replace with ReadAsync
bytes = stream.Read(read, 0, DataChunkSize);
}
That can be an issue with throughput on a lot of requests.
Also, I'm not fully aware of why are you not testing the same code with IIS and Kestrel, I would assume you need to make only environmental changes and not the code.
I'm implementing a Thrift client in order to make connection to a built-in scribe server.
Everything is going OK if I use a standard Log method, like this:
public boolean log(List<LogEntry> messages) {
boolean ret = false;
PooledClient client = borrowClient();
try {
if ((client != null) && (client.getClient() != null)) {
ResultCode result = client.getClient().Log(messages);
ret = (result != null && result.equals(ResultCode.OK));
returnClient(client);
}
} catch (Exception ex) {
logger.error(LogUtil.stackTrace(ex));
invalidClient(client);
}
return ret;
}
However, when I use send_Log instead:
public void send_Log(List<LogEntry> messages) {
PooledClient client = borrowClient();
try {
if ((client != null) && (client.getClient() != null)) {
client.getClient().send_Log(messages);
returnClient(client);
}
} catch (Exception ex) {
logger.error(LogUtil.stackTrace(ex));
invalidClient(client);
}
}
It acctually causes some problems:
Total network connection to port 1463 (default port for a scribe server) is going to increase so much, and always in a CLOSE_WAIT state.
Cause my application got stuck without throwing any error, I think it may be an issue with network connection.
what if send without recv
As this is clearly TCP, the sender will block (in blocking mode), or incur EAGAIN/EWOULDBLOCK in non-blocking mode. EDIT It is now clear that you want to send without receiving the reply. You can do that by just sending and then closing the socket, but that may cause the peer to incur ECONNRESET, which may upset it. You should really implement the application protocol correctly.
1/ Total network connection to port 1463 (default port for a scribe server) is going to increase so much, and always in a CLOSE_WAIT state.
Lots of ports in CLOSE_WAIT state indicates a socket leak on the part of the local application.
2/ Cause my application got stuck without throwing any error. I think it may be an issues with network connection.
It is an issue with sending and not receiving.
Since you labelled this as a Thrift related question, the answer is oneway.
service foo {
oneway void FireAndForget(1: some args)
}
The oneway keyword does exactly what the name suggests. You get a client implementation that only sends and does not wait for anything to be returned from the server. This rule also includes exceptions. Hence a oneway method must always be void and can't throw any exceptions.
However, when I use send_Log instead ...
client.getClient().send_Log(messages);
Neither one of the Thrift-generated send_Xxx and recv_Xxx methods are meant to be public. That's why they are usually either private or protected methods. They should not be called directly, unless you are sure that you know what you are doing (and very obviously the latter is not the case here).
And since the real question is about performance: Why don't you just delegate the call(s) into a secondary thread? That way the I/O will not block the UI.
I'm working my way through boost's asio tutorial. I'm looking into their chat example. More specifically, I'm trying to split their chat client from a sender+receiver, to just a sender and just a receiver, but I'm seeing some behaviour that I can't explain.
The setup consists of:
boost::asio::io_service io_service;
tcp::resolver::iterator endpoint = resolver.resolve(...);
boost::thread t(boost::bind(&boost::asio::io_service::run, &io_service));
boost::asio::async_connect(socket, endpoint, bind(handle_connect, ... ));
The sending portion effectively conisists of:
while (std::cin.getline(str))
io_service.post( do_write, str );
and
void do_write (string str)
{
boost::asio::async_write(socket, str, bind( handle_write, ... ));
}
The receive section consists of
void handle_connect(...)
{
boost::asio::async_read(socket, read_msg_, bind(handle_read, ...));
}
void handle_read(...)
{
std::cout << read_msg_;
boost::asio::async_read(socket, read_msg_, bind(handle_read, ...));
}
If I comment out the content of handle_connect to isolate the send portion, my other client (compiled using the original code) does not receive anything. If I revert, then comment out the content of handle_read, my other client only receives the first message.
Why is it necessary to call async_read() in order to be able to post() an async_write()?
The full unmodified code is linked above.
The problem here is that, your io_service is running out of work and stops processing requests even before you start sending your chat messages.
If you comment out the body of handle_connect, then the only work it had to do was to dispatch the handle_connect handler and then execute it once the connection was done.
std::size_t scheduler::run(asio::error_code& ec)
{
.....
mutex::scoped_lock lock(mutex_);
std::size_t n = 0;
for (; do_run_one(lock, this_thread, ec); lock.lock())
if (n != (std::numeric_limits<std::size_t>::max)())
++n;
return n;
}
So, you have to provide it with something in it's operation queue. This was done with handle_read_header handler in the original code as this handler would always be in the need of servicing till the client gets something from the server.
You can do what you want to do by providing work to the io_service.
asio::io_context io_context;
asio::io_context::work wrk(io_context); // make `run` run forever
tcp::resolver resolver(io_context);
tcp::resolver::results_type endpoints = resolver.resolve(argv[1], argv[2]);
chat_client c(io_context, endpoints);
asio::thread t(boost::bind(&asio::io_context::run, &io_context));