I am using grpc to communicate between a Java server and node client. When either of them die/crash the other dies to.
Here is the exception thrown on Java server when node client dies -
Nov 08, 2016 11:28:03 AM io.grpc.netty.NettyServerHandler onConnectionError
WARNING: Connection Error
java.io.IOException: An existing connection was forcibly closed by the remote host
at sun.nio.ch.SocketDispatcher.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:43)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:192)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
at io.netty.buffer.PooledUnsafeDirectByteBuf.setBytes(PooledUnsafeDirectByteBuf.java:288)
at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1100)
at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:349)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:112)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:571)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:512)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:426)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:398)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:877)
at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
at java.lang.Thread.run(Thread.java:745)
Here is the exception thrown on node when java dies:
events.js:141
throw er; // Unhandled 'error' event
^
Error: {"created":"#1478623914.082000000","description":"An existing connection was forcibly closed by the remote host.\r\n","file":"..\src\core\lib\iomgr\tcp_windows.c","file_line":171,"grpc_status":14}
at ClientReadableStream._emitStatusIfDone ( C:\HarshalDev\Live_TDFX\TDSL0007-TDFXPlatfo rmsDesignandBuild\tdfxlive\node_modules\grpc\src\node\src\client.js:189:19)
at ClientReadableStream._receiveStatus (C:\ HarshalDev\Live_TDFX\TDSL0007-TDFXPlatformsDesignandBuild\tdfxlive\node_modules\grpc\src\node\src\client.js:169:8)
at C:\HarshalDev\Live_TDFX\TDSL0007-TDFXPlatformsDesignandBuild\tdfxlive\node_modules\grpc\src\node\src\client.js:577:14
[nodemon] app crashed - waiting for file changes before starting...
QUESTION - How do I handle those exceptions?
I tried without any success adding try/catch and adding a uncaught exception handlers for threads.
Java code to initialize -
ServerBuilder<?> serverBuilder = ServerBuilder.forPort(getPort());
server = serverBuilder
.addService(createBwayStreamingService())
.addService(ServerInterceptors.intercept(createBwayOrderService(), authServerInterceptor))
.addService(createBwayInstrumentService())
.addService(createBwaySettlementService())
.addService(createBwayDateTimeService())
.addService(ServerInterceptors.intercept(createBwayConfService(), authServerInterceptor))
.addService(ServerInterceptors.intercept(createBwayTradeService(), authServerInterceptor))
.addService(ServerInterceptors.intercept(createBwayAuthService(), authServerInterceptor))
.build();
Preconditions.checkNotNull(server);
server.start();
System.out.println("Server started, listening on " + getPort());
Runtime.getRuntime().addShutdownHook(new Thread() {
#Override
public void run() {
System.out.println("Shutting down gRPC server");
TocGateway.this.stop();
System.out.println("Server shut down");
}
});
server.awaitTermination();
Node client handler (one of the services, all other services use the same pattern) -
let protoDescriptorStreaming = grpc.load((process.env.FX_LIVE_PROTO_DIR || '../tocGateway/src/main/proto') + '/streaming.proto');
let streamingService = new protoDescriptorStreaming.tds.fxlive.bway.BwayStreamService(process.env.TOC_GATEWAY_ADDRESS || 'localhost:8087', grpc.credentials.createInsecure());
The Java warning is harmless. Your application should continue running normally after the exception. It would be nice to reduce the log level to have less logspam, but it also shouldn't impact the correctness of your application.
Related
I have a web service built on spring-camel. I am trying to integrate the grpc server using grpc-spring-boot-starter. My implementation of grpc service is as below.
#GrpcService
public class GreetingServiceImpl extends GreetingServiceGrpc.GreetingServiceImplBase {
#Override
public void processGrpcRequest(GreetingRequest request, StreamObserver<GreetingResponse> responseObserver) {
String receivedMessage = request.getRequest();
GreetingResponse response = GreetingResponse.newBuilder()
.setResponse("Your message received " + receivedMessage).build();
responseObserver.onNext(response);
responseObserver.onCompleted();
}
}
I package my web service as a war and I find no error during deployement of the war file in my application server. However, when I try to communicate with my grpc server, I get the following error message.
Exception in thread "main" 15:18:01.363 [grpc-nio-worker-ELG-1-2] DEBUG io.grpc.netty.shaded.io.grpc.netty.NettyClientHandler - [id: 0x02a01ffb, L:/127.0.0.1:56000 - R:localhost/127.0.0.1:9089] INBOUND PING: ack=true bytes=1234
io.grpc.StatusRuntimeException: UNIMPLEMENTED: HTTP status code 404
invalid content-type: text/html
headers: Metadata(:status=404,content-type=text/html,date=Fri, 04 Mar 2022 09:48:01 GMT,content-length=74)
DATA-----------------------------
<html><head><title>Error</title></head><body>404 - Not Found</body></html>
at io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:262)
at io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:243)
at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:156)
at com.test.grpc.GreetingServiceGrpc$GreetingServiceBlockingStub.processGrpcRequest(GreetingServiceGrpc.java:156)
at com.test.grpc.GrpcClient.main(GrpcClient.java:16)
I have recreated the same issue with minimal setup and code is available here github . Please can anyone help on this. Thanks
Example Code(JAVA):
Cluster cluster = Cluster.open(yml.getFile());
DriverRemoteConnection driver = DriverRemoteConnection.using(cluster, "graph_traversal");
GraphTraversalSource allGraph = AnonymousTraversalSource.traversal().withRemote(driver);
//Compile Script
GremlinScriptEngine engine = new GremlinGroovyScriptEngine();
String script = "graph_traversal.V().outE().inV().path().unfold().dedup().group().by{\"category\"}";
SimpleBindings bind = new SimpleBindings();
GraphTraversal compiled = (GraphTraversal)engine.eval(script, bind);
//Send bytecode to remote server
CompletableFuture<RemoteTraversal<?, Object>> result = driver.submitAsync(compiled.asAdmin().getBytecode());
result.get(); // Exception
I'm trying to send a gremlin bytecode to remote server through driver.
But the codes occurs an exception when the script includes 'lamda'.
The exception message is as following.
Exception:
io.netty.handler.codec.EncoderException: org.apache.tinkerpop.gremlin.driver.exception.ResponseException: An error occurred during serialization of this request [RequestMessage{, requestId=84d5d022-1b08-41a6-b57f-8fdc3b5b6c65, op='bytecode', processor='traversal', args={gremlin=[..., dedup(), unfold(), dedup(), group(), by(Script1$_run_closure1#78b612c6)]], aliases={g=graph_traversal}}}] - it could not be sent to the server - Reason: org.apache.tinkerpop.gremlin.driver.ser.SerializationException: java.lang.IllegalArgumentException: Class is not registered: java.lang.reflect.InvocationHandler
Note: To register this class use: kryo.register(java.lang.reflect.InvocationHandler.class);
if the script doesn't contain the lambda, it won't make any exception.
How can I resolve this?
Thank you.
Solved:
By using Lambda.Methods.
I have a problem with WebSocket being disconnected while trying to firstly save entity to the database with EF Core, then to dispatch a message to the clients using SignalR Core.
Everything works perfectly when I separate these two operations, one with AJAX call to the controller's action, to save entity to the database, one with hub method for dispatching messages to the clients. But I want to ensure that the entity is successfully saved, then to be dispatched.
When I try to merge saving entity and dispatching a message to the clients into the same hub method or controller's action (with dependency injection), I've got the errors which could be found down bellow (with log level - trace).
00:51:03.876 [2020-01-04T23:51:03.877Z] Trace: (WebSockets transport) sending data. String data of length 307. Utils.ts:178:39
00:51:04.147 [2020-01-04T23:51:04.148Z] Trace: (WebSockets transport) socket closed. Utils.ts:178:39
00:51:04.148 [2020-01-04T23:51:04.148Z] Debug: HttpConnection.stopConnection(undefined) called while in state Connected. Utils.ts:178:39
00:51:04.148 [2020-01-04T23:51:04.149Z] Information: Connection disconnected. Utils.ts:174:39
00:51:04.149 [2020-01-04T23:51:04.149Z] Debug: HubConnection.connectionClosed(undefined) called while in state Connected. Utils.ts:178:39
00:51:04.149 [2020-01-04T23:51:04.149Z] Information: Connection reconnecting. Utils.ts:174:39
00:51:04.150 [2020-01-04T23:51:04.150Z] Information: Reconnect attempt number 1 will start in 0 ms. Utils.ts:174:39
00:51:04.150 Chat - Error: Invocation canceled due to the underlying connection being closed. conversation line 2 > scriptElement:27:36
00:51:04.153 [2020-01-04T23:51:04.153Z] Debug: Starting connection with transfer format 'Text'. Utils.ts:178:39
00:51:04.154 [2020-01-04T23:51:04.154Z] Debug: Sending negotiation request: http://localhost:11597/chatHub/negotiate?negotiateVersion=1. Utils.ts:178:39
00:51:04.168 [2020-01-04T23:51:04.169Z] Debug: Selecting transport 'WebSockets'. Utils.ts:178:39
00:51:04.168 [2020-01-04T23:51:04.169Z] Trace: (WebSockets transport) Connecting. Utils.ts:178:39
00:51:04.179 [2020-01-04T23:51:04.180Z] Information: WebSocket connected to ws://localhost:11597/chatHub?id=Amf9CsFZQfnR8-3PoGr8HQ. Utils.ts:174:39
00:51:04.179 [2020-01-04T23:51:04.180Z] Debug: The HttpConnection connected successfully. Utils.ts:178:39
00:51:04.180 [2020-01-04T23:51:04.180Z] Debug: Sending handshake request. Utils.ts:178:39
00:51:04.180 [2020-01-04T23:51:04.181Z] Debug: Hub handshake failed with error 'WebSocket is not in the OPEN state' during start(). Stopping HubConnection. Utils.ts:178:39
00:51:04.181 [2020-01-04T23:51:04.181Z] Trace: (WebSockets transport) socket closed. Utils.ts:178:39
00:51:04.181 [2020-01-04T23:51:04.182Z] Debug: HttpConnection.stopConnection(undefined) called while in state Disconnecting. Utils.ts:178:39
00:51:04.182 [2020-01-04T23:51:04.182Z] Error: Connection disconnected with error 'WebSocket is not in the OPEN state'. Utils.ts:168:39
00:51:04.183 [2020-01-04T23:51:04.184Z] Debug: HubConnection.connectionClosed(WebSocket is not in the OPEN state) called while in state Reconnecting. Utils.ts:178:39
00:51:04.183 [2020-01-04T23:51:04.184Z] Information: Reconnect attempt failed because of error 'WebSocket is not in the OPEN state'. Utils.ts:174:39
00:51:04.184 [2020-01-04T23:51:04.185Z] Information: Reconnect attempt number 2 will start in 2000 ms. Utils.ts:174:39
Here is hub method:
public async Task SendMessage(Message message)
{
// Eager loading conversation with matching Id
var chat = _context.Chat
.Include(c => c.PersonA)
.Include(c => c.PersonB)
.FirstOrDefault(m => m.Id == message.ChatId);
/*
* I'm doing a few validations here
*/
// Saving entity to the database
await _context.AddAsync(new Message
{
SenderId = message.SenderId,
ChatId = message.ChatId,
Text = message.Text
});
await _context.SaveChangesAsync();
// Dispatching message to the clients using strongly-typed hub
var usersId = new List<string> { chat.PersonAId, chat.PersonBId };
await Clients
.Users(usersId)
.ReceiveMessage(message);
}
The potential problem could lie in timing, the dispatching can't wait for the EF to execute all operations, because when I just load the conversation It works good, adding more complexity, it breaks down.
I try to receive user's PushNotification Channel URI (Windows 10 platform) and for some users application generate exception This operation returned because the timeout.
To handle errors in a task chain I should add a task-based continuation at the end of the chain and handle all errors there (as explained here https://learn.microsoft.com/en-us/windows/uwp/threading-async/asynchronous-programming-in-cpp-universal-windows-platform-apps#handling-errors-in-a-task-chain).
So I did that. But after I call t.get(); system generate Platform::COMException^ exception by it not catch in try-catch block. Why?
There is the code:
{
create_task(PushNotificationChannelManager::CreatePushNotificationChannelForApplicationAsync())
.then([this](PushNotificationChannel^ pnChannel)
{
// ..
// DONE: pnChannel->Uri
// ..
}, task_continuation_context::get_current_winrt_context())
.then([](task<void> t)
{
try
{
t.get(); // <<<< After exec this line app crash!
}
catch (Platform::COMException^ e)
{
OutputDebugString(L"Exception catches!");
}
});
}
There is the full exception message:
Exception thrown at 0x00007FFD9D74A388 in GameName.exe: Microsoft C++
exception: Platform::COMException ^ at memory location 0x000000249A9FEB60.
HRESULT:0x800705B4 This operation returned because the timeout period expired.
WinRT information: This operation returned because the timeout
And Visual Studio throws me to the file c:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\include\exception to block:
[[noreturn]] void _RethrowException() const
{
__ExceptionPtrRethrow(this); // <<<< here everything stoped!
}
UPDATED:
Enviroment:
Visual Studio 2017 (15.7.3)
Project based on cocos2d-x (3.16)
Project Targe Platform Version = 10.0.1493.0
Project Platform Toolset = Visual Studio 2015 (v140)
You can clone cocos2dx project and in MainScene paste code I showed before in onEnter method (or anywhere).
I got a situation when I had crash 100% for me
Uninstall app if it was built before;
Disconnect from the internet;
Build app & launch;
The app will try to detect Channel URI & will crash (but with crash message = WinRT information: The network is not present or not started).
I understand that throwing that exception is normal. But I still cannot understand why when I call t.get() it not catch exception Platform::COMException^
I have the Following CouchBase Template Bean:
#PostConstruct
public void initIt() throws Exception {
if(couchbaseDisabled)
return;
couchbaseClient= new CouchbaseClient(
bootstrapUris(Arrays.asList(hosts.split(","))),
CouchbaseConstants.BUCKET,
""
);
couchbaseTemplate();
}
public void couchbaseTemplate() throws Exception {
logger.info("Enabling CouchBase Template");
couchbaseTemplate= new CouchbaseTemplate(couchbaseClient);
//couchbaseTemplate.
}
and
#PreDestroy
public void cleanup() throws Exception {
logger.info("Closing couchbase connection.");
if (couchbaseClient != null) {
couchbaseClient.shutdown();
couchbaseTemplate=null;
couchbaseClient=null;
}
}
While the Server is being Shut Down i am geting the Following Logs:
SEVERE: The web application [] registered the JDBC driver [com.mysql.jdbc.Driver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
Jan 8, 2016 4:57:24 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal#40c94525]) and a value of type [com.couchbase.client.deps.io.netty.util.internal.InternalThreadLocalMap] (value [com.couchbase.client.deps.io.netty.util.internal.InternalThreadLocalMap#5ddaa15d]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
Jan 8, 2016 4:57:24 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal#40c94525]) and a value of type [com.couchbase.client.deps.io.netty.util.internal.InternalThreadLocalMap] (value [com.couchbase.client.deps.io.netty.util.internal.InternalThreadLocalMap#3c9810ce]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
Jan 8, 2016 4:57:24 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal#40c94525]) and a value of type [com.couchbase.client.deps.io.netty.util.internal.InternalThreadLocalMap] (value [com.couchbase.client.deps.io.netty.util.internal.InternalThreadLocalMap#23776376]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
Jan 8, 2016 4:57:24 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal#40c94525]) and a value of type [com.couchbase.client.deps.io.netty.util.internal.InternalThreadLocalMap] (value [com.couchbase.client.deps.io.netty.util.internal.InternalThreadLocalMap#7322ea2a]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.
Jan 8, 2016 4:57:32 PM org.apache.coyote.http11.Http11Protocol destroy
INFO: Stopping Coyote HTTP/1.1 on http-8099
What can be Done Here?
Ok so you have both SDK 1.4.x and 2.x running in your application (since you have com.couchbase.client:java-client in your pom).
The thread leak message comes from the later. You must have instantiated a Cluster somewhere (as in com.couchbase.client.java.Cluster).
Make sure to also clean it up at the end of the application's lifecycle by calling cluster.disconnect() (I guess from a #PreDestroy method, as you did for the CouchbaseClient).
If you also created a custom CouchbaseEnvironment, you have to also properly shut it down (in the same method as the Cluster cleanup) by calling environment.shutdownAsync().toBlocking().single().
Make sure to use the latest version of the 2.x SDK as some older versions had bugs relative to proper thread cleanup on shutdown (see JCBC-773 and JVMCBC-251 issues).