I have a mule application hosted on Cloudhub which invokes the same API say "example.com" in Parallel using Scatter Gather and HTTP Request component.
The application is hosted on Cloudhub runtime 4.1.5. Everything looks working fine but we get intermittent exception message as:
HTTP POST on resource 'https://example.com:443/<path>/<res>' failed: AsyncHttpClient has been closed..
The part of stack trace is:
Root Exception stack trace:
java.io.IOException: AsyncHttpClient has been closed.
at com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider.execute(GrizzlyAsyncHttpProvider.java:121)
at com.ning.http.client.AsyncHttpClient.executeRequest(AsyncHttpClient.java:505)
at org.mule.service.http.impl.service.client.GrizzlyHttpClient.sendAsync(GrizzlyHttpClient.java:364)
at com.mulesoft.service.http.impl.service.client.EEGrizzlyHttpClient.sendAsync(EEGrizzlyHttpClient.java:141)
at org.mule.extension.http.internal.request.HttpRequesterConnectionManager$ShareableHttpClient.sendAsync(HttpRequesterConnectionManager.java:104)
at org.mule.extension.http.internal.request.client.HttpExtensionClient.send(HttpExtensionClient.java:72)
at org.mule.extension.http.internal.request.HttpRequester.doRequestWithRetry(HttpRequester.java:124)
at org.mule.extension.http.internal.request.HttpRequester.doRequest(HttpRequester.java:103)
at org.mule.extension.http.internal.request.HttpRequestOperations.request(HttpRequestOperations.java:131)
at sun.reflect.GeneratedMethodAccessor205.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:216)
Earlier this was an issue in Mule runtime 3.7.3 which they have corrected in later version of 3.x.x runtimes. Not sure if they have considered this correction in mule 4.x.x.
Related
When I run CalendarQuickstart.java as a Java Application I get an IllegalArgumentException Stack Trace. It appears to be a Security Issue. Something to do with web or installed?
I'm not sure how to fix the problem.
Here is the Stack Trace:
Exception in thread "main" java.lang.IllegalArgumentException at com.google.common.base.Preconditions.checkArgument(Preconditions.java:121) at com.google.api.client.util.Preconditions.checkArgument(Preconditions.java:37) at com.google.api.client.googleapis.auth.oauth2.GoogleClientSecrets.getDetails(GoogleClientSecrets.java:82) at com.google.api.client.googleapis.auth.oauth2.GoogleAuthorizationCodeFlow$Builder.(GoogleAuthorizationCodeFlow.java:197) at googlecalapi.CalendarQuickstart.getCredentials(CalendarQuickstart.java:71) at googlecalapi.CalendarQuickstart.main(CalendarQuickstart.java:83)
I am looking for an approach to debugging this scenario. I have verified in Fiddler that there is no HTTP response at all. To be clear, as I understand it a controller method should not simply hang, there is no exception. I have verified the lack of response in Fiddler. The method returns a valid object, verified by stepping through the code to the final return statement.
This is different from the original question in that the controller method is hit, and was not before. The reason for this is explained in the original question. ASP.NET Web Api. Controller not hit. No response at all. Approaches to diagnose?
UPDATE
I am now seeing this behaviour, even though the request completes the handler and returns 200
ExtensionlessUrlHandler and "Recursion too deep; the stack overflowed"
1506. -GENERAL_REQUEST_END
BytesSent
6069
BytesReceived
436
HttpStatus
200
HttpSubStatus
0
From near the end
ErrorDescription
Internal Server Error
0 ms
Warning
1170. -MODULE_SET_RESPONSE_ERROR_STATUS
ModuleName
ManagedPipelineHandler
Notification
EXECUTE_REQUEST_HANDLER
HttpStatus
500
HttpReason
Internal Server Error
HttpSubStatus
0
ErrorCode
Recursion too deep; the stack overflowed.
(0x800703e9)
This turned out to be a crashed instance of RabbitMQ in combination with OWin middleware that was trying to use that instance (to log exceptions such as being unable to connect to the MQ instance; or rather attempting to log them by sending them to.. the MQ instance) and was thus swallowing exceptions in a recursive fashion. The stack overflow was caused by re-entering these middleware instances endlessly. The logging middleware was throwing exceptions because it could not log and the exception handling middleware was handling those exceptions by sending them to the logging middleware. Interesting stuff.
In addition to re-booting to cure the crashed and inaccessible RabbitMQ (restarting the service was not enough) the problem was still not resolved (different symptoms as described above) unless the nuget package MassTransit.RabbitMQ 3.3.2 (old version) and the dependencies (including RabbitMQ.Client) which this exact version brings with it, were installed, rather than the latest versions.
I hope this will help someone.
We are using apache cxf 2.6.1 to call Restful API. Both Server and the client use cxf libraries. On the client side, we use cxf generated client. Everything works in dev environment but we recently saw this error in production environment. I googled and found nothing for this specific namespace. I am wondering if anybody else got this. I already verified that the class path has following 5 cxf libraries (both in dev and prod). Any help would be greatly appreciated.
List of cxf jars we have in class path:
/opt/mhs/lib/cxf-api-2.6.1.jar
/opt/mhs/lib/cxf-rt-bindings-xml-2.6.1.jar
/opt/mhs/lib/cxf-rt-core-2.6.1.jar
/opt/mhs/lib/cxf-rt-frontend-jaxrs-2.6.1.jar
/opt/mhs/lib/cxf-rt-transports-http-2.6.1.jar
Here is the error message. Its very generic and I am having hard time finding the root cause.
apache.cxf.phase.PhaseInterceptorChain - Interceptor for {http://rest.perfmon.kiva.com/}IPerfMonRestEntityAssociationService has thrown exception, unwinding now
org.apache.cxf.interceptor.Fault: No conduit initiator was found for the namespace http://cxf.apache.org/transports/http.
at org.apache.cxf.endpoint.AbstractConduitSelector.getSelectedConduit(AbstractConduitSelector.java:127)
at org.apache.cxf.endpoint.UpfrontConduitSelector.selectConduit(UpfrontConduitSelector.java:77)
at org.apache.cxf.message.ExchangeImpl.getConduit(ExchangeImpl.java:130)
at org.apache.cxf.interceptor.MessageSenderInterceptor.getConduit(MessageSenderInterceptor.java:71)
at org.apache.cxf.interceptor.MessageSenderInterceptor.handleMessage(MessageSenderInterceptor.java:46)
at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:262)
at org.apache.cxf.jaxrs.client.ClientProxyImpl.doChainedInvocation(ClientProxyImpl.java:526)
at org.apache.cxf.jaxrs.client.ClientProxyImpl.invoke(ClientProxyImpl.java:205)
at com.sun.proxy.$Proxy60.addEntityAssociation(Unknown Source)
I want my web application to fail on startup if any of my servlet fails to initialize.
I would expect if I throw an exception form the Servlet.init() method it would cause the entire app to fail, but apparently it doesn't. The web container (Tomcat 7 in my case) just fails to load that particular servlet, but the application is reported to be successfully deployed anyway.
It behaves as expected if an exception thrown from e.g. ServletContextListener.contextInitialized() method, but why the exception thrown from the Servlet.init() doesn't have this effect?
Is there any way to stop the application in case of a Servlet initialization error?
Try this following steps:
Keep your servlet in load-on-startup list such that your servlet
is initialized on application start.
In Servlet.init() method, set an attribute in ServletContext saying the servlet is initialzied. ex, myServletInited
Implement ServletContextListener.contextInitialzied() method. Check if myServletInited is present. If is does not present throw an exception which fails the application.
We have an ASP.Net website that connects to an external web service, and it abruptly stopped working a few days ago.
The basic code is this:
Try
request = New ExternalWebService.ProcessRequestService
' Error occurs here:
response = request.processCommand(parameters)
' Do some other stuff with response here
Catch webEx As System.Net.WebException
' This triggers and generic error gets displayed
Catch ex As Exception
Finally
End Try
The System.Net.WebException gets triggered with a SendFailure exception, and returns this stack trace:
The underlying connection was closed: An unexpected error occurred on a send. -
at System.Web.Services.Protocols.WebClientProtocol.GetWebResponse(WebRequest request)
at System.Web.Services.Protocols.HttpWebClientProtocol.GetWebResponse(WebRequest request)
at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
at ExternalWebService.ProcessRequestService.processCommand(parameters)
When I try to debug this problem on my development machine, it works fine and no exception occurs.
There has been no recent updates to our webserver, and I can connect to the wsdl service from the web server with FireFox (I cannot connect with IE however, but that might be related to another issues we have with IE on the web server)
I've tried all the resolutions posted for Error Message #2 here, including setting KeepAlive = False and ServicePointManager.SecurityProtocol = SecurityProtocolType.Ssl3, however none have worked to fix the issue.
Our webserver is running Windows Server 2003 and IIS6.0
What can cause this error to occur on one machine, but not another?
Edit: Here's the Trace information:
System.Web.Services.Asmx Information: 0 : Calling XmlSerializer [Create XmlSerializer]
Method: System.Xml.Serialization.XmlSerializer#1669721556::FromMappings(System.Xml.Serialization.XmlMapping[]#4002219=[4], ExternalWebService.ProcessRequestService#93032588=ExternalWebService.ProcessRequestService)
Caller: System.Web.Services.Protocols.SoapClientType#42808772::.ctor(ExternalWebService.ProcessRequestService#93032588=ExternalWebService.ProcessRequestService)
ProcessId=2796
LogicalOperationStack=
ThreadId=14
DateTime=2012-12-21T17:02:39.2826250Z
Timestamp=61755143048799345
System.Web.Services.Asmx Information: 0 : Return from XmlSerializer [Create XmlSerializer]
Caller: System.Web.Services.Protocols.SoapClientType#42808772::.ctor(ExternalWebService.ProcessRequestService#93032588=ExternalWebService.ProcessRequestService)
ProcessId=2796
LogicalOperationStack=
ThreadId=14
DateTime=2012-12-21T17:02:39.6576250Z
Timestamp=61755144323396760
System.Web.Services.Asmx Information: 0 : Calling XmlSerializer [Write Request]
Method: Microsoft.Xml.Serialization.GeneratedAssembly.ArrayOfObjectSerializer#1231684::Serialize(System.Xml.XmlTextWriter#8023888=.., System.Object[]#19695157=[40], (null), (null))
Caller: ExternalWebService.ProcessRequestService#60553900::Serialize()
ProcessId=2796
LogicalOperationStack=
ThreadId=14
DateTime=2012-12-21T17:02:39.7357500Z
Timestamp=61755144598320131
System.Web.Services.Asmx Information: 0 : Return from XmlSerializer [Write Request]
Caller: ExternalWebService.ProcessRequestService#60553900::Serialize()
ProcessId=2796
LogicalOperationStack=
ThreadId=14
DateTime=2012-12-21T17:02:39.7357500Z
Timestamp=61755144621621529
System.Web.Services.Asmx Information: 0 : Calling WebRequest.GetResponse
Method: System.Net.HttpWebRequest#2800407::GetResponse()
Caller: ExternalWebService.ProcessRequestService#60553900::GetWebResponse()
ProcessId=2796
LogicalOperationStack=
ThreadId=14
DateTime=2012-12-21T17:02:39.7357500Z
Timestamp=61755144622114240
Apparently in my particular case, it was because the web service vendor had increased their security, and our web server didn't have the correct library needed to decipher the connection.
Our only clue was a single System event viewer entry with an ID of 36874 and a source of Schannel a few days back on the same day our site stopped working that said:
An SSL connection request was received from a remote client
application, but none of the cipher suites supported by the client
application are supported by the server. The SSL connection request
has failed.
However if you're debugging this problem yourself, this link was very useful since it gave some simplified descriptions of the error which pointed me in the right direction, that my problem was related to not being able to connect to their web service.
The underlying connection was closed: An unexpected error occurred on a send
This problem occurs when the client computer cannot send an HTTP
request. The client computer cannot send the HTTP request because the
connection has been closed or is unavailable. Lookup the error in
this article http://support.microsoft.com/kb/915599.
and the linked Microsoft support page was also fairly detailed about some different resolutions you could try to fix the most common causes of this problem.
Just an idea, don't know if it's the good one : the same thing happened few months ago on one of our server, because someone (security team ? :/) changed the write permission on the "%SYSTEMROOT%\Temp" folder. Since that action, XMLSerializer couldn't generate the temporary assembly used to serialize/deserialize.
From http://www.hanselman.com/blog/ChangingWhereXmlSerializerOutputsTemporaryAssemblies.aspx
:
When using the XmlSerializer from ASP.NET there are permissions issues
can can be solved by granting the user account read/write permissions
on the %SYSTEMROOT%\Temp folder
So perhaps you can check that the account used by your application pool has write permissions on the "%SYSTEMROOT%\Temp" folder...
Good luck...