I'm trying to use TransactionScope for unit tests and I keep getting errors on our build server. After following several helpful SO answers and blogs, I installed DTCPing and ran it on both server1 & server2. When I run it in the reverse order it seems to connect in one direction and fail in the other:
++++++++++++Validating Remote Computer Name++++++++++++
Please refer to following log file for details:
D:\KPAHQDEV043372.log
Invoking RPC method on teamcity
RPC test is successful
++++++++++++RPC test completed+++++++++++++++
++++++++++++Start DTC Binding Test +++++++++++++
Trying Bind to teamcity
Binding call to teamcity Failed
Session Down
But when I run it in the intended direction it just fails the RPC
++++++++++++Validating Remote Computer Name++++++++++++
Please refer to following log file for details:
C:\TEAMCITY3024.log
Invoking RPC method on kpahqdev04
Problem:fail to invoke remote RPC method
Error(0x6BA) at dtcping.cpp #303
-->RPC pinging exception
-->1722(The RPC server is unavailable.)
RPC test failed
I found some helpful info here but now I'm just stuck. Any ideas?
You need to add Distributed Transaction Coordinator service as an exception in firewall.
Also make sure RPC (port 135) is enabled and added as en exception in firewall.
Also you can check if firewall is the culprit by turning off firewall and re-running the DTCPing.
Related
I have a Python gRPC server running on AWS Fargate (configured very similar to this AWS guide here), and another AWS Fargate task (call it the "client") that attempts to make a connection to my gRPC server (also using Python gRPC). However, the client is unable to make a call to my server, with the following error:
<_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "failed to connect to all addresses"
debug_error_string = "{"created":"#1619057124.216955000","description":"Failed to pick subchannel",
"file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":5397,
"referenced_errors":[{"created":"#1619057124.216950000","description":"failed to connect to all addresses",
"file":"src/core/ext/filters/client_channel/lb_policy/pick_first/pick_first.cc",
"file_line":398,"grpc_status":14}]}"
Based on my reading online, it seems like there are myriad situations in which this error is thrown, and I'm having trouble figuring out which one pertains to my case. Here is some additional information:
When running client and server locally, I am able to successfully connect by having the client connect to localhost:[PORT]
I have configured an application load balancer target group following the guide from AWS here that makes health check requests to the / route of my gRPC server, using the gRPC protocol, and expect gRPC response code 12 (UNIMPLEMENTED); these health check requests are coming back as expected, which I believe implies the load balancer is able to successfully communicate with the server (although I could be misunderstanding)
I configured a service discovery system (following this guide here) that should allow me to reach my gRPC server within my VPC via the name service-name.dev.co.local. I can confirm that the corresponding DNS record exists in Route 53, and when I SSH into my VPC, I am indeed able to ping service-name.dev.co.local successfully.
Anyone have any ideas? Would appreciate any and all advice, and I'm happy to answer any further questions.
Thank you for your help!
on your grpc server use 0.0.0.0:[port] and expose this port with TCP on your container.
we operate CENM(1.2 and use helm template to run on k8s cluster) to construct our own private network and keep on running CENM network map server for a few week, then launching new node start failing.
with further investigation, its appeared that request timeout for http://nmap:10000/network-map causes problem.
in nmap server’s log, we found following output when access to above url with curl.
[NMServer] - Error while handling socket client message com.r3.enm.servicesapi.networkmap.handlers.LatestUnsignedNetworkParametersRetrievalMessage#760c53ea: HikariPool-1 - Connection is not available, request timed out after 30000ms.
netstat shows there is at least 3 establish connection to the database from the container which network map server runs, also I can connect database directly with using CLI.
so I don’t think it is neither database saturated nor network configuration problem.
anyone have an idea why this happens? I think restart probably solve the problem, but want to know the root cause...
regards,
Please test the following options.
Since it is the HikariCP (connection pool) component that is throwing the error it would be worth seeing if increasing the pool size in the network map configuration may help - see below)
Corda uses Hikari Pool for creating the connection pool. To configure the connection pool any custom properties can be set in the dataSourceProperties section.
dataSourceProperties = {
dataSourceClassName = "org.postgresql.ds.PGSimpleDataSource"
...
maximumPoolSize = 10
connectionTimeout = 50000
}
Has a healthcheck been conducted to verify there are sufficient resources on that postgres database i.e basic diagnostic checks ?
Another option to get more information logged from the network map service is to run with TRACE logging also:
From https://docs.corda.net/docs/cenm/1.2/troubleshooting-common-issues.html
Enabling debug/trace logging
Each service can be configured to run with a deeper log level via command line flags passed at startup:
java -DdefaultLogLevel=TRACE -DconsoleLogLevel=TRACE -jar <enm-service-jar>.jar --config-fi
I am deploying AWS Corda Enterprise Template. The Quick start deployed the stack as per the defined CloudFormation template. I can see 2 AWS instances, up and running as Corda nodes, in Hot-Cold setup with a load balancer.
However the Log for Corda node has following ERROR related to AMQP communication.
[ERROR] 2018-10-18T05:47:55,743Z [Thread-3
(ActiveMQ-scheduled-threads)] core.server.lambda$channelActive$0 -
AMQ224088: Timeout (10 seconds) while handshaking has occurred. {}
What can be possible reason for this error? This error keeps on occurring after a certain time interval. So it looks like some connectivity issue to me.
Note: The load balancer shows the status of this AWS Corda instances as healty (In Service). So I believe the Corda node has booted up successfully.
The ERROR message isn't necessarily tied to AMQP. Perhaps you were confused by the "AMQ" in the error ID (AMQ224088)?
In any event, this error indicates that something on the network is connecting to the ActiveMQ Artemis broker, but it's not completing any protocol handshake. This is commonly seen with, for example, load balancers that do a health check by creating a socket connection without sending any real data just to see if the port is open on the target machine.
So I have been running into this issue since we set up our BizTalk Server on a new network. We have the same MSMQ settings between the two servers.
The data stays in our AX MSMQ folders and has the correct permissions.
The system does not ever throw an error until I stop/restart the Receive Host Instances.
(we get one of these errors per message in any of our MSMQ ports)
Full error:
A message received by adapter "MSMQ" on receive location
"recv_loc_file_ax_2012_customer_message" with URI
"FORMATNAME:DIRECT=OS:AXSERVER\AXOUTPPDCUSTOMER" is suspended. Error
details: The Messaging Engine is shutting down. MessageId:
{65E24FE1-317E-4636-AFC7-B43FACBDBEDF} InstanceID:
{6618EEB3-9B72-4123-BD8C-422661A59BDD}
Then the messages finally appear under suspended instances after this error occurs. I am able to resume them and they all process as expected.
I have looked almost every, Anyone have suggestions for what is causing these messages to not be read into my MSMQ receive ports properly?
EDIT: This BizTalk server is connecting to a remote AX server's MSMQ, but I am doubting this changes anything I have not already looked into.
Thank you very much.
The error was on the guys who installed MSMQ on the remote server; the active directory was not set up thus not properly autenticating my BizTalk Server account.
Answer for BizTalk: The way to find these hidden errors was by changing the MSMQ BizTalk receive port to a WFC-NetMSMQ.
Other: we are a little baffled that BizTalk was able to take the messages out of MSMQ despite the AD not being set up and "force messages through" but this is a minor detail to note.
BizTalk reference
I reinstalled the ESB toolkit using the same process (painful) that worked twice before.
But now when I go try to start the BizTalk application "Microsoft.Practices.ESB" in the BizTalk Administration Console, the application fails to start showing the following error:
Could not enlist Send Port 'ALL.Exceptions'.
Exception from HRESULT: 0xC00CE557 (Microsoft.BizTalk.SnapIn.Framework)
I found this error message in the event log:
Unable to communicate with MessageBox BizTalkMsgBoxDb on SQL Instance [MY SERVER].
Error Code: 0x8004d00e. Possible reasons include:
1) The MessageBox is unavailable.
2) The network link from this machine to the MessageBox is down.
3) The DTC Configuration on either this local machine or the machine hosting this MessageBox is incorrect.
SQL Server seems to be running ok. Really don't know where to start looking...
I had the same problem. Check out the port's filters, if the list is empty try to manually configure the filters on the port as following:
ErrorReport.FailureCode - Exists
Microsoft.Practices.ESB.ExceptionHandling.Schemas.Property.FaultCode - exists
those are the filters that usually exists on this port and somehow disappeared.
Worked for me, hope it will work for u as well
best regards,
David