I am trying to run Example Cordapp in two AWS instances. With Notary and PartyA in 1st Instance and PartyB and PartyC in 2nd Instance.
I followed the steps here,
Corda nodes: how to connect two independent pc as two nodes?
In the conf file of,
Notary and PartyA - I have edited the P2P address to reflect the PrivateIP of Instance 1
PartyB and PartyC - I have edited the P2P address to reflect the PrivateIP of Instance 2
With the above conf files, I ran the Network Bootstrapper jar in Instance 1 and copied the folders PartyB and PartyC to Instance 2 and started the Notary and Parties 1 by 1 respectively in the corresponding Instances.
All nodes started succesfully and when I try to execute a IOU flow from PartA(in Instance 1) to PartyC(in Instance2), it is pausing at Collecting counterparties signature step without proceeding further. Below is what I see in PartyA's Console,
Fri Nov 30 08:39:10 UTC 2018>>> flow start ExampleFlow$Initiator iouValue: 50, otherParty: "O=PartyC,L=Paris,C=FR"
Verifying contract constraints.
Signing transaction with our private key.
Gathering the counterparty's signature.
Collecting signatures from counterparties. (hanging here and not proceeding further)
When I tried to look at the log information in NodeA, it displays as below.,
[INFO ] 2018-11-30T08:39:10,077Z [main] messaging.RPCServer.start - Starting RPC server with configuration RPCServerConfiguration(rpcThreadPoolSize=4, reapInterval=PT1S, deduplicationCacheExpiry=PT24H) {}
[INFO ] 2018-11-30T08:39:10,115Z [Thread-0 (ActiveMQ-client-global-threads)] bridging.BridgeControlListener.processControlMessage - Received bridge control message Create(nodeIdentity=DLHBP432vnpLNpCNwGQJjx3hd6RDz4LiYxmZJo757W8Hbw, bridgeInfo=BridgeEntry(queueName=internal.peers.DL9tRWQ867M3tni7KRqkXEJKPrkyW5KVj6fyRyDBHGaGA6, targets=[[2001:0:9d38:953c:3c:ce3:cbd9:3c59]:10013], legalNames=[O=PartyC, L=Paris, C=FR])) {}
[INFO ] 2018-11-30T08:39:11,072Z [nioEventLoopGroup-2-2] netty.AMQPClient.nextTarget - Retry connect to [2001:0:9d38:953c:3c:ce3:cbd9:3c59]:10013 {}
[INFO ] 2018-11-30T08:39:12,171Z [nioEventLoopGroup-2-3] netty.AMQPClient.operationComplete - Failed to connect to [2001:0:9d38:953c:3c:ce3:cbd9:3c59]:10013 {}
[INFO ] 2018-11-30T08:39:14,172Z [nioEventLoopGroup-2-4] netty.AMQPClient.nextTarget - Retry connect to [2001:0:9d38:953c:3c:ce3:cbd9:3c59]:10013 {}
[INFO ] 2018-11-30T08:39:15,175Z [nioEventLoopGroup-2-1] netty.AMQPClient.operationComplete - Failed to connect to [2001:0:9d38:953c:3c:ce3:cbd9:3c59]:10013 {}
I could able to ping between instances with the private IP's without any issues. Can someone help me where am I missing things.
Thanks in advance.
This issue was caused by a firewall on the node's machine that was preventing the node's messages from reaching the counterparty nodes.
You need to open:
Outbound ports for your node's P2P address
Inbound ports for the other nodes' P2P addresses
On a large network, this can mean opening many inbound ports, which can be an issue for some companies' security policies. This issue is addressed by the Corda Firewall.
Related
Maybe Im just approaching this wrong.
Single Instance mode (non-HA)
AWS-RDS Postgres Database
Deploying via ECS
Currently have Artifactory-Pro building a docker container and deploying to ECS via CI/CD. The initial deploy goes fine. Everything stands up, database migrations occur, and the instance runs.
However, when doing an update to the task, a new task spins up. It then adds entries to the access_topology with the new container-ip and unique node-id, but they stay unhealthy. The logs just then bomb out with failure messages (below - due to existing heartbeat of other node).
If I first stop the running task, and start a new task, it spins up properly (Probably due to heartbeat loss).
In typical ECS world, the new task is spun up till its deemed healthy, and then the older task is killed off.
Either scenario creates orphaned NODE records that stay healthy -- trying to also figure out how to garbage collect on those and purge.
Any thoughts on this?
Errors are below – it appears that it wont properly join because of an active heartbeat, and not being HA. However, I want this node to stand up so I can topple the other. Thanks –
Cluster join: Successfully joined jfmd#01es5dmfhar6gcy5abyj4rwpkc with node id ip-10-10-3-248.us-XXXX-1.compute.internal
Application could not be initialized: Current Artifactory node last heartbeat is: 1607609142483. Stopping Artifactory since the local server is running as PRO/OSS but found other servers in registry
Error occurred when refreshing domain cache all domain endpoint failed : Fetch domains from http://localhost:8046/distribution/api/v1/events/domains failed (returned 404), Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404), [domain_client]"
Retry 20 Elapsed 16.84 secs failed: Couldn't access another access peer. [localhost:8046]. Status code: UNAVAILABLE. HTTP status code 503
Status code: UNAVAILABLE. HTTP status code 503
1607609184634,invalid content-type: text/plain; charset=utf-8
1607609184634,"headers: Metadata(:status=503,content-type=text/plain; charset=utf-8,content-length=19,date=Thu, 10 Dec 2020 14:06:24 GMT)"
1607609184634,DATA-----------------------------
1607609184634,Service Unavailable. Trying again
This is not possible without an HA configuration. Since this is not an HA configuration, the application will not start up if there is another application still "alive". In this case, "alive" is defined as having written the heartbeat within X amount of seconds (I believe this is 10 by default).
I use SpringBoot as a consumer of Kafka, and the program accepts messages from Kafka, but the log contains the following information
[kafka-producer-network-thread | producer-2] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-2] Connection to node -1 could not be established. Broker may not be available.
Please help me to solve this problem
I have a node which is running on my machine along with a spring boot client. Client connects to node's rpc port, every thing works well but when i close my client the node crashes and i have to restart it. Why its happening is it a bug or i am doing some thing wrong. I have also deployed them on cloud and same problem happens.
Corda Open Source 4.0 (503a2ff) May 16 11:37:43 broker java[16853]: Logs can be found in : /opt/corda/logs May 16 11:37:58 broker java[16853]: Advertised P2P messaging addresses : 35.228.97.4:10011 May 16 11:37:58 broker java[16853]: RPC connection address : 10.166.0.2:10012 May 16 11:37:58 broker java[16853]: RPC admin connection address : 10.166.0.2:10050 May 16 11:38:01 broker java[16853]: Loaded 2 CorDapp(s) : Contract CorDapp: Template CorDapp version 1 by vendor Corda Ope May 16 11:38:01 broker java[16853]: Node for "Broker" started up and registered in 19.86 sec May 16 11:38:01 broker java[16853]: SSH server listening on port : 2222 May 16 12:10:03 broker java[16853]: Shutting down ...
It depends how you create your CordaRPCOps class.
If it is a bean, then it will call CordaRPCOps.shutdown when the client is shutdown. This is due to Spring triggering any method named shutdown on any beans by default. Therefore by not creating it as a bean, for example having a wrapper class around CordaRPCOps that is created as a bean instead will resolve this issue.
Or you can tell spring to not trigger the shutdown method by defining your bean like:
#Bean(destroyMethod = "")
public CordaRPCOps proxy() {}
I encountered one error when I created a new transaction with Corda V2.0, from the sender's server log, it shows below error
[WARN ] 2018-05-15T09:34:47,366Z [Messaging] messaging.NodeMessagingClient.sendWithRetry - Reached the maximum number of retries (3) for message ClientMessageImpl[messageID=0, durable=true, address=internal.peers.4qWch9xxxV,userID=null,properties=TypedProperties[platform-version=2,corda-vendor=Corda Open Source,release-version=2.0.0,platform-topic=platform.session,session-id=0,_AMQ_DUPL_ID=dd1d6efb-dd71-4d34-9686-fecdb8d72671]] redelivery to internal.peers.4qWchxxxEmV
In recipient's log, we can see the transaction's dependency(attachment) is retrieved and stored successfully but there is no information about transaction's process(I think this log information should be provided from Corda by default).
We are using raft notaries cluster and I didn't find any transaction information in 3 notaries logs.
So we are not sure where is the root cause for that messaging error. Anyone can help?
In Corda 3, there is a bug which means that after rebooting a Raft notary cluster, the Raft cluster ends up partitioned with multiple leaders and it no longer services client requests.
You can track the solution to this issue on Corda's JIRA board: https://r3-cev.atlassian.net/browse/CORDA-1527.
I am using Corda Version 1. Transaction is not getting completed when I am running nodes on docker environment. It got stuck after this logs:
Flow logs >> Generating transaction based on new Trade Details.
Flow logs >> Verifying contract constraints.
Flow logs >> Signing transaction with our private key.
Flow logs >> Gathering the counterparty's signature.
Flow logs >> Structural step change in child of Gathering the counterparty's signature.
Flow logs >> Collecting signatures from counter-parties.
Flow logs >> Verifying collected signatures.
Flow logs >> Done
Flow logs >> Obtaining notary signature and recording transaction.
Flow logs >> Structural step change in child of Obtaining notary signature and recording transaction.
Flow logs >> Requesting signature by notary service
Flow logs >> Structural step change in child of Requesting signature by notary service
Flow logs >> Requesting signature by Notary service
I can see this message in the controller logs:
[INFO ] 2017-10-27T07:27:57,024Z [Messaging] messaging.NodeMessagingClient.createQueueIfAbsent - Create fresh queue internal.peers.8Kqd4oWdx4KQGHGNkbLbDP3bxDkm43Gy7NpJqGdiL5G2PwRdVu9dngMkJkp bound on same address
[WARN ] 2017-10-27T07:29:57,111Z [Messaging] messaging.NodeMessagingClient.sendWithRetry - Reached the maximum number of retries (3) for message ClientMessageImpl[messageID=0, durable=true, address=internal.peers.8Kqd4oWdx4KQGHGNkbLbDP3bxDkm43Gy7NpJqGdiL5G2PwRdVu9dngMkJkp,userID=null,properties=TypedProperties[platform-version=1,corda-vendor=Corda Open Source,release-version=1.0.0,platform-topic=platform.session,session-id=0,_AMQ_DUPL_ID=0efb502e-15f6-4b0a-bdf9-8296571f228f]] redelivery to internal.peers.8Kqd4oWdx4KQGHGNkbLbDP3bxDkm43Gy7NpJqGdiL5G2PwRdVu9dngMkJkp
I have solved this problem by using actual host machine IP address in p2pAddress while creating node container.
It was mapped as localhost in p2pAddress and i have already exposed container ip with hostmachine ip. therefore ideally this should not be the issue, however it's creating issue while signing notary service if we are using localhost. many thanks for support.