What happens when leader crashes before all followers updates the commit index?
For example, node A, B, C forms the cluster:
only A and B alive and A is leader
A replicates an entry (let's say it's entry1) to B and get successful result from B
A commits entry1, and crashes before it send out the heartbeat message to B (which would results in B updates its commit index)
C online now
My questions is:
would C be elected as new leader? If so, then entry1 is lost? Moreover, if A re-joins later, its data would be inconsistent with others?
I know the raft spec said:
Raft uses a simpler approach where it guarantees that all the
committed entries from previous terms are present on each new leader
from the moment of its election, without the need to transfer those
entries to the leader.
But here entry1 may be not considered as committed entry? because B has not get the confirm from old leader (heartbeat from leader). So C get chances to be new leader?
if B becomes new leader, then how it should deal with the entry1?
It's important to note that an entry is considered committed once it's stored on a majority of servers in the cluster (there are technically some caveats to this but for this conversation we should assume this is the case) and not when a node receives a commit messages from the leader. If a commit message were required to consider an entry committed then every commit would require two round-trips - one for replication and one for commitment - and commit indexes would have to be persisted.
Instead, in your scenario, when A crashes and C recovers the Raft election algorithm will ensure that C cannot be elected leader and so C cannot drop the committed entry. Only B can be elected leader since it has the most up-to-date log. If C tries to get elected leader, it will receive only a rejected vote from B since B's log is more up-to-date than C's (it has the committed entry). Thus, what you'll see in practice is B will eventually be elected and will commit all entries from its prior term, at which time that entry will still be committed. Even if B were then to crash and A were to recover, A would still have a more up-to-date log than C and so it would again be elected leader.
When (not if) B becomes the leader, it will first ensure entries from the prior term are stored on a majority of servers before committing any entry from its current term. Typically this is done by committing a no-op entry at the beginning of the new leader's term. Essentially, the new leader commits a no-op entry, and once that entry is stored on a majority of servers it increments its commit index and sends the new commit index to all followers. So, that entry will not be lost. The new leader will ensure it is committed.
The caveats to considering an entry stored on a majority of the cluster to be committed are described in both the Raft paper and disseration.
Related
I'm fairly new to DLT technologies and dove a bit in Corda. I'm not sure to understand everything well yet so here is my question that i tried to answer myself (using NotaryChangeFlow). Please correct me.
Problem setup :
I'm node A. I share facts with node B and node C. They don't share any fact.
For node B, we have notary B tracking our transactions.
For node C, we have notary C tracking our transactions.
Problem :
If B sends me let's say 5€.
How can I use those 5€ to send it to C
My solution:
A and B creates a transaction updating both states + updating notary B state.
A changes notary B to notary C.
A consumes his state shared with B and sends money to the facts he shared with C using notary C as validator.
A sends money to C.
That way : B knows A used 5€ and C knows 5€ has been added to A balance and is then transfered to him.
Is this the right way?
Thanks!
Yes, you are correct. Let me just reiterate to make it a little more clear. States in Corda are associated with a particular notary and can only be spent with that specific notary. If there is a need to spend a state at a different notary, a Notary Change Transaction is performed first. In a Notary Change Transaction, the state in question is spent at the current notary and reissued with the new notary.
All of this is done to ensure Notaries are able to do their job of preventing double-spending attempts in the network. Any notary other than the one where a state is being transacted would not have any information regarding the previous transaction of a state and would not be able to able to prevent double-spending attempts without that information.
So now in your case, B wants to send the $5 to A using notary N1 and now A wants to send the same $5 to C using notary N2. This will be a two-step process:
A does a Notary Change Transaction, where he spends the $5 state at N1 and reissues the state at N2. All this is taken care of by the library flow NotaryChangeFlow.
Now once the notary has been changed A can now construct a transaction to spend the state at notary N2.
On the statement B know $5 has been used by A, you need to either add A as a participant/ observer for him to be notified of any updates. Generally, this is not a requirement for cash since once I have to send the money to another party, I am not supposed to know what he does with it. But if that's a requirement for some use case the cordapp needs to be designed accordingly.
While upgrading from Corda 3 to Corda 4, I have an issue commiting a State to our node's ledger with only one Party. A single Party is able to create a state, notarize it, but CANNOT commit to the Corda 4 ledger without asking for an external third party.
The error Corda 4 produces (which Corda 3 did not produce) is the following:
(1) java.lang.IllegalArgumentException: A flow session for each external participant to the transaction must be provided. If you wish to continue using this insecure API then specify a target platform version of less than 4 for your CorDapp.
More specific context: Using FinalityFlow without a session yields a 'session required for external parties' error and does not complete. Adding only a session (e.g. session = initiateFlow(PartyA) ) results in an error that 'local nodes should not be included.'
Is there a workaround regarding this solution? It's important (for our use case) that a single Party can create a State and modify the State information without the involvement of other parties. Other use cases (where multiple parties are included) stem from this use case. Any guidance is greatly appreciated.
I think the error message is pretty spot on here. Just change the way you call FinalityFlow during your issuance such that it doesn’t contain a flow session to itself i.e.
return subFlow(new FinalityFlow(signedTransaction));
Although you may get a deprecation warning, in which case, do the following
return subFlow(FinalityFlow(stx, emptyList()))
I've been reading a lot lately and I am now very confused how transactions and locking are working and how are related to each other.
When working with SQLite, imagine the following flow:
begin SQLite transaction
run a select statement on a SQLite connection to return values
if condition is met on the returned values, go to step #4, otherwise go to step #5
do the update
commit SQLite transaction
If two threads run same code, is there a possibility that in one of the threads could get what is called a "dirty read" meaning, between the step #3 and step #4, the other thread could run the update (step #4)?
Yes, it's called isolation level: https://www.sqlite.org/isolation.html
If A creates some state with participants of A and B, and signs it, but doesn't require B's signature, I see that the state still gets replicated to B's ledger.
Does the state's contract's verify function still get run on B's side? Or do I have to code that explicitly?
Yes.
A will send the transaction by invoking SendTransactionFlow. B will respond by invoking ReceiveTransactionFlow.
As part of ReceiveTransactionFlow, B will verify the transaction, and optionally the signatures (based on the checkSufficientSignatures parameter).
I'm a SolrCloud newbie, my setup is 3 shards, 3 replicas, external Zookeeper
Today I found shard3 down, replica3 had taken over as leader, so indexing was occurring to replica3 not shard3. I stopped Tomcat/SOLR in reverse order (R3,R2,R1,S3,S2,S1) and restarted in forward order (S1,S2,S3,R1,R2,R3). I did not delete any tlog or replication.properties files. The cloud graph shows all hosts with their correct assignments. As I understand it these assignments are set in Zookeeper on the first startup.
My question is how does the data that was indexed to replica3 get back to the revived shard3?
And surprisingly shard3 = 87G while replica3 = 80G.
Confused!
Dan,
The size of replicas are not important, only the number of documents that collection has.
The way Solr works, you can have deleted documents in your collection that only are deleted in merge operations, this extra 7G can be deleted documents.
1) As far as I know when the shard3 is up, live and running it is zookeeper which does the data sync job between shard and replica3.
2) Regarding your second question, may be the replica3 is in optimization state and hence you are seeing less data size and shard3 is yet to be optimized by SOLR. (This is just a wild guess)