Can you figure out and let me know if deadlock exists in the below example or not?
Suppose we have four processes named po,p1,p2,p3.
Po holds an instance of R0 and requesting for resource R1.
P1 holds an instance of resource R1 and requesting resource R2and r3.
P2 and P3 are both requesting for resource R3. And R2 is assigned to both P0 and P3.
I guess its not in deadlock because there is no cycle existing.
Related
Say there are 2 nodes - A and B. Each of them maintains a state, which is a number in their memory. If A sends an increment somehow to B, then B will return 1. A sends an increment again, B will return 2. And so on. Same for B. And both A and B can update its state atomically.
For sending the increment, let's say B starts a TCP server to accept connections, and A is the client that establishes the connection with B. A can send increment to B through that established TCP connection.
Now, the question is: Can B also send increment to A through the same connection, and can A respond back with its own state through that connection?
Moreover, can A and B both send increments and respond to each other concurrently through the same connection? So that if A and B send increment simultaneously to each other, they can respond back with 1.
It’s an easy problem if A and B establish 2 connections - one for A as the client to send increment to B, and the other for A as the server to respond increment from B. And since there are 2 connections, A and B can send “increment” concurrently. But I wonder if it’s possible for A and B to exchange data with only one TCP connection? Does any protocol support this?
Yes, it's possible. A and B can both exchange data through the same connection. However, one of them will act as a server and the other as a client. In fact, even if A tries to connect to B and B to A at the same exact time, TCP is designed so that scenario results in only one connection. This situation is called simultaneous opening. Keep in mind that the classic concept of client and server don't exist per se in the TCP specs (they're just peers), although is intuitive and helpful to regard the peer that performs the active opening as a client and the one that perform the passive opening as a server.
As per data exchange, both can send increment messages to each other through the same connection. Check RFC 675 and RFC 793(The TCP reference) for more detailed in depth info.
I have developed a Kafka version : 0.9.0.1 application.
One very important aspect of this application is that messages need to be consumed in the correct sequence. This is because I am propagating Database rows between two databases. This means I need to ensure all records within each Unit Of Work are sent as well as UOW-Insert arriving before UOW-Update.
If I use asynchronous message production, how can I guarantee the messages are consumed in the correct sequence?
I will employ Kafka Producer send with Callback to be notified whether or not each message was successfuly sent.
I will have acks=all, retries=0, batch.size=16384 and my Kafka topics have a single partition.
My consumer can cope with duplicate messages, e.g. in the case of retries being necessary, however consumers cannot cope with messages out of sequence.
My retry approach will be to Fail Fast, e.g. as soon as a messages fails to send I report the records Log Record Sequence Number (LRSN) or Relative Byte Address (RBA) and stop sending any messages.
Then reset the source database logs to the reported LRSN or RBA and restart my message production.
for example I send messages
Message UOW
M1 uow-0000
M2 uow-0000
M3 uow-0000
M4 uow-0001
M5 uow-0001
M6 uow-0001
M7 uow-0001
M8 uow-0002
When message M5 failed to send successfully, I would stop sending anymore messages. However I have an issue that the consumer will have received messages M1, M2, M3, M4, M6, M7, & M8.
To recover this situation I will reset my source database log reader to the reported LRSN or RBA of M5 and start resending messages from that point.
Now the consumer receives
Message UOW Duplicate
M5 uow-0001 No
M6 uow-0001 Yes
M7 uow-0001 Yes
M8 uow-0002 Yes
With this approach I get the speed of Asynchronous messaging and "hopefully" all messages consumed in the desired sequence.
I have previously used Tornado behind Nginx in production, that too with multiple tornado instances, but presently with Websockets implementation in Tornado, I am unable to understand few things.
NOTE : Just running Websockets with Tornado is not my problem, I can get it to work and have used it in many projects fairly easily,
but all as a single Tornado instance with/without a reverse-proxy or
load balancer (nginx), both working fine.
With that said, my doubts are regarding how to handle multiple tornado websocket instances behind a load balancer. Lets look at a use case:
Use Case : 2 Tornado Websocket Instances, T1 and T2 behind Nginx and there are 3 Clients (Browsers), lets say C1, C2, C3.
______|-------|
C1-----------------| | | T1 |
| | |_______|
C2-----------------|---> Nginx --->|
| | |-------|
C3-----------------| |______| T2 |
|_______|
Step 1 - C1 initiated a websocket connection, at location w.x.y.z:80. Now Nginx load balanced the connection to lets say T1, which triggered the open event in T1's(Tornado's) websocket handler. Now T1 knows of a websocket connection object which is opened, let the object be w1.
Step 2 - Now, C2 initiated a websocket connection, same as above, but now Nginx laod balanced it to T2 which triggered T2's(Tornado's) websocket handler's open event. Now T2 knows of a websocket connection object which is opened, let the object be w2.
Step 3 - Now, similarly C3 initiated a websocket connection, Nginx load balanced it to T1, which now has a new opened websocket connection object, let the object be w3.
Step 4 - Now, C1, our first client sent a message via the browser's ws.send() method, where ws is the browser(client) side's websocket object used to create the websocket connection in Step 1. So when the message reached Nginx, it load balanced it to T2.
Now here is my question.
Does Nginx load balance it to T2 or does it know that it should proxy
the request to T1 itself?
Suppose it sends it to T2, now T2 doesn't have the w1 object with it, since it is with T1, so when T2's websocket handler tries to process the request, which should originally trigger the on_message handler, so now,
what happens at this condition, does T2 raise an Exception or what
happens exactly?
Also, how to manage the websocket connections in these cases when there
are multiple tornado instances running behind a load balancer, how to
solve this?
If suppose we use Redis as a solution, then what would redis actually store? The websocket objects? Or, what exactly should it store to let the instances work properly, if Redis is one of the solutions you are going to propose?
Messages are always attached to a connection; they are not load-balanced separately. So in step 4, nginx sends the message on the C1 connection to T1, not T2.
I understand the end-to-end principle from the classic MIT paper, which states that executing a function between 2 remote nodes does not depend on the states of the in-between nodes.
But what is end-to-end encryption, end-to-end guarantees, end-to-end protocols, etc...? I couldn't find a precise definition of end-to-end. The term seems to be over-used.
In other words, when one describes a system property X as end-to-end, what does it mean? What is the opposite of end-to-end?
I don't think end-to-end is over-used. It merely says that the property holds from one end to the other. An "end" can be a node or a layer in the computing stack.
Consider three nodes: A, B and C. Node A wants to talk with C. B sits between A and C and forwards messages between them. B is, for example, a load balancer or a gateway.
Encryption is end-to-end if B cannot read and tamper messages sent from A to C. A concrete example is the following: A is your laptop, C is a remote machine in your network at home or work. B is a VPN gateway. The encryption here is not end-to-end because only the link between A and B is actually encrypted. An attacker sitting between B and C would be able to read the clear text. That might be fine in practice, but it is not end-to-end.
Another example. Say we don't care about encryption, but about reliable message transmission. You know that the network might corrupt bits of messages. Therefore, TCP and other protocols have a checksum field that is checked whenever messages are received. But the guarantees of these checksums is not necessarily end-to-end.
If A sends a message m to C relying on the TCP's checksum, a node B sitting in the middle could corrupt the message in an undetectable way. Abstracting most details, node B basically (1) receives m, (2) checks m's checksum, (3) finds the route to C and creates a new message with m's payload, (4) calculates a new checksum for m, and (5) sends m (with the new checksum) to C. Now, if node B corrupts the message after (2) but before step (4), the resulting message arriving on C is corrupted but that cannot be detected by looking at m's checksum! Therefore, such checksum is not end-to-end. Node B does not even have to be malicious. Such a corruption can be caused by hardware errors or more probably by bugs in node B. This has happened a couple of times in Amazon S3 service, for example: this case and this case and this case.
The solution is, obviously, to use application-level checksums, which are end-to-end. Here, a checksum of m's payload is appended to the payload before calculating the lower layer checksum.
Traceroute is an application to trace the path from A to B. (A is your location and B is the server you want to trace). In Windows, you can type tracert. The main algorithm is:
send UDP with TTL = 1
Server A1 received, and return ICMP packet to A because TTL is expired.
--> know first machine between. For example A1.
send UDP with TTL = 2
Server A1 received, and send this UDP to server A2.
Server A2 received, and return ICMP packet to A because TTL is expired
--> know second machine between. In this example is A2.
Do it until to B. we can track down: A -> A1 -> A2 -> ... ->B
Does this algorithm work correctly? Because at different time, an intermediate server can send a message to different server. For example, at first time, UDP message is sent to A1, but at a later time, it can send to another server, for example, B1. So, trace route will not work properly.
Did I misunderstand something?
From the man page :
traceroute tracks the route packets take from an IP network on
their
way to a given host
So if you are trying to find one of the possible paths your packet may take, you'll find a friend in traceroute .
Now because routing tables do not change every minute, the packets that you send will most probably take the same path as traced by traceroute.
Another important point that cannot be missed is the record route option in the IP v4 header.
Once you specify that you want to use this option, every router in the path will add it's ip address to the options in the header. You can read more about it here. The catch being that the destination gets to know about the intermediate hops , not the source.
I see that you missed the role of icmp echo request and reply messages in the description of traceroute. In case this was not intentional , take a look.
Update : You can see the record route option in action by doing a ping -R
ping -R Turns on route recording for the Echo Request packets, and
display the route buffer on returned packets (ignored by many
routers).
The algorithm works properly. Indeed, routing may change due to considerations of different servers along the way, such as server load or availability. Let's say you want to send message from A to B. If the route is not changeable, what will happen if some server on the route is down? If the routing couldn't be adjusted dynamically, that would result in inability to deliver the message to the destination in this example. Here is a different example: let's say you have a server that is used for some heavy computation during the day but it's idle during the night. It's possible to allow it to pass traffic only during the night, so any routing using it will need to be changed at day.
To conclude all this we can definitely say that without dynamic routing the internet couldn't have existed in its' present form.
Addition:
Tracert sends message from A to B. It shows hops along the way. These hops constitute a valid route from A to B at the time of the execution. There is no guarantee that connection between 2 adjacent points along the way is valid after the hop has been completed. The only thing guaranteed is that for each hop there was a link between it's 2 endpoints when the message sent by tracert passed there.