When we type https://stackoverflow.com/ all the data which is sent to this server would be using HTTPS protocol in transport level
So is it possible for me to know which error detection, flow control or protocols, in general, are used by lower ( physical, data link, network ) layers by my machine and intermediate nodes which lie in path from my machine to the destination server (Assuming there are few intermediate nodes) .
To figure out what kind of lower layers are used, I always use WireShark.
Start the Wireshark, type "tls" as the filter(because this is no "https" filter in WireShark), you will get a lot of network traffic, just looks like this:
Check the details of the "Application Data" network packet,
You will see the network stack(From top to down):
Transport Layer Security (TLS): It's used to securely transmit data above TCP connection
Transmission Control Protocol (TCP): It's used to reliably transmit data between two network applications running on two different hosts connected by the internet. [Transport Layer]
Internet Protocol (IP): It's used to transmit data between hosts connected by the internet. [Network Layer]
Ethernet II: it's used to transmit data on the underlying Ethernet physical layer transport. [Data Link Layer]
You can check the details by selecting any one of them.
For "error detection", "flow control", you may filter the network traffic by the TCP filter provided by Wireshark,
such as:
"tcp.analysis.retransmission",
"tcp.analysis.window_full",
"tcp.analysis.window_update",
and so on.
If you have a good understanding of TCP, you can read them easily.:)
You usually cannot get such low level information from other devices in the path of the data. You can get some layer 2 information inside the local network but from devices after the router (i.e. outside the LAN) such data are no longer accessible.
Sometimes one can make some assumptions based on detected MTU in the path (PMTU), latency issues, buffering behavior or based on strange issues caused by known behavior anomalies or peculiarities of some devices, but there is no generic way to access such lower level information.
Related
I understand that it's different than a hub in that instead of packets being broadcasted to all devices connected to the device, it knows exactly who requested the packet by looking at the MAC layer.
However, is it still possible to use a packet sniffer like Wireshark to intercept packets meant for other users of the switch? Or is this only a problem with ethernet hubs that doesn't affect switches due to the nature of how a switch works?
On a slightly off topic side note, what exactly is classified as a LAN? For example, imagine two separate ethernet switches are hooked up to a router. Would each switch be considered a separate LAN? What is the significance of having multiple LAN's within the same network?
it knows exactly who requested the packet by looking at the MAC layer.
More exactly, the switch uses the MAC destination address to forward a frame to the port associated with that address. Addresses are automatically learned by looking at the MAC source address on received frames.
A switch is stateless, ie. is has no memory who requested which data. A layer-2 switch also has no understanding of IP packets, addresses or protocols. All a basic switch does is learn source addresses and forward by destination address.
is it still possible to use a packet sniffer like Wireshark to intercept packets meant for other users of the switch?
Yes. You'll need a managed switch supporting port mirroring or SPANning. This doesn't intercept frames, it just copies them to the mirror port. If you need to actually intercept frames you have to put your interceptor in between the nodes (physically or logically).
With a repeater hub, every bit is repeated to every node in the collision domain, making monitoring effortless.
what exactly is classified as a LAN?
This depends on who you ask and on the context. A LAN can be a layer-1 segment/bus aka collision domain (obsolete), a layer-2 segment (broadcast domain), a layer-3 subnet (mostly identical with an L2 segment) or a complete local network installation (when contrasted with SAN or WAN).
Adding to #Zac67:
Regarding this question:
is it still possible to use a packet sniffer like Wireshark to
intercept packets meant for other users of the switch?
There are also active ways in which you can trick the Switch into sending you data that is meant for other machines. By exploiting the Switch's mechanism, one can send a frame with a spoofed source MAC, and then the Switch will transfer frames destined to this MAC - to the sender's port (until someone else sends a frame with that MAC address).
This video discusses this in detail:
https://www.youtube.com/watch?v=YVcBShtWFmo&list=PL9lx0DXCC4BMS7dB7vsrKI5wzFyVIk2Kg&index=18
In general, I recommend the following video that explains this in detail and in a visual way:
https://www.youtube.com/watch?v=Youk8eUjkgQ&list=PL9lx0DXCC4BMS7dB7vsrKI5wzFyVIk2Kg&index=17
what exactly is classified as a LAN?
So indeed this is one of the least-well-defined terms in Computer Networks. With regards to the Data Link Layer, a LAN can be defined as a segment, that is - a broadcast domain. In this case, two devices are regarded as part of the same segment iff they are one hop away from one another - that is, they can switch frames in the second layer.
While reading the book on TCP/IP I came across the words which are as "Although it looks as though the use of the flow label may make the source and destination addresses useless, the parts of the Internet that use connection-less service at the network layer still keep these addresses for several reasons.One reason is that part of the packet path may still be using the connection-less service. Another reason is that the protocol at the network layer is designed with these addresses and it may take a while before they can be changed". Now my question to you is if a connection has been formed between hosts in a connection-oriented manner then how come a path of a packet may still be using the connection-less services. Because as per my knowledge prevails the virtual path always be formed at while 3-way handshake is taking place which is the TCP/IP connection (which uses a connection-oriented service) ? And my second question for the second reason is that which protocol they are talking about since these words are stated below the Heading of "Connection-Oriented Services" therefore, it's making me pissed off to understand the literal meaning behind the words(The core conceptual understanding). And correct if anyone thinks I am having a wrong concept at any place. I'll be obliged. Thanks.
TCP as a connection-oriented protocol runs on top of IP which is connection-less. The routers used in transport only look at the IP packet, the TCP segment is simply payload and transported along. TCP provides several algorithms to form a virtual connection over a connection-less network.
The IP packet goes from hop to hop. On each hop, a router makes a forwarding decision solely based on the destination IP address. (More sophisticated devices may inspect more packet elements including source address and payload, but they aren't simple routers.)
The "path" is made up of all these individual hops. Because each hop is based on an independent routing decision the path can change at any time and for any packet. The path is not laid out by the TCP handshake.
Basically, you have to look at each protocol layer individually. Each one serves its own function.
I hope this also answers the second question.
I'm trying to understand Network layers. There are five layers, application, transport, network, link and physical layer. I have studied from various sources but I couldn't understand clearly, especially transport, network and link layers. Can you explain those layers for a dummy person like me? Thanks.
So I think the hardest part people have with the OSI model is that they look at some of the layers and forget its talking about communication... application layer specifically people get confused as to what its talking about.
Application references the protocols applications use to communicate.
Examples being ftp, http, etc.
As stated above, the original OSI model (which is really just theory, nothing actually implements it exactly as it would be too inefficient) has 2 more layers in between application and transport (which are most likely wrapped into the application layer). These are Presentation and Session.
Presentation is responsible for things like HTTP's Accept-Encoding: gzip, deflate. encryption and character encoding are said to be Presentation layer, so SSL/TLS falls under this category (the s in https).
HTTP Sessions is literally would be a literal example of the Session layer (anything that keep your application state alive across connections). If a protocol is connection oriented, it might not have a session layer, which is why HTTP is probably the only example I can think of at the moment.
As you can see HTTP was the answer to all 3 of these top layers, which is why they have all been combined into Application layer in newer versions of the model. Cisco still uses the 7 layer model, but Microsoft uses the 5 layer you're using.
Transport is TCP... it contains information about reordering packets and can adjust the amount of packets per window, allows the computer to know if anything was lost and ask for re-transmission, etc.
UDP is also an example of the transport layer, however its a lot simpler of a protocol, no re-transmission of packets are done on UDP.
Network is the IP protocol (also IPX/SPX from the old netware days, and ICMP (pings) and IGMP (routers)) this allows for addressing computers that rely outside the a collision domain (things separated by switches or routers).
Link or DataLink layer is ethernet, (or ATM, or FDDI) which addresses computers physically connected to a hub or directly with a network cable to each other.
This layer adds the MAC addressing in ethernet and the frame part of a packet is the header it uses.
Physical layer (in original OSI model) is just your cables and network equipment.
Pretty much the only people who still talk about OSI is network techs.
They may still say Layer 2 or Layer 3 switches, etc which is in reference to OSI.. Layer 2 is an ethernet switch, a layer 3 switch adds routing.
The best way to see how this is used is to load up wireshark and snoop your own network traffic. It will actually show you the parts of a packet that are responsible for most of the layers.
Knowing the OSI model is not really overly useful, but it can help you organize the stages of network communication in your mind and help you troubleshoot.
Knowing the protocols and how they interact is extremely useful, learn how tcp negotiates connections, IP addressing and subnet masking, HTTP and Ethernet can help you whether you are a developer or a server admin, or network admin or even a DBA. There is nothing like a bad network card to ruin your weekend or even a whole month if no one thinks to check it, and the only way to tell is use tcpdump, and wireshark to see errors in the Ethernet frames.
The OSI reference model
The OSI model is used to connect to the open systems—these are the systems that are open and communicate with other systems. By using this model, we do not depend on an operating system anymore, so we are allowed to communicate with any operating system on any computer. This model contains seven layers, where each layer has a specific function and defines the way data is handled on certain different layers. The seven layers that are contained in this model are the Physical layer, Data Link layer, Network layer, Transport layer, Session layer, Presentation layer, and the Application layer.
THE PHYSICAL LAYER
This is the first layer in the OSI model and contains a definition of the network's physical specification, including the physical media (cables and connectors) and basic devices (repeaters and hubs). The layer is responsible for the input raw bits transmission data stream into zeros and for the ones that are on the communication channel. It then places the data onto the physical media. It is concerned with data transmission integrity and makes sure that the bits that are sent from one device are exactly the same as the data that is received by the other device
THE DATA LINK LAYER
The main role of the Data Link layer is to provide a link for raw data transmission. Before the data is transmitted, it is broken up into data frames, and the Data Link layer transmits them consecutively. The receiver will send back an acknowledge frame for each frame that has been sent if the service is reliable.
This layer consists of two sublayers: Logical Link Control (LLC) and Media Access Control (MAC). The LLC sublayer is responsible for transmission error checking and deals with frame transmission, while the MAC sublayer defines how to retrieve data from the physical media or store data in the physical media.
We can also find the MAC address, also called as the physical address, in this layer. The MAC address is used to identify every device that connects to the network because it is unique for each device.
The MAC address contains twelve hexadecimal characters, where two digits are paired with each other. The first six digits represent the organizationally unique identifier and the remaining digits represent the manufacturer serial number. If you are really curious to know what this number means, you can go to www.macvendorlookup.com and fill the text box with our MAC address to know more about it.
THE NETWORK LAYER
The Network layer is responsible for defining the best way to route the packets from a source to the destination device. It will generate routing tables using Internet Protocol (IP) as the routing protocol, and the IP address is used to make sure that the data gets its route to the required destination. There are two versions of IP nowadays: IPv4 and IPv6. In IPv4, we use 32-bit addresses to address the protocol and we use 128-bit addresses in IPv6. You are going to learn more about Internet Protocol, IPv4, and IPv6 in the next topic.
THE TRANSPORT LAYER
The Transport layer is responsible for transferring data from a source to destination. It will split up the data into smaller parts, or in this case segments, and then will join all the segments to restore the data to its initial form in the destination.
There are two main protocols that work in this layer: the Transmission Control Protocol (TCP) and the User Datagram Protocol (UDP).
TCP supplies the delivery of data by establishing a session. The data will not be transmitted until a session is established. TCP is also known as the connection-oriented protocol, which means that the session has to be established before transmitting the data.
UDP is a method of delivering data with the best efforts, but does not give a guaranteed delivery because it does not establish a session. Therefore, UDP is also known as the connection-less protocol. In-depth explanation about TCP and UDP can be found in the next topic.
THE SESSION LAYER
The Session layer is responsible for the establishment, maintenance, and termination of the session. We can analogize the session like a connection between two devices on the network. For example, if we want to send a file from a computer to another, this layer will establish the connection first before the file can be sent. This layer will then make sure that the connection is still up until the file is sent completely. Finally, this layer will terminate the connection if it is no longer needed. The connection we talk about is the session.
This layer also makes sure that the data from a different application is not interchanged. For example, if we run the Internet browser, chat application, and download manager at the same time, this layer will be responsible for establishing the session for every single application and ensure that they remain separated from other applications.
There are three communication methods that are used by this layer: the simplex, half-duplex, or full-duplex method.
In the simplex method, data can only be transferred by one party, so the other cannot transfer any data. This method is no longer common in use, since we need applications that can interact with each other.
In the half-duplex method, any data can be transferred to all the involved devices, but only one device can transfer the data in the time, after it completes the sending process. Then, the others can also send and transfer data.
The full-duplex method can transfer data to all the devices at the same time. To send and receive data, this method uses different paths.
THE PRESENTATION LAYER
The Presentation layer role is used to determine the data that has been sent, to translate the data into the appropriate format, and then to present it. For example, we send an MP3 file over the network and the file is split up into several segments. Then, using the header information on the segment, this layer will construct the file by translating the segments.
Moreover, this layer is responsible for data compression and decompression because all the data transmitted over the Internet is compressed to save the bandwidth. This layer is also responsible for data encryption and decryption in order to secure communication between two devices.
THE APPLICATION LAYER
The Application layer deals with the computer application that is used by a user. Only the application that connects to a network will connect to this layer. This layer contains several protocols that are needed by a user, which are as follows:
The Domain Name System (DNS): This protocol is the one that finds the hostname of an IP address. With this system, we do not need to memorize every IP address any longer, just the hostname. We can easily remember a word in the hostname instead of a bunch of numbers in the IP address.
The Hypertext Transfer Protocol (HTTP): This protocol is the one that transmits data over the Internet on web pages. We also have the HTTPS format that is used to send encrypted data for security issues.
The File Transfer Protocol (FTP): This protocol is the one that is used to transfer files from or to an FTP server.
The Trivial FTP (TFTP): This protocol is similar to FTP, which is used to send smaller files.
The Dynamic Host Configuration Protocol (DHCP): This protocol is a method that is used to assign the TCP/IP configuration dynamically.
The Post Office Protocol (POP3): This protocol is an electronic mail protocol used to get back e-mails from POP3 servers. The server is usually hosted by an Internet Service Provider (ISP).
The Simple Mail Transfer Protocol (SMTP): This protocol is in contrast with POP3 and is used to send electronic mails.
The Internet Message Access Protocol (IMAP): This protocol is used to receive e-mail messages. With this protocol, users can save their e-mail messages on their folder on a local computer.
The Simple Network Management Protocol (SNMP): This protocol is used to manage network devices (routers and switches) and detect problems to report them before they become significant.
The Server Message Block (SMB): This protocol is an FTP that is used on Microsoft networks primarily for file and printer sharing.
This layer also decides whether enough network resources are available for network access. For instance, if you want to surf the Internet using an Internet browser, the Application layer decides whether access to the Internet is available using HTTP.
We can divide all the seven layers into two section layers: the Upper Layer and Lower Layer. The upper layer is responsible for interacting with the user and is less concerned about the low-level details, whereas the lower layer is responsible for transferring data over the network, such as formatting and encoding.
There are FIVE LAYERS in TCP/IP Model and SEVEN LAYERS in OSI Reference Model.
The primary difference between our five-layer model and the seven-layer OSI model is that the OSI model abstracts the application layer into three layers total.
The physical layer represents the physical devices that interconnect computers. This includes the specifications for the networking cables and the connectors that join devices together along with specifications describing how signals are sent over these connections. The physical layer is all about cabling, connectors and sending signals.
The second layer in our model is known as the data link layer. Also called the network interface or the network access layer. The data link layer is responsible for defining a common way of interpreting the signals, so network devices can communicate. Lots of protocols exist at the data link layer, but the most common is known as Ethernet.
The third layer, the network layer is also sometimes called the Internet layer. It's this layer that allows different networks to communicate with each other through devices known as routers.
The most common protocol used at this layer is known as IP or Internet Protocol. IP is the heart of the Internet and most small networks around the world.
While the data link layer is responsible for getting data across a
single link, the network layer is responsible for getting data
delivered across a collection of networks.
While the network layer delivers data between two individual nodes, the transport layer sorts out which client and server programs are supposed to get that data. The transport layer Protocol is known as TCP or Transmission Control Protocol and UDP or User Datagram Protocol.
The big difference between the two is that TCP provides mechanisms to
ensure that data is reliably delivered while UDP does not.
The network layer, in our case IP, is responsible for getting data
from one node to another. The transport layer, mostly TCP and UDP, is
responsible for ensuring that data gets to the right applications
running on those nodes.
The fifth layer is known as the application layer. There are lots of different protocols at this layer, and as you might have guessed from the name, they are application-specific. Protocols used to allow you to browse the web or send receive emails are some common ones. Application Layer Protocols are HTTP, SMTP, etc.
You can think of layers like different aspects of a package being
delivered. The physical layer is the delivery truck and the
roads. The data link layer is how the delivery trucks get from
one intersection to the next over and over. The network layer
identifies which roads need to be taken to get from address A to
address B. The transport layer ensures that the delivery
driver knows how to knock on your door to tell you your package has
arrived. And the application layer is the contents of the
package itself.
I know, how to write a C# application that works through a local network.
I mean I know, how to make my client-side application access my server-side application in a single local network.
But I wonder: How do such apps, as Skype, TeamViewer, and many other connect via global network?
I apologise, if this question is simple or obvious, but I couldn't find any information about this stuff.
Please, help me, I'll be very grateful. Any information is accepted - articles, plain info, books,and so on...
Question is very wide and I try to do short overview.
Following major difference between LAN (Local Area Network) and WAN (Wide Area Network):
Network quality:
LAN is more or less stable, WAN can be with network issues like:
Packet loss (you need use loss-tolerant transport like TCP or UDP with retransmits or packet loss concealment)
Packet jitter (interpacket intervals may differ a lot from sending part). Most common thing is packets bursts.
Packet reordering
Packet duplication
Network connectivity
WAN is less stable than LAN. So you need properly handle all things like:
Connection stale
Connection loss
Errors in the middle of the connection (if you use UDP for example)
Addresses:
In WAN you deal with different network equipment between client and server (or peers in case of peer-to-peer communication). You need to take in account:
NATs - most of the clients are behind NAT and you need to pass them through. According technics are called "NAT traversal"
Firewalls - may ISP has own rules what client can do or can't. So if you do something specific like custom transport protocol you may bump into ISP firewalls.
Routing - especially multicast and broadcast communication. In common case multicast is not possible to route. Broadcasts are never routed. So you need to avail this type of communication if you want to use WAN.
May be I forgot something. But these points are major. You can read many articles about any of them.
According to my knowledge if an internet application has to be designed, we should use either a connection-oriented service or connection-less service, but not both.
Internet's connection oriented service is TCP and connection-less service is UDP, and both resides in the transport layer of Internet Protocol stack.
Internet's only network layer is IP, which is a connection-less service. So it means whatever application we design it eventually uses IP to transmit the packets.
Connection-oriented services use the same path to transmit all the packets, and connection-less does not.
Therefore my problem is
if a connection oriented application has been designed, it should transmit the packets using the same path. But IP breaks that rule by using different routes.So how do both TCP and IP work together in this sense? It totally confuses me.
You, my friend, are confusing the functionality of two different layers.
TCP is connection oriented in the sense that there's a connection establishment, between the two ends where they may negotiate different things like congestion-control mechanism among other things.
The transport layer protocols' general purpose is to provide process-to-process delivery meaning that it doesn't know anything about routes; how your packets reach the end system is beyond their scope, they're only concerned with how packets are being transmitted between the two end PROCESSES.
IP, on the other hand, the Network layer protocol for the Internet, is concerned with data-delivery between end-systems yet it's connection-less, it maintains no connection so each packet is handled independently of the other packets.
Leaving your system, each router will choose the path that it sees fit for EACH packet, and this path may change depending on availability/congestion.
How does that answer your question?
TCP will make sure packets reach the other process, it won't care HOW they got there.
IP, on the other hand, will not care if they reach the other end at all, it'll simply forward each different packet according to what it sees most fit for a particular packet.
Note:
Let's assume that IP was connection-oriented, would that mean packets would follow the same-path?
Not necessarily, it depends on what the word 'connection' at this layer means, if it means negotiating certain options related to security, for instance, you may still have all your packets being forwarded through different routes over the Internet.
EDIT:
Not to confuse you though, most connection-oriented services at the network-layer and below mean that the connection, when established, also establishes a virtual-path that all 'packets' must follow, for further information read about:
Virtual circuit and frame-relay networks
This link answers your question pretty well http://www.tcpipguide.com/free/t_ConnectionOrientedandConnectionlessProtocols-3.htm
Some people consider this (TCP) to be like a “simulation” of circuit-switching at higher network layers; this is perhaps a bit of a dubious analogy. Even though a TCP connection can be used to send data back and forth between devices, all that data is indeed still being sent as packets; there is no real circuit between the devices. This means that TCP must deal with all the potential pitfalls of packet-switched communication, such as the potential for data loss or receipt of data pieces in the incorrect order.
The TCP protocol deals with the problem of IP packets arriving out of order or being lost, to give you the feeling they arrive through a single FIFO channel. Yes, TCP is smart enough to do that, there's no need for a dedicated underlying channel.
The TCP protocal is implemented by the sending/receiving machines, once the packets leave the sending machine, the routers they travel along know nothing about TCP, they just use IP to get the packets from the source the to destination. Then, it is the destination machines job to, using TCP, make sure that all the packets arrive and that they arrive in the correct order. The internet itself doesn't know anything about TCP, it's just a layer (often software) that gives connection to a connectionless medium (the internet).
So onces a packet leaves a destination, it can go along any path (mostly) as long as it gets to the desintation, regardless of the higher level protocol (such as TCP or UDP).
I mean, it's a bit more complicated then that, but as far as I can remember that's the general Idea.
Refer my short points properly,
1) Connection oriented means ==> reserving resources(buffer,cpu,bandwidth etc.)..but "Where??".(where resources are reserved?? This where is reason of your confusion, so following is ans.).
2) Connection oriented at Transport Layer means ==> Reserving the resources at Both End processes/Ports.(Since TCP is a transport layer,then its responsibility is to reserve the resources at both end processes only,irrespective of whats happening in the intermediate path.)
3) Connection oriented at Network Layer means ==> Reserving the resources at Network Layers.(Now In the whole journey of a packet from source to destination, Network layer is found at all intermediate routers too(but not transport layer). Hence if any protocol at Network layer is connection oriented then,its responsibility is to reserve resources at all intermediate routeres too i.e. all packets will have to follow same intermediate path, But IP is connection less hence,intermediate resources will not be reserved. i.e journey of a packets may follow different paths etc.)
#CONCLUSION:==> Intermediate path is decided by Network Layer, hence if IP then paths may be different.(IP may contain TCP),But TCP is responsible for Resource reservation at both End processes ,irrespective of Intermediate path of packet.
router works on three layers only (physical , data link and Network layers) , so routers will take decision depending only on the info. of network layer (IP protocol ) hence there is no information available about its TCP or UDP at the router