I would like to understand what happens when we type "google.com" in our browser wrt OSI model. What all protocols comes into picture AT EACH LAYER any how does they know which one to be used?
Also I would like to know, TCP/UDP which one to be used in Transport layer is decided for network/application? That is for a normal web page like google.com TCP must be used but for video streaming UDP. how is this conveyed to network?
Is the browser also part of this as it is helping. Should it also come under application layer?
Http comes under Application layer of OSI model. now for this particular example, what will come under Presentation and Session layers? Will the sessions be maintained in session layer or HTTP will be doing it on their behalf?
It sounds like you would like a tutorial on TCP/IP. May I suggest a couple of good books
URLs in webpages have a prefix that tells the applications what to do with that link. For instance, ftp:// opens an ftp session, which uses tcp (ref RFC 959 - File Transfer Protocol). As a general rule, the URL prefixes correspond to a standardized protocol, which was specified via RFC. The RFC denotes which transport protocol is used.
EDIT
In the case of youtube, their videos have a http:// URL prefix, but after you make the request, it is redirected to a rtsp:// URL. RTSP can be streamed over UDP.
To answer questions like this, you may find that wireshark is very useful to investigate these things... keep in mind that wireshark works best on wired connections.
Applications don't use any layer of the OSI protocol stack. They use TCP or UDP from the TCP/IP stack, and they 'know which' because of they are programmed to use one or the other or both. The OSI model is obsolete and certainly doesn't apply to TCP/IP.
EDIT: The OSI layer model is a Procrustean bed into which TCP/IP does not fit. You should forget about OSI immediately. It doesn't describe anything in the real world.
Related
I am learning about Network Protocols, and see that near the lowest level is the raw ethernet protocol:
Ethernet: this is the basic protocol that sends data to another machine on your local network using your MAC address. This is the building block for all the rest as you need to send data to the router if you want to communicate with the outside world.
Above that is the Internet Protocol (IP), of which TCP and UDP are examples. On top of TCP is HTTP, or SSL on top of which you can put HTTP (so HTTPS).
It seems that the browser only lets you go down to the level of the IP (TCP for your normal message passing connections, and UDP for things like video). But I'm wondering, if it's possible to go any lower than that in the browser. If not, then wondering why not. If so, then what an example might be.
“It seems that the browser only lets you go down to the level of the
IP (TCP for your normal message passing connections, and UDP for
things like video).”
Firstly TCP and UDP are not examples of IP. They are Transport layer protocols, which is a layer above IP.
This representation of the 5 layer model should help make the distinction:
Secondly, if by 'go down to the level of Ethernet’ you mean inspect packets with your browser’s dev tools, you can’t do that. You can only see HTTP packets.
If you want to observe all traffic, you can use tools like Wireshark - or Fiddler to capture and inspect packets across the different layers.
If you are asking if the browser can use any layers other than the Application Layer. No, it cannot. That's also why you can only observe HTTP packets, because that's the layer which the browser operates on.
Information travels from layer to layer, but doesn't skip layers:
Your request for a web page uses all the layers. It starts at the application layer, which is the user-facing layer, all the way down the stack until the Network layer transports your requests to the server's Network layer, where it goes up the stack to fetch the web page.
The packet is the basic unit of information that is transferred across
a network. The packet consists, at a minimum, of a header with the
sending and receiving hosts' addresses, and a body with the data to be
transferred. As the packet travels through the TCP/IP protocol stack,
the protocols at each layer either add or remove fields from the basic
header. When a protocol on the sending host adds data to the packet
header, the process is called data encapsulation. Moreover, each layer
has a different term for the altered packet, as shown in the following
figure.
You can read more about the way packets travel across the protocol stack in this article about Data Encapsulation and the TCP/IP Protocol Stack
Browsers only allow to use a well-defined set of protocols due to security reasons. If this wouldn't be the case, Javascript on websites could send send arbitrary packets and cause malicious behavior.
Therefore the browser APIs are limited to make only HTTP, websockets and webrtc available.
In good old days (until about 3 years ago) browsers solved that problem by allowing to run a 3-rd party plugin, and there were (and there are still are) APIs for these plugins, such as NPAPI and ActiveX control. So you could create a plugin that does anything you want, send data with raw sockets or your own protocol. Flash plugin is the best example, and there are numerous less known ones, used in specific web applications, such as web conferencing, chat, secure data exchange etc... Some browsers still support these plugins - IE, Opera, Firefox, but Chrome and Edge completely disabled them. The reason is security issues - if you can load your own native code into browser then you can have a full access to OS, intercept all data in the browser, etc... As a replacement for this disabled freedom, some browsers tried to invent new more safe APIs such as PPAPI of Chrome. Mostly a failed attempt. Browser's adoption of websockets mitigated the situation, but still, lots of networking things are not possible in the browser. One notorious example is the lack of UDP Multicast support in browsers. Previously you could do it with your own NPAPI plugin; now no replacement is offered by browsers, so your browser-based multicast-involved applications are dead.
Some said HTTP is in the session layer in the OSI model.
But in Tanenbaum's Computer Network, HTTP is said to be in the application layer in the OSI model.
Also some said that HTTP has no concept of session. Does it mean that HTTP can't be in the session layer?
So is HTTP in the session layer? Thanks.
Update: For HTTP/2 what is the layer in OSI model?
In which layer is HTTP in the OSI model?
It's in the application layer. See the following quotes from the RFC 7230, one of the documents that currently defines the HTTP/1.1 protocol:
The Hypertext Transfer Protocol (HTTP) is a stateless application-level request/response protocol that uses extensible semantics and self-descriptive message payloads for flexible interaction with network-based hypertext information systems.
HTTP is a stateless request/response protocol that operates by exchanging messages across a reliable transport- or session-layer "connection".
Also some said that HTTP has no concept of session. Does it mean that HTTP can't be in the session layer?
As previously mentioned in the quotes from the RFC 7230, the HTTP protocol is stateless, where each request from client to server (should) contain all of the information necessary to understand the request, without taking advantage of any stored context on the server.
The RFC 6265 defines some mechanisms for state management in HTTP, such as cookies, allowing session management on server side (but it doesn't make HTTP stateful in any ways).
The concept of session in HTTP is different from the concept of session in the OSI model. Anyways, HTTP is an application layer protocol.
The OSI model
The OSI (Open Systems Interconnection) model is a conceptual model created by the International Organization for Standardization which enables diverse communication systems to communicate using standard protocols.
It provides a standard for different computer systems to be able to communicate with each other and can be seen as a universal language for computer networking. It’s based on the concept of splitting up a communication system into seven abstract layers, each one stacked upon the last.
The following picture borrowed from Cloudflare illustrates pretty well what the OSI model is like:
The application layer is the only layer that directly interacts with data from the user. So software applications like web browsers and email clients rely on the application layer to initiate communications.
But it should be made clear that client software applications are not part of the application layer: rather the application layer is responsible for the protocols (such as HTTP and SMTP) and data manipulation that the software relies on to present meaningful data to the user.
The OSI model vs the TCP/IP model
While the OSI model is comprehensive reference framework for general networking systems, it's important to mention that the modern Internet doesn’t strictly follow the OSI model.
The modern Internet more closely follows the simpler Internet protocol suite, which is commonly known as TCP/IP because the foundational protocols in the suite are the TCP (Transmission Control Protocol) and the IP (Internet Protocol).
The following image illustrates how the OSI and TCP/IP models relate to each other:
Update: This section has been added to address the bounty started by noɥʇʎԀʎzɐɹƆ, who requested to update this answer with HTTP/2 details.
Despite the quotes of the document that defines the HTTP/1.1 protocol, all of the above also applies to HTTP/2. Refer to the following quote from the RFC 7540, the document that defines the HTTP/2 protocol:
An HTTP/2 connection is an application-layer protocol running on top of a TCP connection. The client is the TCP connection initiator.
The HyperText Transfer Protocol (HTTP), is the Web’s application-layer protocol,
is at the heart of the Web. It is defined in [RFC 1945] and [RFC 2616].
HTTP is in the Application layer of the Internet protocol suite model and in the Session Layer of the OSI Model. The Session layer of the OSI Model is responsible for creating and managing sessions and is the first layer that passes data.
HTTP can redirect sessions, reuse them and have persistent connections.
I am wondering about the difference between HTTP and TCP data transfer protocols for online games.
I have heard many people using TCP or UDP to transfer data between client and server for online games.
But can you use http at all? I know http is mostly used for web browsing, but if I could set up web server and let my game applications use GET and POST methods, I can still send data back and forth right? Is it that this way of communicating is too slow or unnecessary?
And just one thing about TCP transmission protocols, if I were to write some gaming application using TCP, is it that the data are usually transferred using something called "sockets" (like Socket classes in Java)? What about UDP?
Thanks very much!
Appreciate any answer!
HTTP is an additional layer on top of TCP that defines what a request looks like, what a response looks like, and how the connection is closed or maintained across requests. You can either use it or not use it, depending on what you actually need to transport. If your game consists of a series of requests that each get a reply, HTTP might make sense. If it's more like unsolicited messages in each direction, making HTTP work is like putting a square peg in a round hole.
Most platforms provide a socket interface that allows you to work with either TCP or UDP depending on the protocol specified when the socket is allocated. Some higher-level APIs look completely different for different protocols.
This might be a dumb question, however, I have been continually frustrated by what seems to be a big gap in every explanation I've seen of protocols like XMPP or HTML. So basically, when I've read documentation on either, in general, it will describe the structure of the data sent back and forth through the protocol, but it does not explain exactly how this data is transferred. It's one thing to provide an example of, say, a generic HTTP request, but it is something else to explain how this text is actually sent to the server.
I guess posed another way, what resources are there out there for learning best practices for implementing text-based protocols? At their core, are all text-based protocols basically the exact same thing? How, for example, would it differ at the binary level, were I to say send the text content of an HTTP request over IRC vs however it is done natively by HTTP?
If I wanted to develop my own, simple textual protocol, what would be the best way to send the text to a client? Does the content itself even really matter? What I mean is that, obviously, HTTP and XMPP are rather different protocols, but do they differ in terms of how the text is transferred between computer to computer?
HTTP, IRC and XMPP are all sent on top of TCP, which is a protocol that provides a bidirectional stream between two endpoints (IP address + port). Under the hood, the data you send is split into separate packets, sent across the network, and reassembled on the other end, so that the recipient just sees a stream of incoming data - except when something goes wrong; there is a somewhat accessible description here.
What that means is that while the application protocol (HTTP, XMPP etc) is different, the underlying transport mechanism is exactly the same. It would be possible (perhaps even interesting) to implement HTTP on top of IRC: an HTTP/IRC client enters a channel, sends the HTTP request as messages to the channel, line by line, a server is present in the channel, reads the request and sends the response the same way - but transporting HTTP over IRC is fundamentally different from transporting HTTP over TCP. The former means layering an application protocol over another application protocol (and the IRC connection needs to go over TCP anyway), while the latter is an application protocol over a transport protocol, which is the way things usually are done (except for various kinds of proxies).
Hope that makes some sense...
What protocols do web cameras use for streaming audio/video feeds over the internet? HTTP? TCP? How is each frame sent inside the protocol? For example, if they use HTTP, does the web cam software encode each frame and tack it on as a query string parameter like:
http://www.some-url.com?encoded-frame=WJDJ84FU84F85594DK3DK
or, is the encoded frame set as the HTTP request's body? Similar question for TCP or any other protocol that is used.
I'm asking because I'd like to stream a web cam to a web server and have software that receives each encoded frame, decodes it, and does something with it. Thanks in advance.
Well the question in OP is open ended because it's not like there is 'one fixed set of protocols(TCP/UDP)' used in this kind of applications and also its scope is large, due to various technologies involved in this end-to-end solution of Camera capturem encoding , streaming, decoding/processing. In the case you mentioned if it is going to be likely that the webcam and the Web server are going to be on same LAN, then well you can as well use TCP/IP and then server can process it. Because on LAN latencies won't be high, so TCP would serve good. Else if on WAN, then UDP/IP can be of help.
There are plenty of tutorials online to get basics of using TCP/IP or UDP/IP sockets and its programming concepts. Then there are tutorials about streaming, packetization etc of Video data.
I don't see how HTTP can be of use here to send from webcam to a server.
For starters
http://streaminglearningcenter.com/streaming-video-consulting.html
Hope this is good to get you started.