In the NetworkExtensions framework on macOS, the only 2 protocols available in configuration are TCP and UDP.
For example, to write a firewall, apple recommends using NEFilterRule and NENetworkRule. An NENetworkRule takes a protocol as an argument, and the only available values are TCP or UDP.
What happens if a rogue application on the machine with the firewall tries to communicate with the outside world via another protocol? Maybe a custom protocol with no clear rules. Does the NetworkExtension framework allow filtering these cases?
Related
Why was UDP used as the standard protocol in NFS systems up to version 3 and TCP in version 4? Unfortunately, I have not yet found anything about the motives that led to this.
The Red Hat NFS documentation provides some insight about why UDP was preferred:
When using NFSv2 or NFSv3 with UDP, the stateless UDP connection under normal conditions has less Protocol overhead than TCP which can translate into better performance on very clean, non-congested networks.
...and why the move to allow and eventually require TCP was made:
However, because UDP is stateless, if the server goes down unexpectedly, UDP clients continue to saturate the network with requests for the server. For this reason, TCP is the preferred protocol when connecting to an NFS server.
The current RFC also notes that TCP facilitates transport security without needing to tunnel:
Historically, NFSv2 and NFSv3 servers have resided on port 2049. The registered port 2049 [RFC3232] for the NFS protocol SHOULD be the default configuration. Using the registered port for NFS services means the NFS client will not need to use the RPC binding protocols as described in [RFC1833]; this will allow NFS to transit firewalls.
Why TCP specifically? Actually, the RFC says SCTP is also acceptable (though TCP support is mandatory):
Where an NFSv4 implementation supports operation over the IP network protocol, the supported transport layer between NFS and IP MUST be an IETF standardized transport protocol that is specified to avoid network congestion; such transports include TCP and the Stream Control Transmission Protocol (SCTP). To enhance the possibilities for interoperability, an NFSv4 implementation MUST support operation over the TCP transport protocol.
Finally, addressing performance, the original reason UDP was chosen, the RFC says:
If TCP is used as the transport, the client and server SHOULD use persistent connections. This will prevent the weakening of TCP's congestion control via short-lived connections and will improve performance for the Wide Area Network (WAN) environment by eliminating the need for SYN handshakes.
Ultimately, only the engineers at Sun Microsystems (as the creators of NFS) know the exact reasons UDP was chosen and only the IETF working group for NFS (as the maintainers) know why the switch to TCP was made.
I'm currently developing a "node-based" system where a server will send out a UDP broadcast on the private network (with a custom protocol), which will be received by several different clients which supports the specified protocol. The server will after the request pick between some of the clients for a more steady TCP connection.
Request for client sequence
Server broadcasting a request-for-ip message to every device/node on the network.
All available clients that supports the protocol will answer with their unique IP to the server.
Server chooses among the clients via a request-for-connection message.
Client that got choosen by the server connects to the server via TCP for a reliable connection.
My question
I've got pretty good knowledge about both TCP and UDP, but I've never designed a system like this before. Do you think this system is built in the right way or is there a more "standard" way doing something similar to this? What are your thoughts?
Thanks!
--- Edit ---
Added a diagram of the program.
There is a standard protocol to advertise services on the network, which you may like to consider: Simple Service Discovery Protocol, based on periodic UDP multicast:
The Simple Service Discovery Protocol (SSDP) is a network protocol based on the Internet protocol suite for advertisement and discovery of network services and presence information. It accomplishes this without assistance of server-based configuration mechanisms, such as Dynamic Host Configuration Protocol (DHCP) or Domain Name System (DNS), and without special static configuration of a network host. SSDP is the basis of the discovery protocol of Universal Plug and Play (UPnP) and is intended for use in residential or small office environments.
In this protocol clients join that UDP multicast group to discover local network services and initiate connections to them, if they wish to. And this is pretty much the intended use case for the protocol, which is somewhat different from your use case.
One benefit of IP/UDP multicast is that multicast packets can be dropped in the network adapter if no process on the host has joined that multicast group. Another one is that IP/UDP multicast can be routed across networks.
From the diagram you posted:
The server is the mediator (design pattern) whose location must be known to every other process of the distributed system.
The clients need to connect/register with the server.
Your master client is a control application.
It makes sense for the server to advertise itself over UDP multi-cast.
Online clients would connect to the server using TCP on start or TCP connection loss. If a client terminates for any reason that breaks the TCP connection and the server becomes immediately aware of that, unless the client was powered off or its OS crashed. You may like to enable frequent TCP keep-alives for the server to detect dead clients as soon as possible, if no data is being transmitted from the server to the clients. Same applies to the clients.
All communications between the server and the clients happen over TCP. Otherwise you would need to implement reliable messaging over UDP or use PGM, which can be a lot of work. Multicast UDP should only be used for server discovery, not bi-directional communication that requires reliable delivery.
The master client also connects to the server, possibly on another port, for control. The master client can discover all available servers (if there is more than one) and allow the user to choose which one to connect to.
Let's assume that I start a server at one of the computers in my private network (192.168.10.10:9900).
Now when making a request from some other computer in the same network, how does the client computer (OS?) knows which protocol to use / which protocol the server follows ? [TCP or UDP]
EDIT: As mentioned in the answers, I was basically looking for a default protocol which will be used by the client in the absence of any transport protocol information.
TCP / UDP protocols work at the transport layer level (TCP / IP MODEL) and its main difference is that TCP has a method to ensure the arrival of messages while UDP is lighter because of its virtue is to be faster in Information delivery. The use of one protocol or another is always defined by the application that will use it.
So the reference you put on the private server with ip: port 192.168.10.10:9900 is very vague to be more precise we could say that we have an Apache web server running on the ip: port 192.168.10.10:9900 (the port for default is 80 when installing the server, but it can be changed in the configuration).
Now the web servers (apache, IIS, etc.) work using the TCP protocol because when a client (computer, cell phone, etc.) consults a page through a browser (Chrome, Firefox, etc.), the ideal thing is that all the website and not just some pieces. This is why this type of servers chose and use this protocol in the first instance since they seek that in the end the result is that the user obtains the complete page regardless of whether a few more milliseconds are sacrificed in the validations involved in using TPC.
Now going to the client side. The user when visiting a web page from any browser (Chrome, Firefox, etc.) will use TCP since this protocol is already configured in the browser to send the query messages and subsequently receive the messages with the same form Website information.
Now this behavior is going to be repeated for any client / server application. For example, to change the type of application on the UDP side, we can observe the operation of DHCP services which are used to receive an IP when connecting any device to a Wi-Fi network. In this case, this service seeks to be as fast as possible (instead of the most reliable) since you want the device to connect as quickly as possible to the network, so use the UDP protocol and in this case any equipment when connecting To a WIFI network you will send your messages using this protocol.
Finally, if you want to know promptly about the type of TCP / UDP protocol used by a specific application, you can search on the Wireshark application which allows you to scan the messages that leave the device or show the protocol used in the different layers of the application.
There is no reason any client would make a request to your server, so why would it care what protocol it follows? Clients don't just randomly connect to things to see if there's a server there. So it doesn't make any difference to any client.
Normally, the client computer will use the TCP protocol as default. If you start the server using UDP protocol mode, then when you use curl -XGET 192.168.10.10:9900/test-page, it will give you back an curl: (7) Failed to connect to 192.168.10.10 port 9900: Connection refused error. You can try it, use the nc -lvp 9900 -u, it will give you that result.
The answers here are pointing to some default protocol. Its' not that, Whenever you start an application let say HTTP server, the server's internal has code to open a socket(which can be TCP or UDP), since HTTP:80 is a TCP protocol the code creates a TCP socket. Similarly for other network application it depends on their requirement what kind of transport layer protocol to use (TCP Or UDP). Like a DNS client will create a UDP socket to connect to DNS server, since DNS:53 is mostly over UDP. Both TCP and UDP have different use cases, advantages and disadvantages. Depending on there uses cases / advantages / disadvantages of UDP/TCP decision is taken to implement server using either of them.
What is the practical difference between NetTcpBinding and NetPeerTcpBinding in WCF? When should I pick one over another?
Peer Network binding
This binding is provided by the NetPeerTcpBinding class. It uses TCP
protocol but uses peer net-working as transport. In this networking
each machine (node) acts as a client and a server to the other nodes.
This is used in the file sharing systems like torrent.
TCP binding
This binding is provided by the NetTcpBinding class. It uses TCP
protocol for communication be-tween two machines with in intranet
(means same network). It encodes the message in binary format. This is
faster and more reliable binding as compared to the Http protocol
bindings. It is only used when communication is WCF - to – WCF means
both client and service should have WCF.
More info about bindings:
Bindings
In simple words:
practical difference is that NetTCP have server side and client side applications, in peer2peer application is working for both client, and server for other clients. P2P is good for file shering (like torrent). NetTCP is better for "normal" apps, where you have server with connection to data store, and clients who use server to get access to data, and server logic.
TCP binding
NetTcpBinding
Offered by the NetTcpBinding class, TCP binding uses TCP for cross-machine communication
on the intranet. It supports a variety of features, including reliability,
transactions, and security, and is optimized for WCF-to-WCF communication. As
a result, it requires both the client and the service to use WCF
Peer network binding
Offered by the NetPeerTcpBinding class, this uses peer networking as a transport:
the peer-network-enabled client and services all subscribe to the same grid and
broadcast messages to it. Peer networking requires an understanding of grid topology and mesh computing strategies. People those who choose the peer bindings do so because they
are really after an implicit discovery mechanism or a publish-subscribe mechanism.
I've read about WebSockets and I wonder why browser couldn't simply open trivial TCP connection and communicate with server like any other desktop application. And why this communication is possible via websockets?
It's easier to communicate via TCP sockets when you're working within an intranet boundary, since you likely have control over the machines on that network and can open ports suitable for making the TCP connections.
Over the internet, you're communicating with someone else's server on the other end. They are extremely unlikely to have any old socket open for connections. Usually they will have only a few standard ones such as port 80 for HTTP or 443 for HTTPS. So, to communicate with the server you are obliged to connect using one of those ports.
Given that these are standard ports for web servers that generally speak HTTP, you're therefore obliged to conform to the HTTP protocol, otherwise the server won't talk to you. The purpose of web sockets is to allow you to initiate a connection via HTTP, but then negotiate to use the web sockets protocol (assuming the server is capable of doing so) to allow a more "TCP socket"-like communication stream.
Web browsers operate at the Application layer, whereas TCP operates at the Transport Layer. As a web application developer, it's easier to send messages over the wire via the Application Layer instead of raw bytes at the Transport Layer.
Underlying WebSockets is TCP, it's just abstracted away for simplicity.
Websocket is a application layer protocol while TCP is transport layer protocol. At transport layer, we usually have TCP and UDP protocol. Any message from application layer need to go through transport layer to be transmitted to other machine. Hence, websocket and tcp have a relationship to each other and can not be comparable.
To make it simple, the websocket communications are done over TCP port number 80 (or 443 in the case of TLS-encrypted connections), which is of benefit for those environments which block non-web Internet connections using a firewall.
Would you like to use existed TCP port or open a new TCP port that might be blocked by firewall?