I am designing an application protocol, and i am wondering if i still need include checksum in the protocol since tcp/ip already has checksum.
what's your opinion?
The BitTorrent protocol has a heavy amount of additional error correction and detection layered on top of TCP, so clearly the protocol designers saw the need for it.
The TCP checksum is quite weak, so you probably want an application level one if you are at all worried about reliability.
In particular the TCP checksum is not a secure hash, and there is no signature, so if you're worried about malicious changes then you need to add the security yourself.
To add to the other answers, you should probably look into Message Authentication Codes. MACs are a more robust way to detect errors than a simple TCP checksum.
If you want something robust, take a look at [HMAC][2]. HMAC provides both error detection and authentication (via shared keys).
If you want something quick and dirty, why not use sha1 hashes?
Related
After reviewing the differences between raw TCP and websocket, I am thinking to use websocket, even though it will be a client/server system with no web browser in the picture. My motivation stems from:
websocket is message-oriented, so I do not have to write down a protocol on top of the tcp layer to delimit messages myself.
The initial handshake of websocket is quite fitting for my use case as I can authenticate/authorize the user in this initial response-request exchange.
Performance does matter a lot here though, I am wondering if, excluding the websocket handshake, there would be a loss of performance between the websocket messages vs writing a custom protocol on raw tcp? If not, then websocket is the most convenient choice to me, even if I don't use the benefits related to the "web" part.
Also would using wss change the answer to the above question?
You are basically asking if using an already implemented library which perfectly fits your requirements and which even has the option for secure connections (wss) is better then designing and implementing your own message based protocol on TCP, assuming that performance and overhead are not relevant for your use case.
If you rephrase your question this way the answer should be obvious: using an existing implementation which fits your purpose saves you a lot of time and hassle for design, implementation and testing. It is also easier to train developers to use this protocol. It is easier to debug problems since common tools like Wireshark understand the protocol already.
Apart from this websockets have an established mechanism to use proxies, use a common protocol so that they can easier pass firewalls etc. So you will likely run into less problems when rolling out your application.
In other words: I can see no reason on why you should not use websockets if they fit your purpose.
After the player is authenticated, he receives a random, unique identification token. Every packet sent by him includes the token. A typical message would be:
token sequence_number commands
The problem is that, if somebody saw a packet, the person would be able to act as the player and give all the player's coins to somebody else for example. Or read the player's messages, or do anything short of stealing personal info.
So I thought encryption would be a requirement. But I've found many people advocating against it, one even says "it's complete paranoia" (https://gamedev.stackexchange.com/a/25551/33993).
Is there a way to avoid MITM attacks without encryption? If not, is avoiding them unnecessary?
The linked question says, "Only if it is an in-game purchase/micro-transaction - and then why not just use something tried and true like HTTPS."
Https is regarded as sufficient for MITM protection.
By which I mean: Yes!, you should send your traffic on Https!
It will have an impact on performance, but (obviously) lots and lots of effort has gone into optimizing that. A lot of the cost happens on establishing an Https connection. For an ongoing game connection, you should be able to keep it open, and the performance impact will be lessened.
As commenters have mentioned, Https uses TLS for encryption. You can build your own TCP or even UDP protocol using TLS, as well, but these days I'd recommend if possible, use boring old TCP/Https. All the platforms have APIs for it, there's ways to make it realtimey, and it plays probably the easiest with home routers & firewalls, and will be unsurprising when you explain it to others.
See pusher and socket.io and long polling for examples of using http/https for realtime.
MITM protection without encrypting the communication channel is difficult. You could use a Message Authentication Code (MAC) to protect key components from being changed. But you still need a shared secret that has to be exchanged over a different channel or using encryption (HTTPS).
You might want to take a look at the Hawk authentication scheme on how to exchange data securely over an unencrypted channel.
Typical encryption (example: HTTPS) is not sufficient to defeat MITM (http://en.wikipedia.org/wiki/Man-in-the-middle_attack) attacks. The wiki page has a simple example of how MITM is able to defeat encryption. There are complex methods to defeat MITM but they are typically overkill for a web game.
To answer your question, encryption does not defend against MITM, but it is necessary to implement encryption as it has more to do with securing sensitive information (example, passwords, session keys etc) as it passes through the internet. Otherwise anyone listening on the network will be able to see those information in plain text.
Just out of curiosity I was wondering if not having a checksum field in the application layer of the protocol is a major design issue? Or since the IP has the inbuilt checksum part in it, shouldn't it be an issue at all? Or you think is a dumb question as there is never a checksum in application layer?
Unless I am much mistaken FTP doesn't have a checksum, and neither does HTTP, and both are used to download enormous pieces of software by the million. Draw your own conclusion. Neither does RMI, or IIOP, or XDR, or ... In fact I can't think of an application protocol that does, other than one I wrote in 1994.
It depends on the integrity requirements of the application.
IP's checksum won't protect the application against packets that are lost or misordered. Applications that seek reliability usually use TCP (which provides a checksum over the data as well as recovering from loss and misordering).
The question then becomes whether an application needs its own checksum when TCP already provides one? That depends on whether the 16b checksum of TCP is sufficient for the integrity needs of the application. e.g. financial or other applications that are very sensitive to data changes might need to use a CRC or message digest to double-check the information after TCP has checked it.
I usually pass data between my web servers (in different locations) using HTTP requests (sometimes using SSL if it's sensitive). I was wondering if there were any lighter protocols that I might be able to swap HTTP(S) for that would also support public/private keys like SSH or something.
I used PHP sockets to build a SMTP client before so I wouldn't mind doing that if required.
There are lots and lots and lots of protocols. Lots. Start here for a list.
http://en.wikipedia.org/wiki/Internet_Protocol_Suite
SFTP is fun for passing data around. It works well. You'll find that it's not much better than HTTP, however, because HTTP is pretty simple.
http://en.wikipedia.org/wiki/SSH_file_transfer_protocol
SMTP would work. http://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol
SNMP can be made to work. http://en.wikipedia.org/wiki/Simple_Network_Management_Protocol You have to really push the envelope.
All of these, however, involve TCP/IP sockets, which involve a fair amount of overhead because of the negotiation for a connection and the acknowledgement of packets.
If you want real fun with very low overhead, use UDP.
http://en.wikipedia.org/wiki/User_Datagram_Protocol
You might want to use Reliable UDP if you're worried about messages getting dropped.
http://en.wikipedia.org/wiki/Reliable_User_Datagram_Protocol
I'd like to mention XMPP in addition to protocols already listed in other answers.
It's lightweight, and it is used in some "realtime" communication systems (for example, in GTalk).
WebSocket is a good option if you are interested in keeping a connection open to pass multiple messages back and forth. It's useful for issuing updates from the server to clients in real time, for example.
Why don't you simply use FTPS:
http://en.wikipedia.org/wiki/FTPS
or SFTP
http://en.wikipedia.org/wiki/SSH_file_transfer_protocol
I have just started writing socket programs. Came to know that single UDP packet has source port destination port and some MAC address representing router..etc. I wonder why anybody cannot create custom packets with a fake information in and send it over internet. I would like to know how safe are our PCs. What should be done to secure it ?
There are a couple of different aspects to the answer.
One is that the web relies on TCP, not UDP. Which means that it is connection-oriented. Your package will be rejected, unless it appears to be part of an existing connection (which means, among other things, that it has to have the right source IP and port as well. And it has to have the right sequence number to fit into the receive window). This can still be faked without too much trouble, of course. But it does require you to know a bit about the packets being sent on the original connection.
Another part is that whenever we need to be sure that the sender of a packet is who they claim to be, we use encryption. :)
Most packets don't really need this. It's not a huge deal if someone sends a request to Google which appears to come from my IP. But when making credit card transactions, it becomes a bit more important.
Most of the TCP/IP stack "leaks trust", as I once put it -- and there isn't much that you, as a software developer (assuming you're looking for a programming solution, otherwise, stackoverflow's the wrong forum, go to serverfault or superuser;-) can do about it -- beyond choosing and carefully implemented protocols that are reasonable in terms of security expectation.
HTTPS (with strong checks of certificates, etc) is one reasonably strong approach; for stronger security, look into SSH and VPN-based approaches. Of course, nobody should assume privacy or strong authentication is in place unless they've taken specific steps towards it (if they HAVE taken such steps, they may be still subject to successful attacks, which is why using existing, more or less "proven" solutions such as HTTPS, SSH, VPNs, is advisable;-).
Yes, anyone can create packets with whatever data they want and send them out over the internet. Especially with UDP, you can pretend to be anyone you want (unless your ISP does egress filtering). Source addresses for UDP cannot be trusted. Source addresses for TCP can to an extent (you know the data has to be coming from the IP address in question, or someone along the route).
Welcome to the internet :)
Edit: just to clarify egress filtering is something the sending ISP would have to do. As a reciever, there's not really anything you can do to verify the address on a UDP packet without communicating back to the sender. The only reason you can at least partially trust an incoming TCP connection is that TCP requires certain control data flow back to the sender (and hence needs a valid IP address/port to set the connection up and maintain it).
Well, many many people create invalid packets and send them over Internet; for instance, read Ping of death.
A [completly] secure computer is a computer turned off. To make your running PC more secure from this thread kind, you should rely on firewall softwares/hardwares, which can detect that malformed packets.
Custom packets with fake information can easily be created. Therefore you have to make sure you're not vulnerable to them.