I have just started writing socket programs. Came to know that single UDP packet has source port destination port and some MAC address representing router..etc. I wonder why anybody cannot create custom packets with a fake information in and send it over internet. I would like to know how safe are our PCs. What should be done to secure it ?
There are a couple of different aspects to the answer.
One is that the web relies on TCP, not UDP. Which means that it is connection-oriented. Your package will be rejected, unless it appears to be part of an existing connection (which means, among other things, that it has to have the right source IP and port as well. And it has to have the right sequence number to fit into the receive window). This can still be faked without too much trouble, of course. But it does require you to know a bit about the packets being sent on the original connection.
Another part is that whenever we need to be sure that the sender of a packet is who they claim to be, we use encryption. :)
Most packets don't really need this. It's not a huge deal if someone sends a request to Google which appears to come from my IP. But when making credit card transactions, it becomes a bit more important.
Most of the TCP/IP stack "leaks trust", as I once put it -- and there isn't much that you, as a software developer (assuming you're looking for a programming solution, otherwise, stackoverflow's the wrong forum, go to serverfault or superuser;-) can do about it -- beyond choosing and carefully implemented protocols that are reasonable in terms of security expectation.
HTTPS (with strong checks of certificates, etc) is one reasonably strong approach; for stronger security, look into SSH and VPN-based approaches. Of course, nobody should assume privacy or strong authentication is in place unless they've taken specific steps towards it (if they HAVE taken such steps, they may be still subject to successful attacks, which is why using existing, more or less "proven" solutions such as HTTPS, SSH, VPNs, is advisable;-).
Yes, anyone can create packets with whatever data they want and send them out over the internet. Especially with UDP, you can pretend to be anyone you want (unless your ISP does egress filtering). Source addresses for UDP cannot be trusted. Source addresses for TCP can to an extent (you know the data has to be coming from the IP address in question, or someone along the route).
Welcome to the internet :)
Edit: just to clarify egress filtering is something the sending ISP would have to do. As a reciever, there's not really anything you can do to verify the address on a UDP packet without communicating back to the sender. The only reason you can at least partially trust an incoming TCP connection is that TCP requires certain control data flow back to the sender (and hence needs a valid IP address/port to set the connection up and maintain it).
Well, many many people create invalid packets and send them over Internet; for instance, read Ping of death.
A [completly] secure computer is a computer turned off. To make your running PC more secure from this thread kind, you should rely on firewall softwares/hardwares, which can detect that malformed packets.
Custom packets with fake information can easily be created. Therefore you have to make sure you're not vulnerable to them.
Related
My knowledge about network programming is limited, so, all the comments are more than welcome. Essentially my question boils down to the following question:
Q1. Is there really such a thing as decentralized asynchronous cross-platform peer-to-peer communication?
Let me explain myself.
If we have two http servers running on computers with actual IP addresses, then clearly the answer is yes, assuming one writes a protocol for the interaction.
To go one step further, if one of them (or both) is (are) behind a router, then, with port forwarding the communication can still be established. However, here the problems start because if someone wants to run such a server on the background, say in a mobile phone, the app that is relying on this server really works when one is at home (we can not really expect to request port forwarding everywhere we go).
But even beyond that,
Q2. do mobile phones obtain an actual IP address from telecommunication companies when someone is not using a wi-fi?
If this is true, then clearly one can have cross-platform asynchronous peer-to-peer communication at the expense of not using wi-fi by running an http server on a smartphone. (I understand that this is not convenient, but it is certainly doable.)
Concluding, the two (perhaps there are more) relevant questions that I can think of are:
Q3. How does Skype really work?
Q4. How does Viber really work?
Based on the answer for Skype, it says: If one of the callee or both of them do not have a public IP, then they send voice traffic to another online Skype node over UDP or TCP.
So, it appears that there is no direct communication in Skype, because they have to use a man-in-the-middle for such a scenario.
Regarding Viber, I could not find a good-thorough answer to this particular question. Do people talk to each other through a Viber centralized server, or, do they establish a direct connection? Of course if they do establish a direct connection, then I really want to know how they manage such a thing since a mobile phone may or may not have a physical address. How is a Viber message routed to my cell phone from a friend of mine even when Viber is not running and I am behind a router?
I guess the answer to Viber is really push notifications, but as far as I can understand, all the variations of push notifications rely on open connections, and then the servers of the applications send the notifications to the clients through such connection(s). So, this approach gives us the feeling that it is asynchronous, but essentially it is not. We are cheating, in the sense that there is a constantly open connection to a server, and moreover, as far as I can understand, the application server has to push the notification through that server. Schematically:
A > Central App Server > Central Server w/ open connection to my cellphone > me
So, this seems to be once again a centralized approach.
Honestly, the only approach that I can think of that is both decentralized and asynchronous (on mobile phones as well) is to run an http server on every platform/device, but this comes at the expense of not using Wi-Fi and assuming that a telecommunication company really assigns a physical IP address to every mobile phone (which I do not know if it is true, do you?).
What about WASTE, darknets, F2Fs, etc? Do they offer advantages in the sense of a more direct asynchronous communication between some interested parties? Are there real-world applications (also including mobile phones) using such approaches for communication.
Really, this is not the actual problem that I would like to work on, but I would like to know what the state of the art is so that I can figure out how I can proceed from there. So, all comments are really more than welcome. If you have references for the state of the art I would like to know about them as well, but a brief description would also be nice.
I appreciate all your time and effort in advance.
You asked many questions, here is the beginning of the answers:
Q1: Yes. For example, take BitTorrent's very successful 10 million+ node network. Aside from the bootstrapping process, the protocol is entirely decentralized and asynchronous. See here for more info.
Q2: Yes! Go to www.whatismyip.com on your mobile telephone, and you will see your assigned IP. However, you are likely to be very filtered (e.g: incoming traffic on port 80 is likely to be blocked).
Q3: It has elements of P2P and clever tricks to get around NAT issues - see here for more info.
Q4: I don't know.
I'm experimenting with my Arduino Mega. I also have an Arduino Ethernet Shield.
I need to send emails using them, without the help of a computer (or any other device; like a smartphone, etc.). Though I could find several articles, I couldn't find any acceptable solution...
How can I do it? As I'm not asking this to be used for any special application, you can
make any assumption about missing details.
From the discussion above in comments it sounds like you either need code from someone who has just done it for you or you need to take the time to learn about the components and find or make the components.
They wouldn't make an Ethernet shield for this platform if it was only useful for non-standard packets. So someone somewhere has created some level of an IP stack.
Backing up though, in order to send mail you need to learn the Simple Mail Transfer Protocol (SMTP). Almost all Internet protocol definitions are defined using something called RFCs (Request for Comments). So if you google SMTP RFC you will find RFC 2821.
IETF is Internet engineering task force. There will be many copies of these documents on many websites. And due to the age of the Internet and these protocols in many cases you will find that one RFC has been created to replace a prior one. Version numbers are not used, but it is kind of like HTML 1.0 then HTML 2.0 and so on. I recommend even though the RFC says that it completely replaces RFC xyz, go find RFC xyz and read it. I go back as far as I can find learn that one then work my way forward.
Many/most protocols that ride on top of TCP (TCP is yet another protocol defined in an RFC, more on that later) are ASCII based, makes it very easy to, for example, Telnet to learn/experiment with the protocol, you can probably use Telnet to learn SMTP.
Most protocols are some sort of a half duplex thing, make a connection and often the server sends you a string, you see that string and then you send some sort of hello string, the server responds with some sort of OKAY or fail status. For SMTP, you then do some sort of I am mailing from this email address, server says OKAY, you say I want to mail this person or this list of people, for each email address you get an okay or fail. Eventually, you tell the server you are ready to send the body of the message, you do that, end the message with the defined termination. Then either the server says okay or fail or maybe there is some more handshaking.
The protocols in general though have this back and forth. Usually you are sending strings with commands and usually the server side sends back a short okay or error. Sometimes, if they want, they send back more detail on the error, but always start with the few bytes that indicate okay or error. The protocols generally have a flow, you must do this first then this then that.
You should learn sockets programming, sometimes called Berkeley sockets. You can write programs that are mostly portable across unixes but also across to Windows using Windows sockets if that is your platform of choice. You need to learn the protocol first, and it is better on your desktop/laptop and not embedded, you can get it done faster there. You do NOT have to learn to fork or thread to use sockets. The examples may show that as it is easy to show it that way, but you can write complete applications using polling only, it is half duplex send something, wait, send something, wait. For these simple learning programs, a little time up front to learn sockets, from there, it is all learning the protocols.
Now that was the very easy part, the hard part is the TCP/IP stack. I do not recommend attempting that without gaining a lot more experience taking baby steps on your way there. For example, learn to respond to ARP first (yet another RFC protocol, address resolution protocol) then ping (ICMP echo, one subset of the ICMP protocols) then IP basics (sniffing packets) then receive and generate UDP packets. TCP is a whole other level above that, more handshaking. It is not fixed packet size, it is streaming, do not have your code operate on packets, it is a stream of bytes, like working with a serial port.
Doing your own TCP stack is very much a non-trivial thing, I don't recommend it, you need to find someone that has done a TCP/IP stack for this platform for the Ethernet shield and just use it, whatever RTOS or environment they use, use it. Then take your desktop/laptop based experience with the protocol and apply that.
From the discussion above, if you don't want to learn the protocols, etc., I think you need to google around looking at Arduino Ethernet shield examples and see if anyone has done something that sends emails.
We're implementing a SIP-based solution and have configured the setup to work with RTPProxy. Right now, we're routing everything through RTPProxy as we were having some issues with media transport relying on ICE. If we're not mistaken, a central relay server is necessary for relaying streaming data between two clients if they're behind symmetric NATs. In practice, is this a large percentage of all consumer users? How much bandwidth woudl we save if we implemented proper routing to skip the relay server when not necessary. Are there better solutions we're missing?
In falling order of usefulness:
There is a direct connection between the two endpoints in both directions. You just connect and you are essentially done.
There is a direct connection between the two endpoints in one direction. In that case you just connect via the right direction by trying both.
Both parties are behind NATs of some kind.
Luckily, UPnP works in one end, you can then upgrade the connection to the above scheme
UPnP doesn't work, but STUN does. Use it to punch a hole in the NAT. There are a couple of different protocols but the general trick is to negotiate via a middle man that coordinates the NAT-piercing.
You fall back to let another node on the network act as a relaying proxy.
If you implement the full list above, then you have to give up very few connections and don't have to spend much time on bandwidth utilization at proxies. The BitTorrent protocol, of which I am somewhat familiar, usually stops at UPnP, but provides a built-in test to test for connectivity through the NAT.
One really wonders why IPv6 did not get implemented earlier - this is a waste of programmers time.
Real world NAT types survey (not a huge dataset, though):
http://nattest.net.in.tum.de/results.php
According to Google, about 8% of the traffic has to be relayed: http://code.google.com/apis/talk/libjingle/important_concepts.html
A large percentage (if not the majority) of home users uses NAT, as that is what those xDSL/cable routers use to provide network access to the local network.
You can theoretically use UPnP to open ports and set-up forwarding rules on the router to go through the NAT transparently. Unfortunately (or fortunately, depending on who you are) many users disable UPnP as a matter of course on their router and may not appreciate having to add forwarding rules manually.
What you might be able to do (and what Skype does AFAIK) is to have some of the users that have clear network paths and enough bandwidth act as relay nodes. Apart from the routing and QoS issues, you would at least have to find some way to ensure the privacy of any relayed data from anyone, including the owner of the relay node. In addition, there might be legal issues to settle with this approach, apart from the technical ones.
I've seen and read a lot of similar questions, and the corresponding Wikipedia articles (NAT traversal, STUN, TURN, TCP hole punching), but the overwhelming amount of information doesn't really help me with my very simple problem:
I'm writing a P2P application, and I want two users of my application behind NAT to be able to connect to each other. The connection must be reliable (comparable to TCP's reliability) so I can't just switch to UDP. The solution should work on today's common systems without reconfiguration. If it helps, the solution may involve a connectible 3rd-party, as long as it doesn't have to proxy the entire data (for example, to get the peers' external (WAN) IP addresses).
As far as I know, my only option is to use a "reliable UDP" library + UDP hole punching. Is there a (C/C++) library for this? I found enet in a related question, but it only takes care of the first half of the solution.
Anything else? Things I've looked at:
Teredo tunnelling - requires support from the operating system and/or user configuration
UPnP port forwarding - UPnP isn't present/enabled everywhere
TCP hole punching seems to be experimental and only work in certain circumstances
SCTP is even less supported than IPv6. SCTP over UDP is just fancy reliable UDP (see above)
RUDP - nearly no mainstream support
From what I could understand of STUN, STUNT, TURN and ICE, none of them would help me here.
ICE collects a list of candidate IP/port targets to which to connect. Each peer collects these, and then each runs a connectivity check on each of the candidates in order, until either a check passes or a check fails.
When Alice tries to connect to Bob, she somehow gets a list of possible ways - determined by Bob - she may connect to Bob. ICE calls these candidates. Bob might say, for example: "my local socket's 192.168.1.1:1024/udp, my external NAT binding (found through STUN) is 196.25.1.1:4454/udp, and you can invoke a media relay (a middlebox) at 1.2.3.4:6675/udp". Bob puts that in an SDP packet (a description of these various candidates), and sends that to Alice in some way. (In SIP, the original use case for ICE, the SDP's carried in a SIP INVITE/200/ACK exchange, setting up a SIP session.)
ICE is pluggable, and you can configure the precise nature/number of candidates. You could try a direct link, followed by asking a STUN server for a binding (this punches a hole in your NAT, and tells you the external IP/port of that hole, which you put into your session description), and falling back on asking a TURN server to relay your data.
One downside to ICE is that your peers exchange SDP descriptions, which you may or may not like. Another is that TCP support's still in draft form, which may or may not be a problem for you. [UPDATE: ICE is now officially RFC 6544.]
Games often use UDP, because old data is useless. (This is why RTP usually runs over UDP.) Some P2P applications often use middleboxes or networks of middleboxes.
IRC uses a network of middleboxes: IRC servers form networks, and clients connect to a near server. Messages from one client to another may travel through the network of servers.
Failing all that, you could take a look at BitTorrent's architecture and see how they handle the NAT problem. As CodeShadow points out in the comments below, BitTorrent relies on reachable peers in the network: in a sense some peers form a network of middleboxes. If those middleboxes could act as relays, you'd have an IRC-like architecture, but one that's set up dynamically.
I recommend libjingle as it is used by some major video game companies which heavily relies on P2P network communication. (Have you heard about Steam? Vavle also uses libjingle , see the "Peer-to-peer networking" session in the page: https://partner.steamgames.com/documentation/api)
However, the always-work-solution would be using a relay server. Since there is no "standard" way to go through NAT, you should have this relay server option as a fall-back strategy if a connection has to be always established between any peers.
How can I detect if another host is using the same MAC address as the current host, e.g. because the other host is spoofing?
I'm working in an embedded environment, so looking for answers on a protocol level, rather than “use such and such a tool”.
Edit: RARP does not solve this problem. For RARP to get any reply at all, there has to be at least one host on the segment which supports RARP. Since RARP is obsolete, modern operating systems don't support it. Furthermore, all RARP can do is tell you your own IP address - the response won't be any different if there’s another host on the segment with the same MAC, unless that host has itself used a different IP address.
This question is too interesting to put down! After several false starts I started thinking about the essential components of the problem and scoured the RFCs for advice. I haven't found a definitive answer, but here's my thought process, in the hope that it helps:
The original question asks how to detect another device with your MAC address. Assuming you're on an IP network, what's required to accomplish this?
The passive method would be simply to listen to traffic and look for any packets that you didn't transmit but have your MAC address. This may or may not occur, so although it can tell you definitively if a duplicate exists, it cannot tell you definitively that it doesn't.
Any active method requires you to transmit a packet that forces an impostor to respond. This immediately eliminates any methods that depend on optional protocols.
If another device is spoofing you, it must (by definition) respond to packets with your MAC address as the destination. Otherwise it's snooping but not spoofing.
The solution should be independent of IP address and involve only the MAC address.
So the answer, it seems, would be to transmit either a broadcast (ethernet) packet or a packet with your MAC address as its destination, that requires a response. The monkeywrench is that an IP address is usually involved, and you don't know it.
What sort of protocol fits this description?
Easy Answer:
If your network supports BOOTP or DHCP, you're done, because this authoritatively binds a MAC address to an IP address. Send a BOOTP request, get an IP address, and try to talk to it. You may need to be creative to force the packet onto the wire and prevent yourself from responding (I'm thinking judicious use of iptables and NAT).
Not-so-easy Answers:
A protocol that's independent of IP: either one that doesn't use the IP layer, or one that allows broadcasts. None comes to mind.
Send any packet that would normally generate a response from you, prevent yourself from responding, and look for a response from another device. It would seem sensible to use your IP address as the destination, but I'm not convinced of that. Unfortunately, the details (and, therefore the answer) are left as an exercise for the OP ... but I hope the discussion was helpful.
I suspect the final solution will involve a combination of techniques, as no single approach seems to guarantee a dependable determination.
Some information is available at http://en.wikipedia.org/wiki/ARP_spoofing#Defenses
If all else fails, you may enjoy this: http://www.rfc-editor.org/rfc/rfc2321.txt
Please post a follow-up with your solution, as I'm sure it will be helpful to others. Good luck!
You could send an ARP request for each possible ip in the subnet.
Of course the source address of the ARP request must be ff:ff:ff:ff:ff:ff, otherwise you might not see the response.
I forged a packet like this with bittwiste and replayed it with PReplay and all the hosts on the network got the response. (I don't know if these forged ARP packets are legal or not... some OSes might ignore them)
Here is what the forged package looked like:
Here is what the reply looked like:
If you watch the responses and see your MAC address in one of the packets (in the red rectangle) , than someone has the same MAC address as you do...
Unfortuantely I couldn't test the theory fully because none of my (Windows) machines care about me trying to set the nic's MAC address...
Two hosts using same MAC address on a single network segment would probably make switches go nuts and you could probably detect it by having an extremely unreliable network connection (as the switches would send some portion of packets that belong to your host to the second one, depending on which one of you sent the last packet in their direction).
This is very late, and a non-answer, but I wanted to follow up with exactly what I did in case anyone else is interested.
I was working with some very weird embedded hardware that doesn’t have a MAC address assigned at manufacture. That means we needed to assign one in software.
The obvious solution is to have the user pick a MAC address that they know is available on their network, preferably from the locally-administered range, and that’s what I did. However, I wanted to pick a reasonably safe default, and also attempt to warn the user if a conflict occurred.
In the end I resorted to picking a random-ish default in the locally-administered range, chosen by making some hardware readings that have moderate entropy. I deliberately excluded the beginning and end of the range on the assumption that those are moderately more likely to be chosen manually. The chances are that there will only be one of these devices on any given network, and certainly less than 20, so the chances of a conflict are very low, albeit not as low as they could be due to the somewhat predictable random numbers.
Given the low chances of there being a problem, and despite the excellent answers above, I decided to dispense with the conflict detection and make do with a warning to the user to look out for MAC conflict problems.
If I did decide to implement conflict detection, then given that I control the whole network stack, I would probably look out for excessive unknown or missing packets, and then trigger a change of MAC address or warn the user when that happens.
Hopefully that will help someone else somewhere – but probably not!