I have a question concerning tcp packet sniffing with golang.
We have written a small tool which captures all incoming TCP packets comming from a fibre Tap.
The current implementation uses a libpcap wrapper under Linux. We need to port this tool to Windows. Of course, it is not possible at the moment.
So my question is, is there a cross platform solution for sniffing packets? We only need TCP packets, IP headers , no Ethernet Data and not all features of libpcap.
If there is no cross platform solution, two Code implementation would be ok, too. I know one can use raw sockets under Linux (and with some limitations under Windows). Does Golang support raw sockets and is there an example implementation for sniffing packets with sockets?
Tanks!! :-)
You should be able to use the ipv4 package from go.net.
Package ipv4 implements IP-level socket options for the Internet Protocol version 4.
The ipv4.RawConn type and it's associated methods should work cross-platform.
A RawConn represents a packet network endpoint that uses the IPv4 transport. It is used to control several IP-level socket options including IPv4 header manipulation. It also provides datagram based network I/O methods specific to the IPv4 and higher layer protocols that handle IPv4 datagram directly such as OSPF, GRE.
There is also an equivalent package for ipv6.
Take a look at https://code.google.com/p/gopacket/ since it supports pcap (requires cgo for this) and can decode a number of protocols including tcp/ip.
Related
As I understand it TCP is required for congestion control and error recovery or reliable delivery of information from one node to another and its not the fastest of protocols for delivering information.
Some routing protocols such as EIGRP and OSPF ride directly on top of IP. Even ICMP rides directly over IP.
Why is UDP even required at all? Is it only required so that developers/programmers can identify what application the inbound packet should be sent to based on the destination port number contained within the packet?
If that is the case then how is information gathered from protocols that ride directly on top of IP sent to the appropriate process when there is no port number information present?
Why are voice and video sent over UDP? Why not directly over IP?
(Note that I do understand thoroughly the use case for TCP. I am not asking why use UDP over TCP or vice versa. I am asking why use UDP at all and how can some protocols use directly the IP layer. Whats the added advantage or purpose of UDP over IP?)
Your question makes more sense in terms of why is UDP useful (than why is UDP required).
UDP is a recognized protocol by the Internet Assigned Numbers Authority. UDP can be useful if you want to write a network protocol that's datagram based and you want to play more nicely with Internet devices.
Routers can have rules to do things like drop any packet that doesn't make sense. So if you try and send packets using say an unassigned IP protocol number between hosts separated by one or more routers, the packets may well never get delivered as you've intended. The same could happen with packets from an unrecognized UDP protocol but that's at least one less door to worry about whether your packet can make it through.
Internet endpoints (like hosts) may do similar filtering too. If you want to write your own datagram based protocol and use a typical host operating system, you're more likely to need to write your software as a privileged process if not as a kernel extension if you're trying to ride it as its own IP protocol (than if you'll be using UDP).
Hope this answer is useful!
First of all, IP and UDP are protocols on the different layers, IP by definition is Internet layer when UDP is transport layer. Layers were introduced to simplify network protocols architecture and to separate concerns. Application layer protocols are supposed to be based on transport layer (with some exceptions).
Most popular transport protocols (in IP network) are UDP and TCP. While TCP is feature rich but with many tradeoffs UDP is very simple but gives a lot of freedom and so typically is a base for other protocols.
The main feature of UDP is multiplexing: ports that allow multiple protocol instances (aka sockets) to coexist on the same node. This means that implementing your own protocol over IP instead of UDP either you won't be able to have multiple instances of your protocol on the same machine or you'll have to implement multiplexing yourself.
There're other features like segmentation and checksum. These features are not mandatory.
And as was mentioned in another answer there're lots of middleware like routers, NATs and firewalls that can ruin the idea of a custom "right over IP" protocol, but it's more like a collateral damage than a feature of UDP.
I looking for an example of SNMP running in a protocol different than UDP. I need to argue with a professor who said that it's only possible to run SNMP over UDP. Anyone knows how it works in ATM? In my mind SNMP is a layer 7 protocol and doesn't matter what protocol is used for transport, but I only found references to UDP. Please post the references.
SNMP of course can go over other protocols than UDP. For example, RFC 3430 defines SNMP over TCP,
https://www.rfc-editor.org/rfc/rfc3430
However, the widely used SNMP implementation is still UDP only in most cases, so rarely you see an application on TCP or other protocols (I knew some internal usage in Cisco).
Well, an argument is not really suggested, and hope you chat in a good manner with your professor.
From RFC 1157 'A Simple Network Management Protocol (SNMP)' #4:
Protocol Specification
The network management protocol is an application protocol by which
the variables of an agent's MIB may be inspected or altered.
Communication among protocol entities is accomplished by the exchange
of messages, each of which is entirely and independently represented
within a single UDP datagram using the basic encoding rules of ASN.1
(as discussed in Section 3.2.2). A message consists of a version
identifier, an SNMP community name, and a protocol data unit (PDU).
A protocol entity receives messages at UDP port 161 on the host with
which it is associated for all messages except for those which report
traps (i.e., all messages except those which contain the Trap-PDU).
Messages which report traps should be received on UDP port 162 for
further processing.
I had a strange idea. I heard of software which from my understanding uses UDP to transfer files decreasing the overhead found in TCP packets.
If my app requires TCP and my LAN has software set up to communicate with another datacenter on the other side of the coast with software setup on their end. Would it be possible to send the actual data via UDP but than simulating TCP on both ends?
Does anyone have any ideas or information about such projects?
If you're asking if you can use UDP as a Layer 2, then the answer is yes, sort of. There are various protocols that allow you to create a tunnel to another network using a UDP transport, such as L2TP and even IPsec (with NAT traversal). You could also do it at the application layer.
If you're asking if TCP can be implemented in UDP, the answer is no. First, TCP packets and UDP packets have an incompatible format. Second, TCP and UDP have different protocol numbers (seen in the IP header) which means that TCP traffic destined for a UDP port would not be passed to the correct upper-layer protocol.
Both TCP and UDP are built on top of the IP, but the TCP uses different packet structure and at the layer-2 it is not possible to mimic the TCP using UDP packets.
Of course, if you have the control on both the source and destination, then it is possible to create a reliable UDP tunnel for the TCP packets. This would require some internal information (packet number, ack/nack flags) in the body of the UDP packet.
There is an interesting project http://udt.sourceforge.net/
It is a broadcast-capable reliable file transfer mechanism built on top the UDP.
PseudoTCP is a protocol which implements TCP algorithms on top of the UDP. It was introduced since the NAT traversal for TCP is much more complicated than UDP. But some P2P applications do need a reliable data transfer among nodes.
So far as I know, there are two PseudoTCP variations: Libjingle and Libnice.Libjingle is an open source library from google which was initially for gtalk. You could take a look at file sharing example from libjingle: https://developers.google.com/talk/libjingle/file_share. Recently, Chrome desktop also use PseudoTCP implementation from libjingle for reliable connections.
Yes, you can develop a protocol on UDP that simulates TCP. However, if you simulated TCP fully, it would technically have more overhead. Because TCP is implement as the packet and your simulated TCP is implemented in the body of the packet.
If you only need one or two features of TCP (such as basic ordering), then implementing it in UDP is useful.
Halo uses 2-3 (IIRC) UDP protocols that simulate different features of TCP, then full fledged TCP for initializing game-states. I Shot You First Networking, GDC publication
For example, in one case, they send 3 duplicate UDP packets to overcome packet loss.
If you control the software on both ends, and it is cost-effective to build your own protocol, then UDP can be versatile.
One way to do it now on Linux-3.18+ is to use Foo over UDP (FOU) which implements Generic UDP Encapsulation (GUE). Here's a good introduction to FOU, and the man page for ip-fou.
Or if you want an [open source] UDP based file transfer system there are things like UDT, UFTP, Tsunami-UDP, and even Google's QUIC (Now deprecated in favour of IETF QUIC).
Update: The QUIC protocol now has been standardised by the IETF which provides for secure reliable and unreliable transport over UDP as an alternative to TCP. There's a wide range of QUIC implementations available. There is also a growing set of protocol mappings on to QUIC such as HTTP/3, DNS over QUIC, etc
If my app requires TCP and my LAN has software setup to communicate
with another datacenter on the other side of the coast with software
setup on their end. Would it be possible to send the actual data via
UDP but than simulating TCP on both ends?
No. A UDP socket is in a different namespace from a TCP socket. You will be unable to write UDP at one end and send or receive TCP at the other end. TCP and UDP are peer protocols; both exist at the layer above IP. You can't use one to spoof the other.
Hmm, I believe so. You'd need to use a proxy at both ends, but it should be possible.
The biggest problem you are going to run into is that UDP is designed with the idea that you don't care if some of the packets don't ever make it to the other end.
Here's a link with some more info:
http://www.cyberciti.biz/faq/key-differences-between-tcp-and-udp-protocols/
IMHO, it's not a good idea to transmit files via UDP.
TCP's problems are in its algorithms, not its headers.
You certainly could implement the TCP algorithms on top of UDP. That would effectively be the same as tunneling TCP datagrams inside of UDP datagrams. But all this accomplishes is to add a few more bytes of overhead to each packet, and require another endpoint to unwrap the packets.
UDP itself is just thin shim on top of IP: its a convenient way to access IP packet switched networking without having to dive into kernels or receive special handling from routers. The main reason to implement reliable transport on top of UDP is to get away from TCP algorithms in favor of something more efficient. FileCatalyst was mentioned above as one company which does this, and my own company Data Expedition, Inc. does so as well.
So you could implement TCP algorithms on top of UDP, but you wouldn't want to.
You can simulate something like a connection over UDP, and you as well can add reliability checks and ordering and retransmission and so on. - but then, it still isn't TCP, it just acts the way.
Of course, one of the ends can be a kind of "hub" or "proxy" which does an adaption. Then you don't have a 2-end solution, but in fact a 4 end solution - one pair with "real" TCP and the other with the "self-knitted" "TCP" - which you put together with an appropriately crafted program.
Are there any networking protocols that are not strictly TCP or UDP but can be used with either one?
For example, HTTP, FTP, STMP, RTMP are always TCP.
DNS, SNMP, DHCP, RIP are always UDP.
Is there anything that can be either TCP or UDP? Or am I wrong in the above assertions?
RTSP is one weird one I know of that uses both, TCP for the control port but UDP for audio/video/quality, but it has strict requirements of what gets sent of each.
I'm asking about standard, published, or at least commonly used protocols, not custom ones.
DNS can use either UDP or TCP; TCP is required when the response data exceeds 512 bytes.
If you examine a Windows' services file you will see a number of protocols registered for both TCP and UDP. Path: C:\Windows\System32\drivers\etc In fact, most of the listings in the services file use both TCP and UDP protocols.
As far as well known apps that use both, I would think that most chat applications use both. sms-chat definitely does but probably most others.
Edit:
From that file, here's a few of the protocols that can be sent over either TCP or UDP (there are exactly 100 listed protocols that use both in the file, many internal MS protocols):
echo
discard
daytime
qotd (Quote of the day)
chargen (Character generator)
time
SIP can use UDP, TCP or SCTP. Using a reliable transport becomes important in SIP if your messages get to be at all large (i.e., significantly larger than the smallest MTU in between user agents). A good example is shared- or bridged-line appearances, which use a form of presence with XML bodies. The larger the number of SIP clients in the shared-line group, the larger the packets are likely to be, making fragmentation and retransmission an issue.
SIP can be either UDP or TCP. However, the reality is that UDP is mostly used for this protocol.
SNMP almost always runs over UDP, but it can and does run over TCP. Theory says that it's a bad idea to do SNMP over an error-correcting transport because because some of the very errors that SNMP intends to detect are masked.
Could i inject packets to Linux TCP stack without modifying the ethernet driver? Could i do this with using a library or sth ?
Thank you,
If by 'inject packets to Linux TCP stack' you mean send some data that the Linux kernel will treat as a frame coming from an Ethernet interface then you can use a 'tap' device. If an IP packet (layer 3) is good enough, then use a 'tun' device.
http://en.wikipedia.org/wiki/TUN/TAP
http://www.kernel.org/pub/linux/kernel/people/marcelo/linux-2.4/Documentation/networking/tuntap.txt
Libnet
Libnet is a generic networking API that provides access to several protocols. It is not designed as a 'all in one' solution to networking. Currently many features that are common in some network protocols are not available with Libnet, such as streaming via TCP/IP. We feel that Libnet should not provide specific features that are possible in other protocols. If we restrict Libnet to the minimal needed to communicate (datagram/packets) then this allows it to support more interfaces.
Otherwise, if you're just wondering about injecting hand-crafted packets into the network, read the man pages and look for online help with raw sockets. Some good places to start are man 7 raw, man packet, and there are some ok tutorials at security-freak.net, though the code there is not written particularly well for my tastes.