I'm testing how windows handle IPv4 ID. I need to generate 10,000 TCP or UDP packets per second with different source IPs(my netmask is 255.255.0.0, so there are 2^16 IPs available. But it's not the case here since I can do IP spoofing). I know that I can change the count parameter in scapy.sendrecv.send to generate a large number of packets at the same time but the configuration of the packets is all the same. Moreover, I also want occasionally to pick out some responses to check the status.
I'm currently thinking about using multithreaded but I'm not sure how to do that. Can anyone give me a structure to start with?
Related
I understand that it's different than a hub in that instead of packets being broadcasted to all devices connected to the device, it knows exactly who requested the packet by looking at the MAC layer.
However, is it still possible to use a packet sniffer like Wireshark to intercept packets meant for other users of the switch? Or is this only a problem with ethernet hubs that doesn't affect switches due to the nature of how a switch works?
On a slightly off topic side note, what exactly is classified as a LAN? For example, imagine two separate ethernet switches are hooked up to a router. Would each switch be considered a separate LAN? What is the significance of having multiple LAN's within the same network?
it knows exactly who requested the packet by looking at the MAC layer.
More exactly, the switch uses the MAC destination address to forward a frame to the port associated with that address. Addresses are automatically learned by looking at the MAC source address on received frames.
A switch is stateless, ie. is has no memory who requested which data. A layer-2 switch also has no understanding of IP packets, addresses or protocols. All a basic switch does is learn source addresses and forward by destination address.
is it still possible to use a packet sniffer like Wireshark to intercept packets meant for other users of the switch?
Yes. You'll need a managed switch supporting port mirroring or SPANning. This doesn't intercept frames, it just copies them to the mirror port. If you need to actually intercept frames you have to put your interceptor in between the nodes (physically or logically).
With a repeater hub, every bit is repeated to every node in the collision domain, making monitoring effortless.
what exactly is classified as a LAN?
This depends on who you ask and on the context. A LAN can be a layer-1 segment/bus aka collision domain (obsolete), a layer-2 segment (broadcast domain), a layer-3 subnet (mostly identical with an L2 segment) or a complete local network installation (when contrasted with SAN or WAN).
Adding to #Zac67:
Regarding this question:
is it still possible to use a packet sniffer like Wireshark to
intercept packets meant for other users of the switch?
There are also active ways in which you can trick the Switch into sending you data that is meant for other machines. By exploiting the Switch's mechanism, one can send a frame with a spoofed source MAC, and then the Switch will transfer frames destined to this MAC - to the sender's port (until someone else sends a frame with that MAC address).
This video discusses this in detail:
https://www.youtube.com/watch?v=YVcBShtWFmo&list=PL9lx0DXCC4BMS7dB7vsrKI5wzFyVIk2Kg&index=18
In general, I recommend the following video that explains this in detail and in a visual way:
https://www.youtube.com/watch?v=Youk8eUjkgQ&list=PL9lx0DXCC4BMS7dB7vsrKI5wzFyVIk2Kg&index=17
what exactly is classified as a LAN?
So indeed this is one of the least-well-defined terms in Computer Networks. With regards to the Data Link Layer, a LAN can be defined as a segment, that is - a broadcast domain. In this case, two devices are regarded as part of the same segment iff they are one hop away from one another - that is, they can switch frames in the second layer.
I am writing a thesis concerning the identification of patterns in a network traffic. The input file contains thousands of data lines, each providing information as timestamps, source and destination IP addresses, source and destination ports, interfaces, number pf bytes and packets being exchanged between the source and the destination and protocols. The start and end-time are always the same in a data line.
My question is if there is possible to assign all IP addresses to categories such as routers/servers/clients only based on the info provided or if there are also other info necessary in order to assign all addresses correctly? (the ports used are about 100-150 and are both registered and unregistered).
Thank you!
Your question is very broad because it depends a lot on what categories you have in mind. For example, what's your definition of a server? Anyway, technically NetFlow does not support any kind of endpoint type qualification so you have to rely on statistics. If a certain destination IP address has a significant (absolute) amount of traffic to for example (destination) port 25 it would likely be an SMTP server. And the sender can perhaps be categorized as a client unless it also received a lot of SMTP traffic (so it would be relaying). Since NetFlow usually runs on routers (and less frequently on switches) your NetFlow origin IP address is likely a router. Large amounts of traffic to or from an IP address on a specific port will likely denominate that IP address as a server. You have to determine the boundaries for that. And - if needed - the type of server. SMTP could also run an a non-standard port (e.g. 80), less likely but you could possibly detect that by measuring the amount of ingress vs egress data. My guess would be that several standard protocols have identifiable ratios on this.
I am trying to prioritize TCP traffic using ToS field in IP header.
I am saturating the interface(ethernet) by sending 1GB data through iperf with ToS field set to 0x10 (Minimize-Delay).
I then start another TCP client with default ToS (0).
Expectation :
My TCP client should not send data till iperf completes sending its data.
Result:
The data from my client is sent even tough iperf is sending packets with higher priority.
I also tried to create the same scenario by creating 2 separate clients and allocating 0x10 and 0x08 ToS to respective clients using iptables.
I used :
iptables -A PREROUTING -t mangle -p tcp --sport 5000 -j TOS --set-tos Minimize-Delay
I am still not able to prioritize one client over other.
Altough I can see the packets marked with ToS in wireshark.
I am using Ubuntu (14.04) with iptables version 1.4.21
Can someone kindly help me solve the issue?
Thanks
Varun
TL;DR
Simply setting ToS or DSCP markings on packets does nothing. You actually need to configure a device to do different things with different markings. If you want that to include queuing, you need to configure queues, and assign different markings to different queues.
A More Complete Explanation
You are wanting to use QoS. QoS is a huge subject, but I will try to explain a few things. A guide to using QoS on Linux can be found at Traffic Control HOWTO.
ToS, which has really been supplanted by DSCP (Differentiated Services Code Point), is simply marking packets to differentiate the various packets for different treatment at some point. The ToS field was part of the original IPv4 packet specification, but there is nothing in the standard that mandates a device must use or respect that field.
QoS involves differentiating (marking) packets, and then doing something based on the marking. What you do with the markings can be things like shaping, policing, queuing (including priority queuing).
A hardware interface will have a FIFO queue, and that is the default and only queue, regardless of packet markings. The hardware is completely unaware of packet headers or markings.
Actually using the markings is usually done in a layer-3 network device, e.g. a router. For instance, you can configure different software queues for a router interface, and you can assign packets with different markings to different queues. Queues are relatively small, not like real buffers. Priority queues will be served before regular queues. Queues don't exist until you define them, and packets are not assigned to different queues unless you have configured rules to do that. You could assign BE (Best Effort, ToS 0) packets to a priority queue, and EF (Expedited Forwarding) packets to a low priority queue.
When a queue fills up, new packets destined for that queue will be dropped (called tail-drop). Tail-drop can be a problem for TCP because it can cause all TCP flows using a queue to become synchronized (global synchronization) where they back off and ramp up synchronously, alternately starving or flooding a queue. There are methods to try to prevent this, e.g. RED (Random Early Detection). RED will actually drop random packets in a queue. This is to force the various TCP flows using the queue to back off and ramp up on different schedules.
Many network switches will automatically assign BE (Best Effort, ToS 0) to anything coming into the switch, unless you have configured the switch to trust the markings on one or more interfaces. Routers will typically trust the markings, but they will not do anything with the markings unless you configure them to do so.
QoS will not work across the Internet, only within your network. You need to have a comprehensive set of QoS policies that are consistently implemented across your network. You may be able to negotiate, for a fee, for your ISP to respect some of your QoS markings and policies, but that only goes as far as your ISP. As your traffic leaves your network, or you ISPs network if you have an agreement with it, your QoS markings and policies will be completely ignored, and the packets will probably be set to BE.
I need to simulate a massive amount of TCP/IP ethernet traffic. For example, I want to simulate the environment that an ISP has where there might be 40,000 different IP addresses sending TCP/UDP IP traffic to different remote hosts. This is my ideal setup:
Traffic generator - > the device I want to test (one inbound interface and one outbound interface) - > traffic receiver.
The device I want to test is a network traffic monitor/QOS appliance. It effectively sits 'in-line', one interface would be connected to the traffic generator and the other interface connected to the traffic receiver. This in-line interface is effectively a bridge and is not assigned an IP address. It can monitor & apply QOS rules on all traffic passing over that bridge interface.
Layer 4 control is important, so that I can set port numbers (80, 443, 22 etc). Layer 7 application information would be ideal as the device I am testing also does deep packet inspection.
Methods I have already tried include using iperf but in order to simulate 40,000 IP addresses I would need to configure 40,000 virtual interfaces on both the traffic generator and the traffic receiver manually, and I have found that iperf is limited to about 1000 simultaneous connections(on my set up). I have also tried replaying large PCAP files, but then I do not have control over the packets to test QOS capabilities.
Other software/solutions I have looked into are:
http://mininet.org/ (can't handle the amount of connections I need).
ns-3
I am looking for someone to point me in the right direction. Thank you.
There are commercial products for this kind of thing. Short of a home-brew setup with a combination of apache bench, siege, and tcpreplay (which would take significant effort to implement).
See www.spirent.com or www.ixiacom.com.
I've seen while searching that it is possible to use two different programs on the same computer communicating over the network using the same port and same network interface provided one use UDP and the other TCP. However I didn't get a good explanation, how does it actually work and why this is possible?
Is it also possible for multiple programs to use the same UDP port since UDP does not establish a real connection between the peers, but just sends the packets to an address? I understand it's not possible with TCP as it creates a synchronized connection between the server and the client, but what about UDP?
Please explain in details if possible, or link a good article on the topic.
The other answers are correct but somewhat incomplete.
An IP (aka "INET") socket "connection" (i.e. communication between two processes, possibly on different machines) is defined by a 5-tuple: protocol, source address, source port, destination address, destination port. You can see that this is not limited to a stateful connection such as TCP.
This means that you can bind different processes to any unique instance of that 5-tuple. Because the "protocol" (e.g. TCP and UDP) is part of the differentiating factor, each can have a different process.
Theoretically, you could bind different services to the same TCP port if they bind to different interfaces (network cards, loopback, etc.) though I've never tried it.
It is standard practice, however, to always use the same service on the same port number. If both UDP and TCP are supported, they're just different ways of communicating with that same service. DNS, for example, uses UDP on port 53 for lookup because they are small requests and it's faster than creating a TCP connection but DNS also uses TCP on port 53 for "transfers" which are infrequent and can have large amounts of data.
Lastly, in complete accuracy, it isn't necessarily a 5-tuple. IP uses the "protocol" to pass to the next layer such as TCP and UDP though there are others. TCP and UDP each seperately differentiate connections based on the remaining 4 items. It's possible to create other protocols on top of IP that use completely different (perhaps port-less) differentiation mechanisms.
And then there are different socket "domains", such as the "unix" socket domain, which is completely distinct from "inet" and uses the filesystem for addressing.
The destination isn't identified by IP Addr:Port alone. There is another thing - IP header has a field called Protocol which differentiates the TCP and UDP endpoint. As such it becomes possible for two process to bind to same IP:Port as long as communication protocol is different.
The endpoint of a connection is for UDP and TCP defined by IP, protocol (TCP or UDP) and port. This means as long as you use a different protocol the endpoint of the communication is different too.
Because they are not the only component of the means of address. It's the same as why you can have two houses with the same number on different streets, or why you know John Whorfin is not the same Red Lectroid as John Bigbooté.
Each IP packet contains a field that says which transport-layer protocol is to be used, and within the domain of that protocol is a set of ports that can be the same as in any other protocol because they are actually a completely separate set.
As for the second question, there are answers elsewhere.