I need to simulate a massive amount of TCP/IP ethernet traffic. For example, I want to simulate the environment that an ISP has where there might be 40,000 different IP addresses sending TCP/UDP IP traffic to different remote hosts. This is my ideal setup:
Traffic generator - > the device I want to test (one inbound interface and one outbound interface) - > traffic receiver.
The device I want to test is a network traffic monitor/QOS appliance. It effectively sits 'in-line', one interface would be connected to the traffic generator and the other interface connected to the traffic receiver. This in-line interface is effectively a bridge and is not assigned an IP address. It can monitor & apply QOS rules on all traffic passing over that bridge interface.
Layer 4 control is important, so that I can set port numbers (80, 443, 22 etc). Layer 7 application information would be ideal as the device I am testing also does deep packet inspection.
Methods I have already tried include using iperf but in order to simulate 40,000 IP addresses I would need to configure 40,000 virtual interfaces on both the traffic generator and the traffic receiver manually, and I have found that iperf is limited to about 1000 simultaneous connections(on my set up). I have also tried replaying large PCAP files, but then I do not have control over the packets to test QOS capabilities.
Other software/solutions I have looked into are:
http://mininet.org/ (can't handle the amount of connections I need).
ns-3
I am looking for someone to point me in the right direction. Thank you.
There are commercial products for this kind of thing. Short of a home-brew setup with a combination of apache bench, siege, and tcpreplay (which would take significant effort to implement).
See www.spirent.com or www.ixiacom.com.
Related
I know, how to write a C# application that works through a local network.
I mean I know, how to make my client-side application access my server-side application in a single local network.
But I wonder: How do such apps, as Skype, TeamViewer, and many other connect via global network?
I apologise, if this question is simple or obvious, but I couldn't find any information about this stuff.
Please, help me, I'll be very grateful. Any information is accepted - articles, plain info, books,and so on...
Question is very wide and I try to do short overview.
Following major difference between LAN (Local Area Network) and WAN (Wide Area Network):
Network quality:
LAN is more or less stable, WAN can be with network issues like:
Packet loss (you need use loss-tolerant transport like TCP or UDP with retransmits or packet loss concealment)
Packet jitter (interpacket intervals may differ a lot from sending part). Most common thing is packets bursts.
Packet reordering
Packet duplication
Network connectivity
WAN is less stable than LAN. So you need properly handle all things like:
Connection stale
Connection loss
Errors in the middle of the connection (if you use UDP for example)
Addresses:
In WAN you deal with different network equipment between client and server (or peers in case of peer-to-peer communication). You need to take in account:
NATs - most of the clients are behind NAT and you need to pass them through. According technics are called "NAT traversal"
Firewalls - may ISP has own rules what client can do or can't. So if you do something specific like custom transport protocol you may bump into ISP firewalls.
Routing - especially multicast and broadcast communication. In common case multicast is not possible to route. Broadcasts are never routed. So you need to avail this type of communication if you want to use WAN.
May be I forgot something. But these points are major. You can read many articles about any of them.
I'm going to develop a network monitoring system for my final year project in college. Before the development, I've gone through some research about how a network monitoring system works. From my understanding, network monitoring system used ICMP packet (Ping) to make sure the devices is "alive" in the network. Here is the problem I had:
If network monitoring system need to ping each and every device in the network, then the network traffic will become heavy, is that good approach to use this method? Or is there any other possible alternative?
Network monitoring system provide set of data of the network traffic, is that possible for the network monitoring system to get the traffic level for all the connection? Assume there are 3 PCs, PC X, PC Y, and PC Z. PC X connected to PC Y, and PC Y connected to both of the PCs, PC X used for monitor the network, when PC Y sending packets to PC Z, is that possible that PC X get the information that PC Y is sending packets to PC Z?
Thanks
The network traffic load of monitoring is negligible compared to normal network traffic. Typically, you would poll devices eveny minute or every 5min, with a single packet getting a single packet in return.
Network traffic monitoring is typically done using SNMP to poll the ifInOctets and ifOutOctets counters for each interface. This will allow you to calculate total traffic and transmission rate (bandwidth utilisation) on a per-interface basis. However, it does not allow you to break this down by traffic type or destination.
I would suggest you take a look at software such as MRTG which do this sort of monitoring very neatly.
If you are wanting to monitor the health of the entire network, not just traffic on links, you may want to look into catching SNMP traps from devices to hear about problems, and also monitoring other things such as reachability (via ping packets), routes (to detect flapping routes), and even check services on hosts (such as DNS, HTTP, SMTP and so on). To do this, you might like to investigate the Nagios software and its check plugins that do the various tests.
No, the network traffic will not become heavy. You seem to have greatly underestimated the network capacity, or greatly overestimated the amount of traffic caused by ping (or both). Increase in network load caused by periodic pings from the network monitoring server to all hosts in the network will be minuscule unless your network is some sort of a slow ancient half duplex network. I would be more worried about the required CPU usage on the network monitoring server to be able to process all the pings and do useful stuff with them.
It is only possible if you are running a specialized software on the PCs to be able to grab and send this data off to the network monitoring server. sflow is a good example. Do you have any control over what can be configured on the PCs? If so, sky is your limit. If we treat the PCs as a blackbox, there really isn't that much that you can do other than pinging and possibly port scanning.
Monitoring individual hosts is only part of the picture. It is also very important for the network monitoring system to monitor the routers and switches that form the network. The amount of information that you can gather will depend hugely on what these network equipments support and whether you have the access to configure them. If SNMP is enabled, that would be a great start.
If you have a switch with at least one subscriber to a multicast address, how much additional load would each additional subscriber add?
Example:
You have a 10G switch (with IGMP) with 10 servers and no other activity.
When Server1 subscribers to a 1G multicast feed, the switch will have 1G of load.
What would the load be after Server2 and Server3 subscribed?
Obviously traffic to the switch would not increase, but what about the switch's internal load?
Houw would the answer be different without IGMP?
The whole idea of multicast is that it is efficient. The presence of one subscriber downstream causes the switch to send an IGMP join request of its own upstream and pass incoming multicasts downstream, without duplication. The addition of further downstream subscribers has no effect at all except to increment an internal subscriber count for that group. When that goes back to zero it sends an IGMP leave request of its own upstream.
I don't know what you mean by 'without IGMP'. There is no such thing as UDP multicast without IGMP. It is a contradiction in terms.
Firstly, some background information for you.
The traditional definition of routers and switches are along the lines of:
Router: a device capable of routing a packet form one IP subnet to a different IP subnet
Switch: a device capable of switching a packet within the same IP subnet
However, this traditional definition no longer holds these days because we have switches that can route traffic from one IP subnet to another IP subnet and even perform complex operations such as QoS at wire speed.
Therefore it is often easier to redefine Routers and Switches as follows:
Router: a device that uses the CPU to route packets, often inspects parts of packets that are higher up the OSI layer.
Switch: a device with ASIC(s) (a.k.a switching chips) that switches/routes traffic at full wire speed. What this means is that if the switch has 24 1Gbps ports, it will be able to switch 24Gbps bi-directional traffic without dropping any packets.
Now, to answer your question, it is important to determine whether the ASIC in your switch is capable of handling multicast traffic or not. If so, adding "load" really isn't an issue, as long as you ensure that each switch port is not congested (e.g. 2Gbps of traffic trying to egress out of 1Gbps port). If the ASIC in your switch is NOT capable of handling multicast traffic, it is highly likely that the switch will simply send all multicast traffic up to the CPU. Then it would be up to the software to determine where each packet goes. CPUs on switches are not powerful, because their primary role isn't to route/switch packets, but to manage the switch (e.g. configure the ASIC so that packets get switched properly). Therefore, if your switch is sending packets up to the CPU, the switch will struggle. You won't get anywhere near 1Gbps of multicast via the CPU.
Without IGMP, switches, by default, will flood out the traffic on all ports. Again, this is not a problem for the switch itself because it can handle this at wirespeed. It may cause problems for other parts of the network because traffic is needlessly being duplicated.
The reason for this long answer is because the phrase "10G switch" in your example is quite misleading, and it led me to believe that you maybe thinking that a powerful CPU sits at the center of the switch that is capable of performing 10Gbps bi-directional switching. This is simply not the case, and talking about "load" on a switch therefore often makes little sense.
I hope this helps.
2 computers are in different subnets.
Both are Windows machines.
There are 2-5 IGMP-ready routers between them.
They can connect each other over multicast protocol (they have joined the same multicast group and they know about each other's existance).
How to establish a reliable TCP connection between them without any public server?
Programming language: C++, WinAPI
(I need a TCP connection to send some big critical data, which I can not entrust to UDP)
You haven't specified a programming language, so this whole question may be off-topic.
Subnets are not the problem. Routability is the problem. Either there is routing set up or there isn't. If they are, for example, both behind NAT boxes, then you're at the mercy of the configuration of the nat boxes. If they are merely on two different subnets of a routed network, it's the job of the network admin to have set up routing. So, each has an IP address, and either can address the other.
On one machine, you are going to create a socket, bind it to some port of your choice, and listen. On the other, you will connect to the first machine's IP + the selected port.
edit
I'm going to try again, but I feel like there's a giant conceptual gap here.
Once upon a time, the TCP/IP was invented. In the original conception, every item on the network has an IPV4 address, and every machine could reach every other machine, via routing, except for machines in the 'private' address space (10.x, etc).
In the very early days, the only 'subnets' were 'class A, class B, class C'. Later the idea of subdividing a network via bitmasks was added. The concept of 'subnet' is just a way of describing a piece of network in which all the hosts can deliver packets to each other by one hop over some transport or another. In a properly configured network, this is only of concern to operating system drivers. Ordinary programs just address packets over the network and they arrive.
The implementation of this connectivity was always via routing protocol. If you have a (physical) ethernet A over here, and a (physical) ethernet B over there, connected by some sort of point-to-point link, the machines on A need to know where to send packets for B. Or, to be exact, they need to know where to send 'not-A' packets, and whatever they send them needs to know where to send 'B' packets. In simple cases, this is arranged via explicit configuration: routing rules stuffed into router boxes or even computers with multiple physical interfaces. In more complex cases, routing boxes intercommunicate via protocols like EGP or BGP or IGMP to learn the network topology.
If you use the Windows 'route' command, you will see the 'default route' that the system uses to send packets that need to leave the local subnet. It is generally the address of the router box responsible for moving information from the local subnet to everywhere else.
The whole goal of this routing is to arrange that a packet sent from a.b.c.d to e.f.g.h will get there. TCP is no different than UDP, except that you can't get there by multicast or broadcast: you need to know the exact address of your correspondent.
DNS was invented to allow hosts to learn each other's IP addresses without having human being send them around in email messages.
All this stops working when people start using NAT and firewalls to turn off routing. The whole idea of NAT is that the computers behind the NAT box are not addressable at all. They all appear to have one IP address. They can send stuff out, but they can only receive stuff if the NAT box has gone to extra trouble to map them a port.
From your original message, I sort of doubt that NAT is in use here. I just don't understand your comment 'I don't have access to the network.' You say that you've sent UDP packets here and there. So how did you do that? What addresses did you use?
How can I connect a system to a network and sniff for virus/spyware related traffic? I'd like to plug in a network cable, fire up an appropriate tool sand have it scan the data for any signs of problems. I don't expect this to find everything, and this is not to prevent initial infection but to help determine if there is anything trying to actively infect other system/causing network problems.
Running a regular network sniffer and manually looking through the results is no good unless the traffic is really obvious,but I havn't been able to find any tool to scan a network data stream automatically.
I highly recommend running Snort on a machine somewhere near the core of your network, and span (mirror) one (or more) ports from somewhere along your core network path to the machine in question.
Snort has the ability to scan network traffic it sees, and automatically notify you via various methods if it sees something suspicious. This could even be taken further, if desired, to automatically disconnect devices, et cetera, if it finds something.
Use snort: An open source network intrusion prevention and detection system.
Wireshark, formerly ethereal is a great tool, but will not notify you or scan for viruses. Wireshark is a free packet sniffer and protocol analyzer.
Use the netstat -b command to see which processes have which ports open.
Use CPorts to see a list of ports and the associated programs, and have the ability to close those ports.
Download a free anti-virus program such as free AVG.
Setup your firewall more tightly.
Setup a gateway computer to let all network traffic go through. Take the above recommendataions to the gateway computer instead. You will be checking your whole network instead of just your one computer.
You can make Snort scan traffic for viruses. I think this will be the best solution for you.
For watching local network traffic your best bet (with a decent switch) is to set your switch to route all packets out a specific interface (as well as whatever interface it would normally send). This lets you monitor the entire network by dumping traffic down a specific port.
On a 100 megabit network, however, you'll want a gigabit port on your switch to plug it into, or to filter on protocol (e.g. trim out HTTP, FTP, printing, traffic from the fileserver, etc.), or your switch's buffers are going to fill up pretty much instantly and it'll start dropping whatever packets it needs to (and your network performance will die).
The problem with that approach is that most networks today are on switches, not hubs. So, if you plug a machine with a packet sniffer into the switch, it will only be able to see traffic to and from the sniffing machine; and network broadcasts.
As a followup to Ferruccio's comment you will need to find some method of getting around your switches.
A number of network switches have the option of setting up port mirrors, so that all traffic (regardless of the destination) will be copied, or "mirrored", to a nominated port. If you could configure your switch to do this then you would be able to attach your network sniffer here.
Network Magic, if you don't mind something that's not open source.
You can use an IDS, hardware or software
http://en.wikipedia.org/wiki/Intrusion-detection_system