I have created a bandwidth meter application to measure total Internet traffic. I need to test the application with relatively high data transfer rates, such as 4 Mbps. I have a slow Internet connection, so I need a simulator to test my application to see the behavior with high throughput rates.
As an option, you can run some HTTP server in one virtual machine with NAT'ed network adapter and test your bandwidth meter against it from the host system or a similar VM.
There are commercial packet generators that do this, and also a few freely available ones like PackETH and Bit-Twist.
There are also other creative solutions. For example, do the packets need to be IP packets for your purpose? If not, you could always get a "dumb" switch or hub (no spanning-tree or other loop protection) and plug a crossover cable into it. (or a straight-through Ethernet cable would work if the switch supports Auto-MDIX) The idea would be that with a loop in your network, the hub/switch will flood the network to 100% for you since it will continually re-forward the same packets.
If you try this, be sure yours is the only computer on the network, since this technique will effectively render it useless. ;-)
You could always send some IP broadcast packets to "seed" the loop. Otherwise, the first thing I think you'd likely see is broadcast ARP packets, which won't help if you're measuring layer 3 traffic only.
Lastly, (and especially if this sounds like too much trouble) I recommend you read up on dependency injection and refactor your code so you can test it without the need for a high-speed interface. Of course, you'll still need to test your code in a real high-speed environment, but doing this will give you much more confidence in your code.
Related
I'm working on ovs-dpdk, I want to test whether a port has packet loss. For hardware switch, you could use IXIA or some to send continuous packet, but this is virtual switch and I have no IXIA.
So I use ping to test this, but ping's packet rate is too low, could I use pktgen to test this? If I use pktgen, how to verify if there is packet loss?
Or is there some other method? Thank you~
You can generate a literate of small 64-byte packets using DPDK applications, like DPDK Pktgen, Cisco TRex or even the testpmd app included in DPDK. All those software generators will be able to generate quite a lot of traffic in virtualized environment as well as on the host.
If all that you are interested in is a packet loss, you can use any of the listed above options. TRex and Pktgen do support RFC 2544 tests as well.
A typical setup would include one VM with a generator, another VM with either a generator or a forwarding DPDK application (like l2fwd or l3fwd).
The packet loss is basically the difference between sent and received packets, so just run the test for a while and then see the difference.
Overall, it might be a bit scary at the beginning, but once you understand the basics, it is quite easy to setup and use. And you can always ask a question on StackOverflow...
I ask myself that for some days now, and couldn't find a clear answer.
Imagine a network with several rooms, each room with it's own (Layer 2) Switch.
Those are then connected in star topology to another switch which is then connected to the router.
All Clients get IPs from the same range, all clients are in the same subnet and in the same VLAN.
Do you know a way how to determine, which client is connected to which switch, without involving the switch. E.g only with technologies and mechanisms taking place on the client.
The goal would be to be able to draw automated maps of the client distribution in the network, but, as I said, without involving the switch (for example access it's API or somethign like that).
Regards
Me
I am afraid that you are right: In general, in a "normal" Network (which means: In a Network with just Standard Switches and Routers), there is no way to do so.
Only if the Switches were managed, you could write a script which polls the arp-tables from the Switches, but of course this doesn't work with unmanaged Switches.
I realised that there is no way to do this.
The problem is, that a normal network switch, which works on Layer 2 of the OSI model, is not "visible" for tools like traceroute or such.
If you want to traceroute to, lets say: google.com, your traceroute application sends a packet with a TTL of 1 to your default gateway. Your default gateway then decreases the TTL and discards the package, because it has reached a TTL of 0.
Because of that, it answers to your computer and your computer can record, from where it received the first answer.
It then sends again a package to google.com, but this time with a TTL of 2, and the process begins from start.
Now you can't do with switches what you can do with routers, because (normal layer 2) switches do not know about IP...
Cisco offers a layer2 traceroute utility, but this is limited to the cisco ecosystem and only works with cisco hardware.
So, I fear the answer is, that it is not possible in general.
I'm going to develop a network monitoring system for my final year project in college. Before the development, I've gone through some research about how a network monitoring system works. From my understanding, network monitoring system used ICMP packet (Ping) to make sure the devices is "alive" in the network. Here is the problem I had:
If network monitoring system need to ping each and every device in the network, then the network traffic will become heavy, is that good approach to use this method? Or is there any other possible alternative?
Network monitoring system provide set of data of the network traffic, is that possible for the network monitoring system to get the traffic level for all the connection? Assume there are 3 PCs, PC X, PC Y, and PC Z. PC X connected to PC Y, and PC Y connected to both of the PCs, PC X used for monitor the network, when PC Y sending packets to PC Z, is that possible that PC X get the information that PC Y is sending packets to PC Z?
Thanks
The network traffic load of monitoring is negligible compared to normal network traffic. Typically, you would poll devices eveny minute or every 5min, with a single packet getting a single packet in return.
Network traffic monitoring is typically done using SNMP to poll the ifInOctets and ifOutOctets counters for each interface. This will allow you to calculate total traffic and transmission rate (bandwidth utilisation) on a per-interface basis. However, it does not allow you to break this down by traffic type or destination.
I would suggest you take a look at software such as MRTG which do this sort of monitoring very neatly.
If you are wanting to monitor the health of the entire network, not just traffic on links, you may want to look into catching SNMP traps from devices to hear about problems, and also monitoring other things such as reachability (via ping packets), routes (to detect flapping routes), and even check services on hosts (such as DNS, HTTP, SMTP and so on). To do this, you might like to investigate the Nagios software and its check plugins that do the various tests.
No, the network traffic will not become heavy. You seem to have greatly underestimated the network capacity, or greatly overestimated the amount of traffic caused by ping (or both). Increase in network load caused by periodic pings from the network monitoring server to all hosts in the network will be minuscule unless your network is some sort of a slow ancient half duplex network. I would be more worried about the required CPU usage on the network monitoring server to be able to process all the pings and do useful stuff with them.
It is only possible if you are running a specialized software on the PCs to be able to grab and send this data off to the network monitoring server. sflow is a good example. Do you have any control over what can be configured on the PCs? If so, sky is your limit. If we treat the PCs as a blackbox, there really isn't that much that you can do other than pinging and possibly port scanning.
Monitoring individual hosts is only part of the picture. It is also very important for the network monitoring system to monitor the routers and switches that form the network. The amount of information that you can gather will depend hugely on what these network equipments support and whether you have the access to configure them. If SNMP is enabled, that would be a great start.
Can someone explain the concepts of IPoIB and TCP over infiniband? I understand the overall concept and data rates provided by native infiniband, but dont quite understand how TCP and IPoIB fit in. Why do u need them and what do they do? What is the difference when someone says their network uses IPoIB or TCP with infiniband? Which one is better? I am not from a strong networking background, so it would be nice if you could elaborate.
Thank you for your help.
InfiniBand adapters ("HCAs") provide a couple of advanced features that can be used via the native "verbs" programming interface:
Data transfers can be initiated directly from userspace to the hardware, bypassing the kernel and avoiding the overhead of a system call.
The adapter can handle all of the network protocol of breaking a large message (even many megabytes) into packets, generating/handling ACKs, retransmitting lost packets, etc. without using any CPU on either the sender or receiver.
IPoIB (IP-over-InfiniBand) is a protocol that defines how to send IP packets over IB; and for example Linux has an "ib_ipoib" driver that implements this protocol. This driver creates a network interface for each InfiniBand port on the system, which makes an HCA act like an ordinary NIC.
IPoIB does not make full use of the HCAs capabilities; network traffic goes through the normal IP stack, which means a system call is required for every message and the host CPU must handle breaking data up into packets, etc. However it does mean that applications that use normal IP sockets will work on top of the full speed of the IB link (although the CPU will probably not be able to run the IP stack fast enough to use a 32 Gb/sec QDR IB link).
Since IPoIB provides a normal IP NIC interface, one can run TCP (or UDP) sockets on top of it. TCP throughput well over 10 Gb/sec is possible using recent systems, but this will burn a fair amount of CPU. To your question, there is not really a difference between IPoIB and TCP with InfiniBand -- they both refer to using the standard IP stack on top of IB hardware.
The real difference is between using IPoIB with a normal sockets application versus using native InfiniBand with an application that has been coded directly to the native IB verbs interface. The native application will almost certainly get much higher throughput and lower latency, while spending less CPU on networking.
How can I connect a system to a network and sniff for virus/spyware related traffic? I'd like to plug in a network cable, fire up an appropriate tool sand have it scan the data for any signs of problems. I don't expect this to find everything, and this is not to prevent initial infection but to help determine if there is anything trying to actively infect other system/causing network problems.
Running a regular network sniffer and manually looking through the results is no good unless the traffic is really obvious,but I havn't been able to find any tool to scan a network data stream automatically.
I highly recommend running Snort on a machine somewhere near the core of your network, and span (mirror) one (or more) ports from somewhere along your core network path to the machine in question.
Snort has the ability to scan network traffic it sees, and automatically notify you via various methods if it sees something suspicious. This could even be taken further, if desired, to automatically disconnect devices, et cetera, if it finds something.
Use snort: An open source network intrusion prevention and detection system.
Wireshark, formerly ethereal is a great tool, but will not notify you or scan for viruses. Wireshark is a free packet sniffer and protocol analyzer.
Use the netstat -b command to see which processes have which ports open.
Use CPorts to see a list of ports and the associated programs, and have the ability to close those ports.
Download a free anti-virus program such as free AVG.
Setup your firewall more tightly.
Setup a gateway computer to let all network traffic go through. Take the above recommendataions to the gateway computer instead. You will be checking your whole network instead of just your one computer.
You can make Snort scan traffic for viruses. I think this will be the best solution for you.
For watching local network traffic your best bet (with a decent switch) is to set your switch to route all packets out a specific interface (as well as whatever interface it would normally send). This lets you monitor the entire network by dumping traffic down a specific port.
On a 100 megabit network, however, you'll want a gigabit port on your switch to plug it into, or to filter on protocol (e.g. trim out HTTP, FTP, printing, traffic from the fileserver, etc.), or your switch's buffers are going to fill up pretty much instantly and it'll start dropping whatever packets it needs to (and your network performance will die).
The problem with that approach is that most networks today are on switches, not hubs. So, if you plug a machine with a packet sniffer into the switch, it will only be able to see traffic to and from the sniffing machine; and network broadcasts.
As a followup to Ferruccio's comment you will need to find some method of getting around your switches.
A number of network switches have the option of setting up port mirrors, so that all traffic (regardless of the destination) will be copied, or "mirrored", to a nominated port. If you could configure your switch to do this then you would be able to attach your network sniffer here.
Network Magic, if you don't mind something that's not open source.
You can use an IDS, hardware or software
http://en.wikipedia.org/wiki/Intrusion-detection_system