Are there any known problems in running two instances of tcpdump at the same time on the same network interface (with non-conflicting filters)?
I'm experiencing some weird timestamping and I thought that that could be the cause.
packet filtering is usually done in the kernel so it shouldn't cause any missed packets or out of order packets in each tcpdump process. If you're comparing the two different tcpdumps and seeing the slightly off time stamps I could see a couple of reasons for that but it's hard to say without knowing what specifically the issue is you're having.
Related
My professor told me that sequential IP sequence numbers is typical behavior of most IP stacks (and showed us examples of packet sniffers), but I thought IP sequence numbers are supposed to be randomly generated to avoid attacks?
Which one is it. I am taking a digital forensics class and I need to know how to differentiate "normal" tcp/ip stack traffic from "abnormal" traffic
Linux and Solaris are the two major offenders of predictable IP id's. I unfortunately have to run to a meeting right now so I can't explain but i'll try and remember to edit this with a better explanation when I get back. Most other OS's were patched over a decade ago to have psuedo random starting IP id's.
Suppose, there is a network which gives a lot of Timeout errors when packets are transmitted over it. Now, timeouts can happen either because the network itself is inherently lossy (say, poor hardware) or it might be that the network is highly congested, due to which network devices are losing packets in between, leading to Timeouts. Now, what additional statistics about the traffic being transmitted (like Missing Packets errors etc.) are required that might help us to find out whether timeouts are happening due to poor hardware, or too much network load.
Please note that we have access only to one node in the network (from which we are transmitting packets) and as such, we cannot get to know the load being put by other nodes on the network. Similarly, we don't really have any information about the hardware being used in the network. Statistics is all that we have.
A network node only has hardware information about its local collision domain, which on a standard network will be the cable that links the host to the switch.
All the TCP stack will know about lost packets is that it is not receiving acknowledgements so it needs to resend, there is no mechanism for devices (E.g. switches & routers) between a source and destination to tell the source that there is a problem.
Without access to any other nodes the only way to ascertain if your problem is load based would be to run a test that sends consistent traffic over the network for a long period, if the packet retry count per second/minute/hour remains the same then it would suggest that there is a hardware issue, if the losses only occur during peak traffic periods then the issue could be load related. Of course there could be a situation where misconfigured hardware issues will only be apparent during high traffic periods, this takes things back to the main problem which is that you need access to network stats from beyond your single node.
In practice, nearly all loss on terrestrial network paths is due to either congestion or firewalls. Loss due to bit-errors is extremely rare. Even on wireless networks, forward error correction handles most bit/media/transmission errors. Congestion can be caused by a lot of different factors: any given network path will involve dozens of devices and if any one of them becomes overloaded for even a moment, packets will be dropped.
The only way to tell the difference between congestion induced packet loss and media errors is that media errors will occur independent of load. In other words, the loss rate will be the same whether you are sending a lot of data or only a little data.
To test that, you will need some control, or at least knowledge, of the load on the path. Since you don't have control and the only knowledge you have is from source-node observation, the best you can do is to take test samples (using ping is the easiest) around the clock and throughout the week, recording loss rates and latencies. These should give you an idea of when the path is relatively idle. If loss rates remain significant even when the path is (probably) idle, then there might be a media-loss issue. But again, that is extremely rare.
For background, I have written a few articles on the subject:
Loss, Latency, and Speed, discussing what statistics you can observe about a path and what they mean.
Common Network Performance Problems, discussing the most common components in a network path and how they affect performance (congestion).
I am doing a measurement project where I send and receive data from numerous devices on my network. The send/receive can be considered fast and intensive, as there is almost no pause and a continuous flow of data. However, the data to/from each device is quite small, on the order of a couple of bytes each. For some reason, I am experiencing a reset of my entire ethernet connection where my internet connection also goes down, and I lose connection to all my devices as well. I have never experienced such a situation and am wondering what are some of the common situations that might lead to resets like this?
Actually, it turns out this had to do with the way I constructed my thread, in which I would create a new socket continuously without discarding the previous one. Stupid right? Well I fixed the code and the ethernet no longer crashes.
I've been checking out using a system called ROS (http://www.ros.org) for some work.
There are lots of different types of data that get sent between network nodes in ROS.
You define a struct of data that you want to send in a message, and ROS will handle opening a specific port between the two nodes that will only send that struct of data.
So if there are 5 different messages, there will be 5 different ports.
As opposed to this scenario, I have seen other platforms that just push all the different messages across one port. This means that there needs to be a sort of multiplexing/demultiplexing (done by some sort of message parsing on the receivers end).
What I wonder is... which is better from a performance perspective?
Do operating systems switch based on ports quickly, so that a system like ROS doesn't have to do too much work to work out what is in the message and interpreting it?
OR
Is opening lots of ports going to mean lots of slower kernel calls, and the cost of having to work out and translate message types end up being more then the time spent switching between ports?
When this scales to a large amount of data at high rates and lots of different messages types there will be lots of ports. So I imagine that when scaling each of these topologies that performance will be a big factor in selecting the way to work.
I should also point out that these nodes usually exist on one small network, or most of the time on the one machine in which networking is used as a force of inter-process communication. So the transmission time is only a very small factor in the overall system timing.
ROS being an architecture for robots may have one node for every sensor and actuator, so depending on the complexity of your system we may be talking about 20-30 nodes pushing small-ish (100bytes or so) data between 10-100Hz
It depends. I do not know the specifics of ROS but in networking it comes down to the following constraints:
Distance: speed of light is fast but over a distance it starts making a difference
Protocol Overhead: connection oriented vs. connection-less
On the OS side, maintaining a list of free ports isn't such much of an overhead - of course there is a cost to it but everything is relative: if you are talking about a distributed system with long distance links, then it is easy to argue that cycling through OS network ports ranks as lower concern compared to managing communication quality.
Without a more specific question, I'll stop here.
I don't have any data on this, but it seems plausible that multiple ports might be handled more efficiently by multi-core systems, as opposed to demultiplexing within the program.
I am sending real-time-critical data over the internet between two dedicated computers, using my own protocol.
There is, of course, latency involved.
For debugging and optimization, I like to have both computers use the same timebase. I.e, I need to know the time difference of their clocks so that I can judge the latencies better.
Of course, relativism and such doesn't really allow me to sync them perfectly, but I like to get as close as possible.
Relying on NTP alone does not appear good enough - clocks can be off by half a second in my experience (clarification: I relied so far on the default one provided by Apple).
I need precision in the 1/10s range, at least. The two computers won't be too far apart, ICMP ping times are less than 100ms, usually.
Any suggestions how to do this?
(currently, the machines involved run OS X, so if you know a solution just for them, that'll be a start)
Get the time from a GPS receiver connected to the machines. If they are in a data centre it can be difficult getting an antenna into a location that it can get a lock though unfortunately.
I would suggest that your best bet is to install an ntp server on one of the machines and get the other to sync to it.
Did you try to have one of the machine as NTP server for the other? May be they won't be in sync with the 'real' time but this may bring you within the precision you require.
Relying on NTP alone does not appear
good enough - clocks can be off by
half a second in my experience.
That's strange. NTPd over Internet is supposed to give you much greater precision.