How, if at all, is it possible to add timestamps to each line of an output generated by the & PowerShell operator?
Example:
PS H:\> $result = & ping 192.168.1.1
PS H:\> echo $result
Pinging 192.168.1.1 with 32 bytes of data:
Reply from 192.168.1.1: bytes=32 time=104ms TTL=250
Reply from 192.168.1.1: bytes=32 time=106ms TTL=250
Reply from 192.168.1.1: bytes=32 time=102ms TTL=250
Reply from 192.168.1.1: bytes=32 time=102ms TTL=250
Ping statistics for 192.168.1.1:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 102ms, Maximum = 106ms, Average = 103ms
Desired result:
PS H:\> echo $result
2014-12-08T14:45:48.8898125+00:00:Pinging 192.168.1.1 with 32 bytes of data:
2014-12-08T14:45:48.8932661+00:00:Reply from 192.168.1.1: bytes=32 time=104ms TTL=250
2014-12-08T14:45:48.9233451+00:00:Reply from 192.168.1.1: bytes=32 time=106ms TTL=250
2014-12-08T14:45:48.9765438+00:00:Reply from 192.168.1.1: bytes=32 time=102ms TTL=250
2014-12-08T14:45:49.0233105+00:00:Reply from 192.168.1.1: bytes=32 time=102ms TTL=250
2014-12-08T14:45:49.0233201+00:00:
2014-12-08T14:45:49.0238753+00:00:Ping statistics for 192.168.1.1:
2014-12-08T14:45:49.0239210+00:00: Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
2014-12-08T14:45:49.0233318+00:00:Approximate round trip times in milli-seconds:
2014-12-08T14:45:49.0237209+00:00: Minimum = 102ms, Maximum = 106ms, Average = 103ms
I know how to split / join a PowerShell array, but this can only happen AFTER the & operator completes. I am looking for more realtime-like solution, where timestamps are added to the output while the & operator is running.
By the way, the timestamp itself is $($(Get-Date -Format o) + ":")
You could use a filter:
filter timestamp {"$(Get-Date -Format o): $_"}
$result = & ping 192.168.1.1 | timestamp
Sample output from $result:
2014-12-08T11:42:59.2827202-05:00:
2014-12-08T11:42:59.2857205-05:00: Pinging 192.168.1.1 with 32 bytes of data:
2014-12-08T11:43:03.1241043-05:00: Request timed out.
2014-12-08T11:43:08.1236042-05:00: Request timed out.
2014-12-08T11:43:13.1241042-05:00: Request timed out.
2014-12-08T11:43:18.1246042-05:00: Request timed out.
2014-12-08T11:43:18.1246042-05:00:
2014-12-08T11:43:18.1246042-05:00: Ping statistics for 192.168.1.1:
2014-12-08T11:43:18.1246042-05:00: Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),
For anyone that is looking for more information on filter, here is the documentation. It was surprising difficult to find since searching any combination of the word "filter" and "powershell" will give a million examples and no documentation. Also help filter in powershell provides no obvious help either.
The answer provided by mjolinor is the best way to do something like this, but I wanted to expand on it.
filter timestamp {"$(Get-Date): $_"}
Is a shortcut for calling this
function timestamp { Process{"$(Get-Date): $_"} }
Both of these create named functions that accept input from the pipeline. Run help pipline in powershell to learn more. The pipeline will operate on a single object at a time, and that object can be referred to with the automatic variable $_. So each function will iterate over every item in the pipeline that was piped to it using the | pipe character.
This behaves differently than a normal function in that it's working on the objects as they arrive, instead of all at once. For example running
function timestamp {
"$(Get-Date): $input"
}
$result = & ping 127.0.0.1
$result | timestamp
Would dump the entire $result object on a single line and result in a response that looks like this
03/14/2018 15:23:16: Pinging 127.0.0.1 with 32 bytes of data: Reply from 127.0.0.1: b
ytes=32 time<1ms TTL=128 Reply from 127.0.0.1: bytes=32 time<1ms TTL=128 Reply from 12
7.0.0.1: bytes=32 time<1ms TTL=128 Reply from 127.0.0.1: bytes=32 time<1ms TTL=128 Pi
ng statistics for 127.0.0.1: Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds: Minimum = 0ms, Maximum = 0ms, Avera
ge = 0ms
Since the function is operating on the object as a whole you would have to iterate over each line. Changing it to this
function timestampfunction {
foreach ($i in $input){
"$(Get-Date): $i"
}
}
Would give you the nicely formatted
03/14/2018 15:23:16:
03/14/2018 15:23:16: Pinging 127.0.0.1 with 32 bytes of data:
03/14/2018 15:23:16: Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
03/14/2018 15:23:16: Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
03/14/2018 15:23:16: Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
03/14/2018 15:23:16: Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
03/14/2018 15:23:16:
03/14/2018 15:23:16: Ping statistics for 127.0.0.1:
03/14/2018 15:23:16: Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
03/14/2018 15:23:16: Approximate round trip times in milli-seconds:
03/14/2018 15:23:16: Minimum = 0ms, Maximum = 0ms, Average = 0ms
Here is a nicely written article on the differences between these approaches.
You can just use ForEach-Object cmdlet for it (used % alias in below example)
ping 192.168.1.1 | %{ "{0:HH:mm:ss:fff}: {1}" -f (Get-Date), $_ }
result:
23:41:51:301:
23:41:51:302: Pinging 192.168.1.1 with 32 bytes of data:
23:41:55:255: Request timed out.
23:42:00:266: Request timed out.
23:42:05:254: Request timed out.
23:42:10:253: Request timed out.
23:42:10:261:
23:42:10:263: Ping statistics for 192.168.1.1:
23:42:10:265: Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),
Or use format like in mjolinor answer:
ping 192.168.1.1 | %{ "{0:o}: {1}" -f (Get-Date), $_ }
result:
2019-04-23T23:45:40.5816185+02:00:
2019-04-23T23:45:40.5845856+02:00: Pinging 192.168.1.1 with 32 bytes of data:
2019-04-23T23:45:44.2560567+02:00: Request timed out.
2019-04-23T23:45:49.2549104+02:00: Request timed out.
2019-04-23T23:45:54.2547535+02:00: Request timed out.
2019-04-23T23:45:59.2547932+02:00: Request timed out.
2019-04-23T23:45:59.2577788+02:00:
2019-04-23T23:45:59.2607707+02:00: Ping statistics for 192.168.1.1:
2019-04-23T23:45:59.2627647+02:00: Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),
You could use the Start-Transcript cmdlet in combination with a custom prompt in your profile:
# Configure PS prompt to show date and time
function prompt
{
$promptStringStart = "PS:" + (Get-Date -format MM/dd/yy` hh:mm:ss)
$promptStringEnd += ">"
Write-Host $promptStringStart -NoNewline -ForegroundColor Yellow
Write-Host $promptStringEnd -NoNewline -ForegroundColor Yellow
return " "
}
This works great on my Windows 7 workstation. However, on some newer Server 2012 R2 installations Start-Transcript seems to be slightly broken. This fixes part of it: https://support.microsoft.com/en-us/help/3014136/powershell-transcript-file-doesn-t-contain-the-correct-information-in-windows-server-2012-r2
... but it does not fix the issue with the custom prompt, still.
For example, I see this on the console:
PS:02/14/17 08:28:20> hostname
server463
And this is what's written to the log:
PS:02/14/17 08:28:20
>
PS>hostname
server463
Related
I am trying to do a handshake with a server I downloaded from the internet. But when the client receives [SYN, ACK] it sends back a [RST]. Have no idea what is happening. Already checked the acknowledge and sequence number but everything seems ok.
In wireshark I got this:
Here is the handshake client source code:
from scapy.all import *
src_ip = "192.168.43.34"
dst_ip = "192.168.43.115"
src_port = random.randint(1024, 65535)
dst_port = 502
seq_nr = random.randint(444, 8765432)
ack_nr = 0
# Create SYN packet
ip = IP (src = src_ip, dst = dst_ip)
syn = TCP(sport = src_port, dport = dst_port, flags='S', seq = seq_nr, ack = ack_nr)
pkt_syn = ip / syn
pkt_syn.show()
# send SYN packet and receive SYN/ACK packet
print('Sending SYN')
pkt_syn_ack = sr1(pkt_syn)
print('ACK received')
pkt_syn_ack.show()
# Create the ACK packet
ack_nr = pkt_syn_ack.seq + 1
seq_nr = seq_nr + 1
ack = TCP(sport = src_port, dport = dst_port, flags = 'A', seq = seq_nr, ack = ack_nr)
send(ip / ack)
...
The problem is that your OS is receiving the SYN-ACK packet, has no idea why it was sent (as the OS itself didn't start a handshake) and reset the connection.
You can find some solutions here (for Linux)-
Unwanted RST TCP packet with Scapy
Another option is to use a different IP than the OS's, or in Windows turn off the IP stack of the used interface (only if this is the only thing that you use this interface for!)
I am getting a lot of error like below mentioned,
read tcp xx.xx.xx.xx:80: use of closed network connection
read tcp xx.xx.xx.xx:80: connection reset by peer
//function for HTTP connection
func GetResponseBytesByURL_raw(restUrl, connectionTimeOutStr, readTimeOutStr string) ([]byte, error) {
connectionTimeOut, _ /*err*/ := time.ParseDuration(connectionTimeOutStr)
readTimeOut, _ /*err*/ := time.ParseDuration(readTimeOutStr)
timeout := connectionTimeOut + readTimeOut // time.Duration((strconv.Atoi(connectionTimeOutStr) + strconv.Atoi(readTimeOutStr)))
//timeout = 200 * time.Millisecond
client := http.Client{
Timeout: timeout,
}
resp, err := client.Get(restUrl)
if nil != err {
logger.SetLog("Error GetResponseBytesByURL_raw |err: ", logs.LevelError, err)
return make([]byte, 0), err
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
return body, err
}
Update (July 14):
Server : NumCPU=8, RAM=24GB, GO=go1.4.2.linux-amd64
I am getting such error during some high traffic.
20000-30000 request per minutes, and I have a time frame of 500ms to fetch response from third party api.
netstat status from my server (using : netstat -nat | awk '{print $6}' | sort | uniq -c | sort -n) to get frequency
1 established)
1 Foreign
9 LISTEN
33 FIN_WAIT1
338 ESTABLISHED
5530 SYN_SENT
32202 TIME_WAIT
sysctl -p
**sysctl -p**
fs.file-max = 2097152
vm.swappiness = 10
vm.dirty_ratio = 60
vm.dirty_background_ratio = 2
net.ipv4.tcp_synack_retries = 2
net.ipv4.ip_local_port_range = 2000 65535
net.ipv4.tcp_rfc1337 = 1
net.ipv4.tcp_fin_timeout = 5
net.ipv4.tcp_keepalive_time = 300
net.ipv4.tcp_keepalive_probes = 5
net.ipv4.tcp_keepalive_intvl = 15
net.core.rmem_default = 31457280
net.core.rmem_max = 12582912
net.core.wmem_default = 31457280
net.core.wmem_max = 12582912
net.core.somaxconn = 65536
net.core.netdev_max_backlog = 65536
net.core.optmem_max = 25165824
net.ipv4.tcp_mem = 65536 131072 262144
net.ipv4.udp_mem = 65536 131072 262144
net.ipv4.tcp_rmem = 8192 87380 16777216
net.ipv4.udp_rmem_min = 16384
net.ipv4.tcp_wmem = 8192 65536 16777216
net.ipv4.udp_wmem_min = 16384
net.ipv4.tcp_max_tw_buckets = 1440000
net.ipv4.tcp_tw_recycle = 0
net.ipv4.tcp_tw_reuse = 1
net.ipv6.bindv6only = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.icmp_echo_ignore_broadcasts = 1
error: "net.ipv4.icmp_ignore_bogus_error_messages" is an unknown key
kernel.exec-shield = 1
kernel.randomize_va_space = 1
net.ipv4.conf.all.log_martians = 1
net.ipv4.conf.default.log_martians = 1
net.ipv4.icmp_ignore_bogus_error_responses = 1
net.ipv4.ip_forward = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.all.secure_redirects = 0
net.ipv4.conf.default.secure_redirects = 0
When making connections at a high rate over the internet, it's very likely you're going to encounter some connection problems. You can't mitigate them completely, so you may want to add retry logic around the request. The actual error type at this point probably doesn't matter, but matching the error string for use of closed network connection or connection reset by peer is about the best you can do if you want to be specific. Make sure to limit the retries with a backoff, as some systems will drop or reset connections as a way to limit request rates, and you may get more errors the faster you reconnect.
Depending on the number of remote hosts you're communicating with, you will want to increase Transport.MaxIdleConnsPerHost (the default is only 2). The fewer hosts you talk to, the higher you can set this. This will decrease the number of new connections made, and speed up the requests overall.
If you can, try the go1.5 beta. There have been a couple changes around keep-alive connections that may help reduce the number of errors you see.
I recommend implementing an exponential back off or some other rate limiting mechanism on your side of the wire. There's not really anything you can do about those error, and using exponential back off won't necessarily make you get the data any faster either. But it can ensure that you get all the data and the API you're pulling from will surely appreciate the reduced traffic. Here's a link to one I found on GitHub; https://github.com/cenkalti/backoff
There was another popular option as well though I haven't used either. Implementing one yourself isn't terribly difficult either and I could provide some sample of that on request. One thing I do recommend based off my experience is make sure you're using a retry function that has an abort channel. If you get to really long back off times then you'll want some way for the caller to kill it.
I started to write an application which will read RTP/H.264 video packets from an existing .pcap file, I need to read the packet size.
I tried to use packet->len or header->len, but it never displays the right number of bytes for packets (I'm using wireshark to verify packet size - under Length column). How to do it?
This is part of my code:
while (packet = pcap_next(handle,&header)) {
u_char *pkt_ptr = (u_char *)packet;
struct ip *ip_hdr = (struct ip *)pkt_ptr; //point to an IP header structure
struct pcap_pkthdr *pkt_hdr =(struct pcap_pkthdr *)packet;
unsigned int packet_length = pkt_hdr->len;
unsigned int ip_length = ntohs(ip_hdr->ip_len);
printf("Packet # %i IP Header length: %d bytes, Packet length: %d bytes\n",pkt_counter,ip_length,packet_length);
Packet # 0 IP Header length: 180 bytes, Packet length: 104857664 bytes
Packet # 1 IP Header length: 52 bytes, Packet length: 104857600 bytes
Packet # 2 IP Header length: 100 bytes, Packet length: 104857600 bytes
Packet # 3 IP Header length: 100 bytes, Packet length: 104857664 bytes
Packet # 4 IP Header length: 52 bytes, Packet length: 104857600 bytes
Packet # 5 IP Header length: 100 bytes, Packet length: 104857600 bytes
Another option I tried is to use:
pkt_ptr-> I get:
read_pcapfile.c:67:43: error: request for member ‘len’ in something not a structure or union
while (packet = pcap_next(handle,&header)) {
u_char *pkt_ptr = (u_char *)packet;
Don't do that; you're throwing away the const, and you really should NOT be modifying what the return value of pcap_next() points to.
struct ip *ip_hdr = (struct ip *)pkt_ptr; //point to an IP header structure
That will point to an IP header structure ONLY if pcap_datalink(handle) returns DLT_RAW, which it probably will NOT do on most devices.
If, for example, pcap_datalink(handle) returns DLT_EN10MB, packet will point to an Ethernet header (the 10MB is historical - it's used for all Ethernet speeds other than the ancient historical 3MB experimental Ethernet at Xerox, which had a different header type).
See the list of link-layer header type values for a list of the possible DLT_ types.
struct pcap_pkthdr *pkt_hdr =(struct pcap_pkthdr *)packet;
That won't work, either. The struct pcap_pkthdr for the packet is in header.
unsigned int packet_length = pkt_hdr->len;
As per my earlier comment, that won't work. Use header.len instead.
(And bear in mind that, if a "snapshot length" shorter than the maximum packet size was specified in the pcap_open_live() call, or specified in a pcap_set_snaplen() call between the pcap_create() and pcap_activate() calls, header.caplen could be less than header.len, and only header.caplen bytes of the packet data will actually be available.)
unsigned int ip_length = ntohs(ip_hdr->ip_len);
And, as per my earlier comment, that probably won't work, either.
You should be using header.len.
unsigned int packet_length = header.len;
I need to send a bunch of IP packets that I'm sure will trigger an ICMP TTL-expired error message. How exactly can I associate each error message with the packet that generated it? What field in the ICMP header is used for this?
Should I rather use some custom ID number in the original IP header, so that I can tell which error message corresponds to which packet? If so, which field is most suitable for this?
The body of ICMP TTL Expired messages must include the IP header of the original packet (which includes the source-port / destination-port) and 64 bits beyond the original header.
Based on timing and that header information, you can derive which packet triggered the TTL-expired message.
I am including a sample triggered by an NTP packet below...
See RFC 792 (Page 5) for more details.
ICMP TTL-Expired Message
Ethernet II, Src: JuniperN_c3:a0:00 (b0:c6:9a:c3:a0:00), Dst: 78:2b:cb:37:4c:7a (78:2b:cb:37:4c:7a)
Destination: 78:2b:cb:37:4c:7a (78:2b:cb:37:4c:7a)
Address: 78:2b:cb:37:4c:7a (78:2b:cb:37:4c:7a)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
Source: JuniperN_c3:a0:00 (b0:c6:9a:c3:a0:00)
Address: JuniperN_c3:a0:00 (b0:c6:9a:c3:a0:00)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
Type: IP (0x0800)
Internet Protocol, Src: 172.25.116.254 (172.25.116.254), Dst: 172.25.116.10 (172.25.116.10)
Version: 4
Header length: 20 bytes
Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)
0000 00.. = Differentiated Services Codepoint: Default (0x00)
.... ..0. = ECN-Capable Transport (ECT): 0
.... ...0 = ECN-CE: 0
Total Length: 56
Identification: 0x86d7 (34519)
Flags: 0x02 (Don't Fragment)
0.. = Reserved bit: Not Set
.1. = Don't fragment: Set
..0 = More fragments: Not Set
Fragment offset: 0
Time to live: 255
Protocol: ICMP (0x01)
Header checksum: 0xb3b1 [correct]
[Good: True]
[Bad : False]
Source: 172.25.116.254 (172.25.116.254)
Destination: 172.25.116.10 (172.25.116.10)
Internet Control Message Protocol
Type: 11 (Time-to-live exceeded)
Code: 0 (Time to live exceeded in transit)
Checksum: 0x4613 [correct]
Internet Protocol, Src: 172.25.116.10 (172.25.116.10), Dst: 172.25.0.11 (172.25.0.11)
Version: 4
Header length: 20 bytes
Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)
0000 00.. = Differentiated Services Codepoint: Default (0x00)
.... ..0. = ECN-Capable Transport (ECT): 0
.... ...0 = ECN-CE: 0
Total Length: 36
Identification: 0x0001 (1)
Flags: 0x00
0.. = Reserved bit: Not Set
.0. = Don't fragment: Not Set
..0 = More fragments: Not Set
Fragment offset: 0
Time to live: 0
[Expert Info (Note/Sequence): "Time To Live" only 0]
[Message: "Time To Live" only 0]
[Severity level: Note]
[Group: Sequence]
Protocol: UDP (0x11)
Header checksum: 0xee80 [correct]
[Good: True]
[Bad : False]
Source: 172.25.116.10 (172.25.116.10)
Destination: 172.25.0.11 (172.25.0.11)
User Datagram Protocol, Src Port: telindus (1728), Dst Port: ntp (123)
Source port: telindus (1728)
Destination port: ntp (123)
Length: 16
Checksum: 0xa7a1 [unchecked, not all data available]
[Good Checksum: False]
[Bad Checksum: False]
I need to know what the largest UDP packet I can send to another computer is without fragmentation.
This size is commonly known as the MTU (Maximum Transmission Unit). Supposedly, between 2 computers, will be many routers and modems that may have different MTUs.
I read that the TCP implementation in windows automatically finds the maximum MTU in a path.
I was also experimenting, and I found out that the maximum MTU from my computer to a server was 57712 bytes+header. Anything above that was discarded. My computer is on a LAN, isn't the MTU supposed to be around 1500 bytes?
The following doesn't answer your question directly but you might find it interesting; it says that IP packets can be disassembled/reassembled, and therefore bigger than limit on the underling media (e.g. 1500-byte Ethernet): Resolve IP Fragmentation, MTU, MSS, and PMTUD Issues with GRE and IPSEC
More on this topic:
Re: UDP fragmentation says you should use ICMP instead of UDP to discover MTU
Path MTU Discovery says that a TCP connection might include implicit MTU negotiation via ICMP
I don't know about generating ICMP via an API on Windows: at one time such an API was proposed, and was controversial because people argued that would make it easy to write software that implements denial-of-service functionality by generating a flood of ICMP messages.
No, it looks like it is implemented: see for example Winsock Programmer's FAQ Examples: Ping: Raw Sockets Method.
So, to discover MTU, generate ping packets with the 'do not fragment' flag.
Maybe there's an easier API than this, I don't know; but I hope I've given you to understand the underlying protocol[s].
In addition to all the previous answers, quoting the classic:
IPv4 and IPv6 define minimum reassembly buffer size, the minimum datagram size that we are guaranteed any implementation must support. For IPv4, this is 576 bytes. IPv6 raises this to 1,280 bytes.
This pretty much means that you want to limit your datagram size to under 576 if you work over public internet and you control only one side of the exchange - that's what most of the standard UDP-based protocols do.
Also note that PMTU is a dynamic property of the path. This is one of the things TCP deals with for you. Unless you are ready to re-implement lots of sequencing, timing, and retransmission logic, use TCP for any critical networking. Benchmark, test, profile, i.e. prove that TCP is your bottleneck, only then consider UDP.
This is an interesting topic for me. Perhaps some practical results might be of interest when delivering chunky UDP data around the real world internet via UDP, and with a transmission rate of 1 packet a second, data continues to turn up with minimal packet loss up to about 2K. Over this and you start running into issues, but regularly we delivered 1600+ bytes packets without distress - this is over GPRS mobile networks as well as WAN world wide. At ~1K assuming the signal is stable (its not!) you get low packet loss.
Interestingly its not the odd packet, but often a squall of packets for a few seconds - which presumably is why VoIP calls just collapse occasionally.
Your own MTU is available in the registry, but the MTU in practice is going to the smallest MTU in the path between your machine and the destination. Its both variable and can only be determined empirically. There are a number of RFCs showing how to determine it.
LAN's can internally have very large MTU values, since the network hardware is typically homogeneous or at least centrally administrated.
For UDP applications you must handle end-to-end MTU yourself if you want to avoid IP fragmentation or dropped packets. The recommended approach for any application is to do your best to use PMTU to pick your maximum datagram, or send datagrams < minimum PMTU
https://www.rfc-editor.org/rfc/rfc5405#section-3.2
Unicast UDP Usage Guidelines for Application Designers "SHOULD NOT send datagrams that exceed the PMTU, SHOULD discover PMTU or send datagrams < minimum PMTU
Windows appears to settings and access to PMTU information via it's basic socket options interface:
You can make sure PMTU discover is on via IP_MTU_DISCOVER, and you can read the MTU via IP_MTU.
https://learn.microsoft.com/en-us/windows/desktop/winsock/ipproto-ip-socket-options
Here's a bit of Windows PowerShell that I wrote to check for Path MTU issues. (The general technique is not too hard to implement in other programming languages.) A lot of firewalls and routers are configured to drop all ICMP by people who don't know any better. Path MTU Discovery depends on being able to receive an ICMP Destination Unreachable message with Fragementation Needed set in response to sending a packet with Don't Fragment set. The Resolve IPv4 Fragmentation, MTU, MSS, and PMTUD Issues with GRE and IPsec actually does a really good job of explaining how discovery works.
function Test-IPAddressOrName($ipAddressOrName)
{
$ipaddress = $null
$isValidIPAddressOrName = [ipaddress]::TryParse($ipAddressOrName, [ref] $ipaddress)
if ($isValidIPAddressOrName -eq $false)
{
$hasResolveDnsCommand = $null -ne (Get-Command Resolve-DnsName -ErrorAction SilentlyContinue)
if ($hasResolveDnsCommand -eq $true)
{
$dnsResult = Resolve-DnsName -DnsOnly -Name $ipAddressOrName -ErrorAction SilentlyContinue
$isValidIPAddressOrName = $null -ne $dnsResult
}
}
return $isValidIPAddressOrName
}
function Get-NameAndIPAddress($ipAddressOrName)
{
$hasResolveDnsCommand = $null -ne (Get-Command Resolve-DnsName -ErrorAction SilentlyContinue)
$ipAddress = $null
$validIPAddress = [ipaddress]::TryParse($ipAddressOrName, [ref] $ipAddress)
$nameAndIp = [PSCustomObject] #{ 'Name' = $null; 'IPAddress' = $null }
if ($validIPAddress -eq $false)
{
if ($hasResolveDnsCommand -eq $true)
{
$dnsResult = Resolve-DnsName -DnsOnly $ipAddressOrName -Type A -ErrorAction SilentlyContinue
if ($null -ne $dnsResult -and $dnsResult.QueryType -eq 'A')
{
$nameAndIp.Name = $dnsResult.Name
$nameAndIp.IPAddress = $dnsResult.IPAddress
}
else
{
Write-Error "The name $($ipAddressOrName) could not be resolved."
$nameAndIp = $null
}
}
else
{
Write-Warning "Resolve-DnsName not present. DNS resolution check skipped."
}
}
else
{
$nameAndIp.IPAddress = $ipAddress
if ($hasResolveDnsCommand -eq $true)
{
$dnsResult = Resolve-DnsName -DnsOnly $ipAddress -Type PTR -ErrorAction SilentlyContinue
if ($null -ne $dnsResult -and $dnsResult.QueryType -eq 'PTR')
{
$nameAndIp.Name = $dnsResult.NameHost
}
}
}
return $nameAndIp
}
<#
.Synopsis
Performs a series of pings (ICMP echo requests) with Don't Fragment specified to discover the path MTU (Maximum Transmission Unit).
.Description
Performs a series of pings with Don't Fragment specified to discover the path MTU (Maximum Transmission Unit). An ICMP echo request
is sent with a random payload with a payload length specified by the PayloadBytesMinimun. ICMP echo requests of increasing size are
sent until a ping response status other than Success is received. If the response status is PackeTooBig, the last successful packet
length is returned as a reliable MTU; otherwise, if the respone status is TimedOut, the same size packet is retried up to the number
of retries specified. If all of the retries have been exhausted with a response status of TimedOut, the last successful packet
length is returned as the assumed MTU.
.Parameter UseDefaultGateway
If UseDefaultGateway is specified the default gateway reported by the network interface is used as the destination host.
.Parameter DestinationHost
The IP Address or valid fully qualified DNS name of the destination host.
.Parameter InitialTimeout
The number of milliseconds to wait for an ICMP echo reply. Internally, this is doubled each time a retry occurs.
.Parameter Retries
The number of times to try the ping in the event that no reply is recieved before the timeout.
.Parameter PayloadBytesMinimum
The minimum number of bytes in the payload to use. The minimum MTU for IPv4 is 68 bytes; however, in practice, it's extremely rare
to see an MTU size less than 576 bytes so the default value is 548 bytes (576 bytes total packet size minus an ICMP header of 28
bytes).
.Parameter PayloadBytesMaximum
The maximum number of bytes in the payload to use. An IPv4 MTU for jumbo frames is 9000 bytes. The default value is 8973 bytes (9001
bytes total packet size, which is 1 byte larger than the maximum IPv4 MTU for a jumbo frame, minus an ICMP header of 28 bytes).
.Example
Discover-PathMTU -UseDefaultGateway
.Example
Discover-PathMTU -DestinationHost '192.168.1.1'
.Example
Discover-PathMTU -DestinationHost 'www.google.com'
#>
function Discover-PathMtu
{
[CmdletBinding(SupportsShouldProcess = $false)]
param
(
[Parameter(Mandatory = $true, ParameterSetName = 'DefaultGateway')]
[switch] $UseDefaultGateway,
[Parameter(Mandatory = $true, Position = 0, ValueFromPipeline = $true, ParameterSetName = 'IPAddressOrName')]
[ValidateScript({ Test-IPAddressOrName $_ })]
[string] $DestinationHost,
[Parameter(ParameterSetName = 'IPAddressOrName')]
[Parameter(ParameterSetName = 'DefaultGateway')]
[int] $InitialTimeout = 3000,
[Parameter(ParameterSetName = 'IPAddressOrName')]
[Parameter(ParameterSetName = 'DefaultGateway')]
[int] $Retries = 3,
[Parameter(ParameterSetName = 'IPAddressOrName')]
[Parameter(ParameterSetName = 'DefaultGateway')]
$PayloadBytesMinimum = 548,
[Parameter(ParameterSetName = 'IPAddressOrName')]
[Parameter(ParameterSetName = 'DefaultGateway')]
$PayloadBytesMaximum = 8973
)
begin
{
$ipConfiguration = Get-NetIPConfiguration -Detailed | ?{ $_.NetProfile.Ipv4Connectivity -eq 'Internet' -and $_.NetAdapter.Status -eq 'Up' } | Sort { $_.IPv4DefaultGateway.InterfaceMetric } | Select -First 1
$gatewayIPAddress = $ipConfiguration.IPv4DefaultGateway.NextHop
$pingOptions = New-Object System.Net.NetworkInformation.PingOptions
$pingOptions.DontFragment = $true
$pinger = New-Object System.Net.NetworkInformation.Ping
$rng = New-Object System.Security.Cryptography.RNGCryptoServiceProvider
}
process
{
$pingIpAddress = $null
if ($UseDefaultGateway -eq $true)
{
$DestinationHost = $gatewayIPAddress
}
$nameAndIP = Get-NameAndIPAddress $DestinationHost
if ($null -ne $nameAndIP)
{
Write-Host "Performing Path MTU discovery for $($nameAndIP.Name) $($nameAndIP.IPAddress)..."
$pingReply = $null
$payloadLength = $PayloadBytesMinimum
$workingPingTimeout = $InitialTimeout
do
{
$payloadLength++
# Use a random payload to prevent compression in the path from potentially causing a false MTU report.
[byte[]] $payloadBuffer = (,0x00 * $payloadLength)
$rng.GetBytes($payloadBuffer)
$pingCount = 1
do
{
$pingReply = $pinger.Send($nameAndIP.IPAddress, $workingPingTimeout, $payloadBuffer, $pingOptions)
if ($pingReply.Status -notin 'Success', 'PacketTooBig', 'TimedOut')
{
Write-Warning "An unexpected ping reply status, $($pingReply.Status), was received in $($pingReply.RoundtripTime) milliseconds on attempt $($pingCount)."
}
elseif ($pingReply.Status -eq 'TimedOut')
{
Write-Warning "The ping request timed out while testing a packet of size $($payloadLength + 28) using a timeout value of $($workingPingTimeout) milliseconds on attempt $($pingCount)."
$workingPingTimeout = $workingPingTimeout * 2
}
else
{
Write-Verbose "Testing packet of size $($payloadLength + 28). The reply was $($pingReply.Status) and was received in $($pingReply.RoundtripTime) milliseconds on attempt $($pingCount)."
$workingPingTimeout = $InitialTimeout
}
Sleep -Milliseconds 10
$pingCount++
} while ($pingReply.Status -eq 'TimedOut' -and $pingCount -le $Retries)
} while ($payloadLength -lt $PayloadBytesMaximum -and $pingReply -ne $null -and $pingReply.Status -eq 'Success')
if ($pingReply.Status -eq 'PacketTooBig')
{
Write-Host "Reported IPv4 MTU is $($ipConfiguration.NetIPv4Interface.NlMtu). The discovered IPv4 MTU is $($payloadLength + 27)."
}
elseif ($pingReply.Status -eq 'TimedOut')
{
Write-Host "Reported IPv4 MTU is $($ipConfiguration.NetIPv4Interface.NlMtu). The discovered IPv4 MTU is $($payloadLength + 27), but may not be reliable because the packet appears to have been discarded."
}
else
{
Write-Host "Reported IPv4 MTU is $($ipConfiguration.NetIPv4Interface.NlMtu). The discovered IPv4 MTU is $($payloadLength + 27), but may not be reliable, due to an unexpected ping reply status."
}
return $payloadLength + 27
}
else
{
Write-Error "The name $($DestinationHost) could not be resolved. No Path MTU discovery will be performed."
}
}
end
{
if ($null -ne $pinger)
{
$pinger.Dispose()
}
if ($null -ne $rng)
{
$rng.Dispose()
}
}
}