I'm trying to scan a network by rotating source port and targeting one single destination port as per:
for source_port in `seq $start $end`
do
nmap -sS -p $dest_port --source-port $source_port -P0 $dest_ip
done
the problem is that I'm launching a nmap process every time
nmap seem design to rotate destination ports easily at command line, but for source port, I cannot find a way to specify a range even by using an nmap script (don't know how to access source-port variable in the script)
anyone has an idea?
Thanks
For this task, you may want to use a packet-crafting framework like scapy or PacketFu, since these will let you specify exactly what type of packets to send without requiring a new process to be launched each time.
You could also use a packet-crafting tool like Nping, hping3, or nemesis, but these will also require launching a new process for each probe (as far as I know).
You could do something similar in an Nmap script by using socket:bind to choose a source port for each connect, but you'd have to do a lot of tweaking to get anywhere near the performance of a full Nmap port scan. First, you'd have to parallelize it to send more than one packet at a time, but then you might overload the target or the network and run into congestion problems. Still, it might be worth a shot with a small number (20 or so) of worker threads to speed things up.
To speed up your current method, use -n to skip reverse name resolution. You can parallelize it by starting some of your nmap processes in the background or by using GNU Parallel. You could try to reduce the number of retries with --max-retries, since the default maximum is 10, but a quick test here shows that with -Pn and a single port, Nmap only tries 2 times to reach the port. If you want to get really fancy, you can use --initial-rtt-timeout and --max-rtt-timeout since Nmap is deliberately conservative when it starts, assuming that it will send many packets and get a better idea of network conditions. Scanning one port on one host without host discovery means that it doesn't get a chance to tune itself. If you run ping -c 10 against your target, you can use its last line of output to set these values. For example, if I see:
$ ping -c 10 scanme.nmap.com
PING scanme.nmap.com (74.207.244.221) 56(84) bytes of data.
64 bytes from scanme.nmap.org (74.207.244.221): icmp_req=1 ttl=56 time=75.0 ms
64 bytes from scanme.nmap.org (74.207.244.221): icmp_req=2 ttl=56 time=75.5 ms
64 bytes from scanme.nmap.org (74.207.244.221): icmp_req=3 ttl=56 time=73.9 ms
64 bytes from scanme.nmap.org (74.207.244.221): icmp_req=4 ttl=56 time=75.4 ms
64 bytes from scanme.nmap.org (74.207.244.221): icmp_req=5 ttl=56 time=74.7 ms
64 bytes from scanme.nmap.org (74.207.244.221): icmp_req=6 ttl=56 time=78.9 ms
64 bytes from scanme.nmap.org (74.207.244.221): icmp_req=7 ttl=56 time=73.6 ms
64 bytes from scanme.nmap.org (74.207.244.221): icmp_req=8 ttl=56 time=79.4 ms
64 bytes from scanme.nmap.org (74.207.244.221): icmp_req=9 ttl=56 time=74.8 ms
64 bytes from scanme.nmap.org (74.207.244.221): icmp_req=10 ttl=56 time=75.1 ms
--- scanme.nmap.com ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 9012ms
rtt min/avg/max/mdev = 73.646/75.691/79.459/1.857 ms
then I can set --initial-rtt-timeout to avg + 4 * mdev = 83ms to give myself a nice wide margin of 4 standard deviations around the average round trip time. Nmap's default is to wait 1000ms (1 second), so this is quite a big speedup. Set the --max-rtt-timeout to 10 times the initial value just in case.
Related
I am using ::http::geturl -query to issue an HTTP POST request with a small json payload to an ESP8266 (a 3rd party commercial device) from a rPi. It works when sent over eth0 but fails when sent over wlan0. tcpdump shows that sent over eth0, the message is sent as a single packet but when sent over wlan0 the payload is being split from the headers and sent in a second packet. The ESP8266 most likely due to having overly simple implementation of its packet receivers and/or http server doesn't appear to handle this splitting. It issues a 200 OK response after receiving the packet containing the headers and doesn't process the payload part of the request.
Experimentally I composed the same request message text being sent by ::http::geturl and sent it over wlan0 using nc; it was sent as a single packet and was successfully processed by the ESP8266.
Does anyone happen to know why sending the request using ::http over wlan0 is ending up with this split message, and what if anything can be done to prevent it?
Code fragment:
set s [::http::geturl http://$ip/con?com=cli -query $data -type application/json]
set r [::http::ncode $s]
::http::cleanup $s
Raspbian package versions:
tcl8.6 8.6.9+dfsg-2
tcllib 1.19-dfsg-2
tcl_platform(engine) = Tcl
tcl_platform(machine) = armv7l
tcl_platform(os) = Linux
tcl_platform(osVersion) = 5.4.79-v7+
$ ifconfig wlan0
wlan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.101 netmask 255.255.255.0 broadcast 192.168.0.255
inet6 fe80::ed38:71ab:13af:ae30 prefixlen 64 scopeid 0x20<link>
ether b8:27:eb:26:bf:94 txqueuelen 1000 (Ethernet)
From /proc/cpuinfo:
Hardware : BCM2835
Revision : a020d3
Model : Raspberry Pi 3 Model B Plus Rev 1.3
$ uname -a
Linux raspberrypi 5.4.79-v7+ #1373 SMP Mon Nov 23 13:22:33 GMT 2020 armv7l GNU/Linux
Tcl's http package flushes the headers to the socket (i.e., performs an actual write()/send()) between writing the headers and the body of a query. For any correct implementation of an HTTP server this is fineā¦ but you're not working with that. For some reason, the wlan and eth drivers in the OS kernel have different policies for what to do with that case, with the eth driver deciding to wait a bit before sending; Tcl definitely doesn't configure this aspect of sockets at all, staying with the system defaults. (I don't know how to configure the OS defaults.)
You can always take a copy of the http code and comment out the flush. It's this one:
https://core.tcl-lang.org/tcl/artifact/d9f8dc4bd7211a37?ln=1463
line 1463: flush $sock
There's a Download button/link at the top of the page for that exact version of that file (it's changed only very slightly from the one in your version of Tcl and should be compatible provided you source the file explicitly before doing any package require calls).
I am running a traceroute command to host [YYY.YYY.Y.Y], and at the output is as follows:
...
e1-25-20-cr-rt-bb2-area2.ns.uwaterloo.ca (ZZZ.ZZ.ZZ.ZZ) 4.164 ms 6.741 ms 6.444 ms
8 wms.uwaterloo.ca (MMM.MM.MMM.MM) 3.034 ms 3.612 ms 3.334 ms
9 wms.uwaterloo.ca (MMM.MM.MMM.MM) 3.802 ms !N 5.354 ms !N 5.588 ms !N
Several sources have told me that !N means that the destination host is unreachable. However, this seems to be the case when the output is as follows:
9 wms.uwaterloo.ca (129.97.208.23) !N !N !N
What does it mean when !N appears beside an elapsed time ?
I don't think I should be interpreting {elapsed_time} !N as destination unreachable since I was definitely able to reach my target destination through a web browser and through "ping".
wms.uwaterloo.ca sends back an ICMP Destination network unreachable message. Some protocols and port used by your traceroute are filtered. If you can create a packet capture then you will see and understand what happens under the hood.
I installed Ubuntu 18.04 on Hyper-V Win Server 2016.
And network performance of the Ubuntu is bad: I'm hosting few sites (Apache + PHP) and sometime response time is > 10 seconds. Sometimes it is fast.
As I troubleshooted, I see this netstat results:
# netstat -s | egrep -i 'loss|retran'
3447700 segments retransmitted
226 times recovered from packet loss due to fast retransmit
Detected reordering 6 times using reno fast retransmit
TCPLostRetransmit: 79831
45 timeouts after reno fast retransmit
6247 timeouts in loss state
2056435 fast retransmits
107095 retransmits in slow start
TCPLossProbes: 220607
TCPLossProbeRecovery: 3753
TCPSynRetrans: 90564
What can be cause of such high "segments retransmitted" number? And how to fix it?
Few notes:
- VMQ is disabled for Ubuntu VM
- The host system Network adapter is Intel I210
- I disabled IPv6 both on host and in VM
Here is WireShark showing, that it takes ~7 seconds to connect (just initial connection) to my site Propovednik.com:
Sep 20: So far, the issue seems to be caused by OVH / SoYouStart bad network:
This command shows 20-30% packets loss:
sudo ping us.soyoustart.com -c 10 -i 0.2 -p 00 -s 1200 -l 5
The problem could be anywhere along the network, including the workstation where you work from. I suggest you check the network as retransmissions and packetloss means that either something is malfunctioning or misconfigured. If this is on a wireless network, you could be out of range of your router.
I am pinging the website you noted from my computer and there is no packetloss.
Say I want to timestamp a command like ping google.com so that the output will look something like
[9:48:56]PING google.com (116.28.94.14): 56 data bytes
[9:48:55]64 bytes from 116.28.94.14: icmp_seq=0 ttl=54 time=13.118 ms
[9:48:57]64 bytes from 116.28.94.14: icmp_seq=1 ttl=54 time=13.943 ms
[9:48:58]64 bytes from 116.28.94.14: icmp_seq=2 ttl=54 time=19.103 ms
[9:48:59]64 bytes from 116.28.94.14: icmp_seq=3 ttl=54 time=12.854 ms
I've looked into fancy ways to use pipes and commands like xarg and awk, but I just can't get this right. Also, let's assume the environment I have doesn't allow me to use fancy things like stdbuff or unbuff. If possible, it'd be awesome to achieve this with the most unix-like approach possible. Thank you!
Here is the absolute simplest and stupidest way to do it:
$ while read -r line ; do echo "$(date "+%H:%M:%S"): $line" ; done < <(while true ; do echo foo ; sleep 1 ; done)
00:07:08: foo
00:07:09: foo
00:07:10: foo
00:07:11: foo
00:07:12: foo
You can format the timestamp as you desire and run any command you'd like:
$ while read -r line ; do echo "$(date "+%s"): $line" ; done < <(ping google.com)
1472533833: PING google.com (216.58.216.78): 56 data bytes
1472533833: 64 bytes from 216.58.216.78: icmp_seq=0 ttl=55 time=11.391 ms
1472533834: 64 bytes from 216.58.216.78: icmp_seq=1 ttl=55 time=12.746 ms
1472533835: 64 bytes from 216.58.216.78: icmp_seq=2 ttl=55 time=10.912 ms
1472533836: 64 bytes from 216.58.216.78: icmp_seq=3 ttl=55 time=13.065 ms
1472533837: 64 bytes from 216.58.216.78: icmp_seq=4 ttl=55 time=10.684 ms
Here it as a shell function:
function logger {
while read -r line ; do
echo "$(date "+%H:%M:%S"): $line"
done < <($*)
}
Weird commands and command output might break it, so some additional tweaking is likely required, but hopefully this is a good start for you.
EDIT: Some quick Googling turns up a thread with some great answers here.
The ts command (for "timestamp") in moreutils1 was made exactly for this. To add timestamps to ping output formatted like your example you can use
ping google.com | ts '[%H:%M:%S]'
I don't know if this falls under "fancy ways to use pipes", but I'd say that it is a good unix-way of doing it in the sense that you're using a tool that does one thing and does it well.
1You can install moreutils in Debian/Ubuntu, for example, with sudo apt-get install moreutils.
This is strange. How this actually works. So far I know it is "impossible" to have a network like this.
I'm going to explain in details how my network works.
I have a LAN. 192.168.1.0/24 and router is on 192.168.1.1, This router has a public address.
I can share the IP address because I'm running there a server for tests nothing more. It is ok so far.
Now the magic happens.
When I trace the route to an IP I got this (To google DNS):
traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
1 zonhub.home (192.168.1.1) 1.160 ms 1.676 ms 1.340 ms
2 * * *
3 10.137.211.97 (10.137.211.97) 12.915 ms 12.526 ms 12.145 ms
4 10.255.49.90 (10.255.49.90) 10.349 ms 10.255.49.102 (10.255.49.102) 11.483 ms 11.042 ms
5 80.157.128.249 (80.157.128.249) 34.577 ms 80.157.130.41 (80.157.130.41) 32.917 ms 80.157.130.33 (80.157.130.33) 30.602 ms
6 mad-sa3-i.MAD.ES.NET.DTAG.DE (217.5.95.161) 33.396 ms 80.157.128.22 (80.157.128.22) 27.107 ms mad-sa3-i.MAD.ES.NET.DTAG.DE (217.5.95.161) 29.510 ms
7 80.157.128.22 (80.157.128.22) 28.050 ms 72.14.235.20 (72.14.235.20) 32.767 ms 80.157.128.22 (80.157.128.22) 27.932 ms
8 72.14.235.20 (72.14.235.20) 29.780 ms 72.14.235.18 (72.14.235.18) 27.020 ms 26.706 ms
9 216.239.43.233 (216.239.43.233) 49.456 ms 209.85.240.191 (209.85.240.191) 44.034 ms 216.239.43.233 (216.239.43.233) 51.935 ms
10 72.14.236.191 (72.14.236.191) 53.374 ms 209.85.253.20 (209.85.253.20) 50.699 ms 216.239.43.233 (216.239.43.233) 44.918 ms
11 209.85.251.231 (209.85.251.231) 50.151 ms * 216.239.49.45 (216.239.49.45) 47.309 ms
12 google-public-dns-a.google.com (8.8.8.8) 51.536 ms 50.180 ms 45.505 ms
What is that 2nd, 3rd and 4th hop? How can it be on class A private address when 192.168.1.1 is running the NAT service where I have my LAN and my external 3 publics addresses (yes I have 3 and is 3 "class A" IP's on 88,89,93 network).
Another thing is how on 4th hop we have the 2nd octet 255?
Anyone feel free to traceroute my no-ip domain: synackfiles.no-ip.org
Just don't mess with my router (it blocks if you port scan or fail to log in in ssh or http auth so you get banned for this. If you just traceroute it is fine) :P
Now, second magic and weird stuff happens.
I'm going to run nmap. So i GOT THIS:
sudo nmap -sV -A -O 10.137.211.113 -vv -p 1-500 -Pn
Starting Nmap 6.00 ( http://nmap.org ) at 2013-11-14 15:24 WET
NSE: Loaded 93 scripts for scanning.
NSE: Script Pre-scanning.
NSE: Starting runlevel 1 (of 2) scan.
NSE: Starting runlevel 2 (of 2) scan.
Initiating Parallel DNS resolution of 1 host. at 15:24
Completed Parallel DNS resolution of 1 host. at 15:24, 0.04s elapsed
Initiating SYN Stealth Scan at 15:24
Scanning 10.137.211.113 [500 ports]
SYN Stealth Scan Timing: About 30.40% done; ETC: 15:26 (0:01:11 remaining)
SYN Stealth Scan Timing: About 60.30% done; ETC: 15:26 (0:00:40 remaining)
Completed SYN Stealth Scan at 15:26, 101.14s elapsed (500 total ports)
Initiating Service scan at 15:26
Initiating OS detection (try #1) against 10.137.211.113
Initiating Traceroute at 15:26
Completed Traceroute at 15:26, 9.05s elapsed
Initiating Parallel DNS resolution of 1 host. at 15:26
Completed Parallel DNS resolution of 1 host. at 15:26, 0.01s elapsed
NSE: Script scanning 10.137.211.113.
NSE: Starting runlevel 1 (of 2) scan.
Initiating NSE at 15:26
Completed NSE at 15:26, 0.00s elapsed
NSE: Starting runlevel 2 (of 2) scan.
Nmap scan report for 10.137.211.113
Host is up (0.0010s latency).
All 500 scanned ports on 10.137.211.113 are filtered
Device type: general purpose|specialized|media device
Running: Barrelfish, Microsoft Windows 2003|PocketPC/CE|XP, Novell NetWare 3.X, Siemens embedded, Telekom embedded
OS CPE: cpe:/o:barrelfish:barrelfish cpe:/o:microsoft:windows_server_2003::sp1 cpe:/o:microsoft:windows_server_2003::sp2 cpe:/o:microsoft:windows_ce cpe:/o:microsoft:windows_xp:::professional cpe:/o:novell:netware:3.12
Too many fingerprints match this host to give specific OS details
TCP/IP fingerprint:
SCAN(V=6.00%E=4%D=11/14%OT=%CT=%CU=%PV=Y%G=N%TM=5284EBAB%P=armv7l-unknown-linux-gnueabi)
T7(R=Y%DF=N%TG=80%W=0%S=Z%A=S+%F=AR%O=%RD=0%Q=R)
U1(R=N)
IE(R=N)
TRACEROUTE (using proto 1/icmp)
HOP RTT ADDRESS
1 2.32 ms zonhub.home (192.168.1.1)
2 ... 30
NSE: Script Post-scanning.
NSE: Starting runlevel 1 (of 2) scan.
NSE: Starting runlevel 2 (of 2) scan.
Read data files from: /usr/bin/../share/nmap
OS and Service detection performed. Please report any incorrect results at http://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 122.65 seconds
Raw packets sent: 1109 (49.620KB) | Rcvd: 4 (200B)
Well, this is odd. I don't know how my country's WAN is designed and built.
I'm from Portugal and my ISP is "ZON TVCABO". You can search now. :P
This is very very very interesting..
Sincerely,
int3
I cannot tell you how your providers WAN is built - but in order to
save public IPs - one can design an ISPs internal network with private
IPs. The routers that are not needed to be available from public will
have private IPs only - the IPs assigned to you can be routed to your
uplink over routers that are ISP internally only.
the 2nd hop has no tracing allowed, but allows to forward them.
the 4th - 10.255.x.x is a private IP in the 10.0.0.0/8 A range. (you
can use numbers from 0-255)