Rate-Limited Network Port Scanner Pipeline

What this fixes

Fast scanners are easy to misuse:

• Too-fast rates can overwhelm your own network links or trip defenses.
• Wide ranges without exclusions can hit out-of-scope addresses.
• "Discovery" results often need verification before they are report-ready.

This recipe standardizes the pipeline: discover → dedupe → verify → export.

Prerequisites

• Approved target CIDRs and an explicit exclusion list (out-of-scope ranges)
• A scanning host with stable network connectivity
• `masscan` and `nmap` installed (or available via containers)
• Permission to store scan artifacts securely

Steps and commands

1. Build an exclude file (one CIDR or IP per line):

`cat > exclude.txt << 'EOF'

10.0.0.0/8

192.168.0.0/16

EOF`

1. MASSCAN discovery (rate-limited; output JSON or XML):

• Conservative example (internal):

`sudo masscan -p80,443,445,3389 --rate 1000 --excludefile exclude.txt -oJ masscan.json`

• Full port range (slower, safer rate):

`sudo masscan -p0-65535 --rate 500 --excludefile exclude.txt -oX masscan.xml`

1. Parse ports per host and generate a verification target list:

• Quick JSON extract (jq):

`jq -r '.[] | select(.ports) | "\(.ip):\(.ports[].port)"' masscan.json | sort -u > ip_port.txt`

• Extract unique hosts:

`jq -r '.[] | select(.ports) | .ip' masscan.json | sort -u > hosts.txt`

1. Verify with Nmap (service/version detection):

• Single host:

`nmap -sV -Pn -n -p -oX nmap-.xml`

• Many hosts with reasonable timing:

`nmap -sV -Pn -n -iL hosts.txt --top-ports 1000 -T3 -oX nmap.xml`

1. Export report-friendly output:

• HTML from Nmap XML:

`xsltproc nmap.xml -o nmap.html`

Expected outputs

• `masscan.json` / `masscan.xml`: discovery results (host:port)
• `hosts.txt` / `ip_port.txt`: parsed target lists for verification
• `nmap.xml` (and optional `nmap.html`): verified ports + service fingerprints
• A short audit note: scope + rate + exclusions + timestamps

Common errors and troubleshooting

• Network "melting"/packet loss
• Lower `--rate` and limit ports.
• Scan fewer targets at once; prefer change windows.

• Too many false positives
• Always verify a sample with Nmap before declaring findings.
• Check whether you are hitting load balancers or tarpits.

• Out-of-scope risk
• Use `--excludefile` and store scope files with the engagement artifacts.
• If scope is hostname-based, resolve first and scan only resolved IPs.

References

• https://github.com/robertdavidgraham/masscan
• https://github.com/robertdavidgraham/masscan#how-to-scan-the-entire-internet
• https://news.ycombinator.com/item?id=24728123
• https://nmap.org/book/man-misc-options.html

Example inputs

• ``: 172.16.0.0/16
• Target ports: 80,443,445,3389
• Rate: 500–2000 pps (start low; increase only if safe)