TODO $Id: TODO 11866 2009-01-24 23:10:05Z fyodor $ -*-text-*- o Finish the version detection submission integration o Do the very latest Nmap IPv4 OS detection (last was done with snapshot from May). o Make sure the new version detection sigs have appropriate CPE’s. o Integrate latest IPv6 OS detection submissions and corrections o Make Windows 8.1 VM with VS 2013 and do more testing of Nmap compilation/running o Make and test build on a newer OS X than 10.6 (10.10 was recently released) o Deal with our out-of-date CA root certificate bundle by either using OS-specific mechanisms and/or updating the latest from Mozilla or another source. See http://seclists.org/nmap-dev/2014/q4/200 o Change Ncat so that it does SSL certificate trust checking by default (even without --ssl-verify) and provides a warning and the key fingerprint if there is no valid trusted chain or the cert is expired, etc. The warning should happen (to STDERR) even if -v is not specified. We should add a new option to force Ncat to quit if cert not valid, and --ssl-verify should become an undocumented alias for that. o Figure out what nmap-update is doing for SSL certificate verification (it uses libsvn to our SSL svn server). o Audit ncat's ssl algorithm and ciphersuite choices o Do a test/beta release (more, if necessary) o Make sure people have tested on Mac OS 10.10 o Do CHANGELOG for new release[Fyodor] o Web updates for new release o Build and post new release ==Items we need to finish before next big release go above this line== o Make Ncat avoid linking with libpcap even when it's available. Currently this conflicts with how Nmap needs to configure Nsock. One way ahead would be building Nsock separately for each project in different build directories. o Add a --defeat-icmp-ratelimit (or similar) argument to modify UDP scans. This would only find open and responsive ports, ignoring open|filtered. See http://seclists.org/nmap-dev/2014/q4/183 o Work on Nmap on Mobile devices, particularly Android. Would be great to get it in Google Play store, for example. An official version with a workable GUI. For now, people have to do manual work and it isn't as well tested either: https://secwiki.org/w/Nmap/Android . If this is successful, we could consider iOS. o Nmap performance work. Particularly with --min-rate. o Consider re-architecting Nmap to have more of a scanning pipeline approach rather than fixed sets of hosts which start and finish one phase and then move into the next in parallel. This could potentially allow us to add hosts one by one to a phase as other hosts finish that phase and, ideally, the phases could run in parallel too. o Nmap Network Scanning, 2nd Edition work [placeholder] o Organize nselib into a hierarchy. A library "dirname/filename.lua" can be required as "dirname.filename". We would need to ensure the installers (Makefile, OS X, Windows, RPM) can handle this. See http://seclists.org/nmap-dev/2014/q3/364 o We should work to reduce Zenmap's memory consumption. We used to commonly get error reports from people who load so many systems that Zenmap gives an out of memory error and crashes. For example, see this thread: http://seclists.org/nmap-dev/2014/q2/46 After committing patch at http://seclists.org/nmap-dev/2014/q2/429, we no longer get the error report but the problem still exists. The problem seems to lie in a very large Nmap Output being stored in memory and a possible fix seems to be to use a file based paging system. o Consider making a version of Nmap for Apple's official Mac App Store. A particular concern with the downloadable Mac version of Nmap is that Apple's new "Mountain Lion" release may require users to jump through hoops to install unsigned non-app-store content per their "Gatekeeper" "feature". Though maybe signing the app will be enough. There may also be an issue with the "Sandboxing" requirement for App Store apps starting June 2012. Will Nmap be able to request all the permissions it needs? Ignoring the technical challenges for the moment, what will users prefer? o Do a roll up on (state, TTL) pair instead of just state so that TTL info is not lost when doing roll up on port states. See thread at http://seclists.org/nmap-dev/2014/q3/93 o Consider doing TTL analysis (using NSE script) to detect if a firewall might exist in front of a host. See thread at http://seclists.org/nmap-dev/2014/q3/171 o Consider looking into differring TTL values during OS detection phase and choose a port that is (hopefully) not firewalled to get a better chance at correct result. See thread at http://seclists.org/nmap-dev/2014/q3/33 o [Zenmap] Look into and refactor code which uses the (very slow) += operation on strings. http://seclists.org/nmap-dev/2014/q2/432 helped improve speeds for opening files (from hours to seconds) and it seems like more speedups can be done in other places. o Consider using a binary decision diagram for --exclude list to make it more efficient for large exclude lists. See http://seclists.org/nmap-dev/2012/q4/420. o Augment the configure script to list unmet dependencies. Currently, configure works just fine without a C++ compiler installed, but make generates an error. The configure script should be able to detect this. Also, a list of features that are/are-not available would be nice at the end of the script, so folks can see that they've e.g. missed the OpenSSL dependency. o Integrate latest version detection submissions and corrections o Look into moving our Mac building/testing system into a virtual machine or leased server sort of environment so that multiple Nmap developers can access it and nobody has to keep a stack of Mac Minis in their closet. o INFRASTRUCTURE: Upgrade seclists to use Mailman 3, which apparently has many improvements. o We should fix nsedoc generation so it doesn't fail when blocks like @usage, @output, etc. are followed by a local declaration. See http://seclists.org/nmap-dev/2014/q2/331. If for some reason this just can't be fixed, we will have to document the heck out of it, I suppose. o When scanning your own IP from Windows, Nmap currently recognizes the problem (can't do a raw scan like that on Windows) and skips the SYN scan, leading to Nmap printing a bunch of ports in "unknown" state at the end. Nmap should probably act like unprivileged mode in this case (e.g. do a connect scan, etc.). See http://seclists.org/nmap-dev/2013/q3/519 o Investigate Checkmarx static analysis report of Nmap source tree that someone sent us on Feb 12. It looks like mostly false positives, but we should go through to check for any real bugs or even possible security issues. Fyodor has the report. o Make CONCURRENCY_LIMIT in nse_main.lua at least the min-parallelism. Otherwise NSE is limited to 1000 socket-using threads even if you've requested more. o INFRASTRUCTURE: Add IPv6 support to secwiki - We probably just have to designate a new IPv6 address for it and add it to Apache config. o INFRASTRUCTURE: Consider updating our svn-mailer.py (and conf file) to the latest official version. First check whether there is a later official version and whether it has material changes. We're currently using one from subversion-1.4.2/tools/hook-scripts/mailer/mailer.py. o Consider a two-stage model for IPv6 subnet/pattern support o Right now you can try to scan a /64, for example, and Nmap will try to iterate through them all (and of course never complete). So perhaps Nmap should first look at a specification and decide if it should use other techniques like multicast discovery instead. o Move advanced IPv6 host discovery features from NSE into core Nmap. We'll probably add the functionality of targets-ipv6-multicast-invalid-dst, targets-ipv6-multicast-echo, and maybe targets-ipv6-multicast-slaac. - The idea is that Nmap does them automatically if it gets a large target specification and sees that it is local so can be multicast pinged. o We should figure out why (at least with Nping) raw ethernet frame sends seem to be taking significantly longer than raw socket sends (e.g. using --send-ip or the OS-provided ping utility). This has been reproduced on Linux and Windows. Here's a thread: http://seclists.org/nmap-dev/2012/q4/424 o Note that David and I tried to reproduce this on his machine and on 'web' and 'research' machines and could not reproduce. Still happens with Fyodor's machine connected with WiFi. Fyodor should test on the same machine using wired and see if that changes anything. o Consider a continuous integration system for automating tests of Nmap on various platforms to detect and resolve issues more quickly. Things to look at include the Supercell cluster (provides VMs of various hardware/software for testing) and projects like Buildbot, Travis, Hudson, Jenkins, etc. o Implement some improvements to dns-ip6-arpa.nse, as describe at http://seclists.org/nmap-dev/2012/q2/45. - Also consider a move to "fire and forget" logic. Just blast out the queries that we know we have to make, and then read any replies that may happen to come back. (but still try not to introduce inaccuracy (missed hosts) by flooding the network. o We should fix service detection so it can handle 0-byte captures without crashing. See http://seclists.org/nmap-dev/2014/q2/105 o Our http library should allow the client to specify a max size in advance and should probably enforce some sort of maximum by default (unless turned off by the script). That way sites can't DoS Nmap by feeding enormous files. o NSE digest auth should use the more robust parsing from http.parse_www_authenticate as described at http://seclists.org/nmap-dev/2012/q3/868 o Treat the input to the escape function in xml.cc as UTF-8, not just ASCII. Good UTF-8 should survive into the output; i.e., "\xe2\x98\xbb" should become "\xe2\x98\xbb" in the output, not "☻". If the input happens not to be UTF-8, (like the file name in http://seclists.org/nmap-dev/2013/q1/180), I suppose we can individually encode each byte of each invalid sequence: "\xba\xda\xbf" becomes "ºÚ¿". Can probably do this with simple byte->rune and rune->byte functions as in http://plan9.bell-labs.com/sys/doc/utf.html. o We should probably redo the Nmap header (e.g. on http://nmap.org) to make it more attractive. Or, at a minimum we should update the screenshots and think about which links we really need (some of those pages aren't really updated any more). o Investigate ways to limit Winpcap privileges so that only administrative users or a certain accounts can sniff. Maybe there is a solution people use for Wireshark or does it always cause this issue (allowing any user to sniff the network) when it is installed? - CACE says they will add a feature to do this. See this thread: http://seclists.org/nmap-dev/2010/q3/826 [ This might be a good one for Sean or James if they feel that they have the low-level Windows driver programming experience. The idea is to produce a patch that we can then try to convince the WinPcap folks to apply ] o Test a hierarchical classifier for IPv6 OS detection. Our classifier currently treats, for example, some localhost Linux fingerprints as separate classes from remote Linux fingerprints, simply because we lose precision if we lump them together (for example TCP window size differs across certain Linux versions when measured remotely, but not on localhost). This leads to the linear classifier having to use narrow margins between fingerprints that are really very similar. I want to try a tree of classification where each non-leaf node is a separately trained classifier and each leaf node is a final classification. The first layer of the hierarchy would be something like (linux windows solaris aix ... other) where "linux" would contain *all* the Linux fingerprints in a single class. Lower levels would be like (linux-2.4 linux-2.6) (windows-xp windows-vista windows-7) Lower levels will include only those fingerprints in their parent class, so we don't even think about Windows when classifying Linux. Probably three or four levels will be sufficient. There may be a principled or automatic way to build this hierarchy, but I suspect playing it by ear will be sufficient. Talk to David for more of his thinking on this topic. o [INFRASTRUCTURE] Improve our main web server http configuration to better handle high load situations and DoS attacks. As part of this, we may have to raise the max client limits. But then there is a risk of running out of RAM, which can be even worse. So we need to figure out a good balance. o Maybe we should rename dns-brute to dns-brute-enum since it is so different from our traditional brute force authentication cracking -brute scripts? o Investigate WinPcap support for NDIS 6. - Preferably the CACE/WinPcap folks will do this - We need to retain support for Windows back to Windows XP SP3 -- I'm not sure what Windows releases support NDIS 6 or what the backward compatability is like. o NSE WORK (note that this is mostly infrastructure because script ideas are generally put on the script ideas page instead: https://secwiki.org/w/Nmap_Script_Ideas) o Review NSE-based port scanning and RST idle scan. http://seclists.org/nmap-dev/2011/q2/307. [Henri and Hani?] o Revive the Nmap Public Source License project (need to find an open source attorney to review it). http://nmap.org/npsl/ o Also take close look at Mozilla's license modernization project: http://mpl.mozilla.org/scope/ o Migrate web.insecure.org to a RHEL-6 derived distro (probably CENTOS 6, since Linode doesn't currently offer ScientificLinux images). o Actually, if we can wait until "second half of 2013", we might be able to jump straight to RHEL 7. And RHEL 5 support looks like it will go on for many more years for critical/security patches. o Maybe start with svn server, since we've had reports of our current one giving people unexpected password prompts. There is a thread about that at http://seclists.org/nmap-dev/2012/q2/17 o UPDATE on this - adding read-only rights (rather than no rights) to the root of the svn repo seems to have solved this problem. o Maybe we should add an analysis or reporting or intelligence (or different name) for our NSE scripts which don't send any packets, but simply analyze Nmap's existing data and report when useful. o Install some sort of svnview webapp for svn.nmap.org which is wrapped in Insecure chrome, allows people to click link for direct file download, probably shows revision history and allows users to see older versions, etc. o Process Nmap survey and send out results [Fyodor] o Nping (we think) will stop after 2^32 rounds even when "-c 0" is given. We should probably make this a 64-bit integrer so that "-c 0" will go essentially forever and so that users can give values higher than 4 billion. o Add CPE entries to the service detection DB entries which don't yet have them. o [UPDATER] Create a way to send an error message to the user (e.g. "your account has expired" or "updates denied due to overuse--please wait 24 hours before trying again", or "account suspended due to abuse")? David: I've seen svn errors like this: svn: Repository moved temporarily to 'http://www.metasploit.com/svn/framework3/trunk/modules'; please relocate so maybe there is a standard way to do it. o [UPDATER] Create webapp for account creation (can be deferred until later) o [UPDATER] Release to community, probably starting with a small test group of people. o [UPDATER] When it runs, it should give user more status about what happened. Maybe it could give the number of new/updated files and mention what directory it put them in (e.g. /home/fyodor/.nmap/updates/5.61TEST4). And if there are no updates available, it should say so. o Nscan work [placeholder] - Hosted Nmap system o Add CPE entries to OS fingerpting DB entries which still lack them. This is a gradual process since almost all of the missing ones aren't in the official CPE dictionary either and it can take a lot of research to decide on an appropriate entry. Milestones so far: - 3/21/12: We have entries for 2,601 of 3,572 fingerprints (971 missing; 73% coverage) - 11/5/12: We have entries for 3,285 of 3,907 fingerpritns (622 missing; 84% coverage) - 11/12/12: We have entries for 3,558 of 3,946 fingerprints (388 missing; 90% coverage). o [Zenmap] should actually parse and use script results. See http://seclists.org/nmap-dev/2010/q1/1108 - We have an initial prototype, but probably need to redo because it doesn't present the results in the way we'd like yet due to problems implementing such a presentation with GTK, etc. o Make Zenmap settings get upgraded when the Zenmap executable is upgraded. The per-user configuration files such as scan_profile.usp and zenmap.conf are never overwritten once installed by Zenmap, so changes and fixes to those files don't reach anyone who has installed Zenmap already. This is most noticeable with changes to profiles and highlight definitions are notably affected. This fix may involve hard-coding settings that are not normally configured by users (like highlighting) or updating the per-user files at startup (only those parts that haven't been changed by the user). o We should offer partial results when a host timeouts. I (Fyodor) have been against this in the past, but maybe the value is sufficient to be worth the maintenance headaches. Many users have asked for this. If we do implement this, we may want to only print results for the COMPLETED phases (e.g. host discovery, port scanning, version detection, traceroute, NSE, etc.) Trying to print partial results of a port scan or NSE or the like might be a pain. And if we print some results for a host which timeouts, we should give a very clear warning that the results for that host are incomplete. As an example, here is someone who hacked Nmap source code to achieve this: http://seclists.org/pen-test/2010/Mar/108. o Another benefit would be that it would allow us to clean up/regularize the host output code. Right now there are I think three places where a host's final output can be printed. If, instead, that code just looked at what information was available and printed that out only, we could potentially isolate it in just one place. o This also might let us provide a feature for skipping the rest of an Nmap phase which is going too slowly (I think that has its own Nmap TODO item). o Consider providing an option which causes Nmap to scan ALL IP addresses returned for a given name. So if "google.com" returns 4 names, scan them all (right now we print them all but only scan the one which happens to be the first on the current list). We then might want to make -A imply that option. Here is a thread on the topic: http://seclists.org/nmap-dev/2010/q2/302 - Need to decide what to do with e.g. google.com/24 -- scan four class C ranges? That's probably what we do. - Note that we now have a script which does something similar this--resolveall.nse. But it is a bit akward because you need to pass the targets as a script arg. And this is valuable enough functionality that we should probably have a simple Nmap command-line option to do it. Once this is added, we can probably remove the script. o [Nsock] Some SSL connections that used to work now fail; find out why. http://seclists.org/nmap-dev/2010/q4/788. Narrowed down to r19801 in http://seclists.org/nmap-dev/2011/q1/12. o [NSE] Consider a system where scripts can tell if any other scripts depend on them. They could then use that to determine whether they should bother storing information in the registry. For example, snmp-interfaces could store the discovered table if another script (such as a mac address geolocator script) depends on it. o Add parallel IPv6 reverse DNS support (right now we use the system functions). o [NSE] Consider whether we need script.db for performance reasons at all or should just read through all the scripts and parse on the fly. See: [http://seclists.org/nmap-dev/2009/q2/0221.html] o A couple minor nsedoc issues (see http://seclists.org/nmap-dev/2011/q1/1095): o After the ssh-hostkey portrule was added, nsedoc seems to be generating a blank "Script types" filed for the script: http://localhost:8082/nsedoc/scripts/ssh-hostkey.html o This is happening because "portrule" and "hostrule" appear later in the script, and NSEDoc thinks it is their definition, and there is no NSEDoc there. local ActionsTable = { -- portrule: retrieve ssh hostkey portrule = portaction, -- postrule: look for duplicate hosts (same hostkey) postrule = postaction } o ssh-hostkey and rmi-dumpregistry each have two @output sections, and NSEDoc is only showing the second one. We should probably just combine them into one @output section, and maybe make nsedoc give a warning in this case. Or we could make nsedoc handle multiple @outputs. o Add general regression unit testing system to Nmap o David has created a system for Ncat which could serve as a model. o Make version detection and NSE timing system more dynamic so that the concurrency can change based on network conditions/ability. After all, beefy systems on fast connections should be able to handle far more parallel connections than slower systems. o At a minimum, this at least warrants more benchmark testing. o We should run at least one SCTP service on scanme. Daniel Roethlisberger has made available dummy services which support IPv4 and IPv6 (see http://seclists.org/nmap-dev/2011/q2/450). Alternatively, we could run some sort of "real" SCTP application(s) (preferably one which is relatively simple, easy to install, secure, and supports IPv6). o Create new default username list: http://seclists.org/nmap-dev/2010/q1/798 o Could be a SoC Ncrack task, though should prove useful for Nmap too o We probably want to support several lists. Like an admin/default list like "root", "admin", "administrator", "web", "user", "test", and also a general list which we obtain from spidering from emails, etc. o Improve Nsock proxies system - Add SSL support - Add proxy authentication - Switch Ncat to using Nsock proxy system rather than it's own built-in support. - Move the code which is shared with ncat to nbase (URL parsing code, for instance). - Add socks4a/socks5 support. This requires to figure out how to enter the nsock proxy code w/o having the target IP address. No huge technical blocker there though, only design choices to make. - Nping could potentially use it as well (could be useful for measuring latency and reliability of a given proxy chain, for example). - Add proxy support to connect() scan. This would mean moving connect scan to nsock. o [NCAT] Send one line at a time when --delay is in effect. This is cumbersome to do until Nsock supports buffered reading. o [NCAT] Make the HTTP proxy support the chunked transfer encoding, then change it to be HTTP/1.1 and support pipelining. o [NCAT] Drop privileges once it has started up, bound the ports it needs to, etc. o [NCAT] Work as a SOCKS4a/SOCKSv5 proxy. o [NCAT] Resolve names through the proxy when possible. http://seclists.org/nmap-dev/2012/q2/768 o [NSE] Script writing contest (something to think about) o We should document an official way to compile/test refguide.xml so people can more easily test their changes to it. This will probably involve moving legal-notices.xml into /nmap/docs, among other things. o Note that nping has its own /nmap/nping/docs/genmanpage.sh - we could look at how that could apply to Nmap. o Move Zenmap man page from nmap/docs/ to nmap/zenmap/docs to match the man page location for ncat and ndiff. o Don't break packaging/build system o Don't break the system for posting html to web site. o Consider standardizing names for nping and ncrack man pages as well. [Fyodor] o [NSE] MSRPC - Improve domain support all around -- in particular, let the user give the domain in the format DOMAIN\username or username@DOMAIN anywhere that usernames are accepted. Suggested at http://seclists.org/nmap-dev/2010/q2/389 o [NSE] Combine similar MSRPC scripts, especially the "get info" stuff. See this thread on combining (http://seclists.org/nmap-dev/2010/q1/1023). This was suggested by Ron at http://seclists.org/nmap-dev/2010/q2/389. o [Zenmap] Investigate getting new OS icon art. See http://seclists.org/nmap-dev/2010/q1/1090 o We should probably enhance scan stats--maybe we can add a full-scan completion time estimate? Some ideas here: http://seclists.org/nmap-dev/2010/q1/1007 o [NSE] Do some benchmarking of our brute.nse. We should check the performance with different levels of thread parallelism. Our initial results show that it isn't helping much for vnc-brute or for drda-brute (which is currently using the multi-thread feature directly rather than through brute.nse library). We should figure out why the threads aren't helping more, and whether there is something we can do to fix it. It would also be interesting to compare speed with Ncrack for services we have in common. o Start project to make Nmap a Featured Article on Wikipedia. - See http://seclists.org/nmap-dev/2010/q1/614 o Add Nmap web board/forum - First step is looking at the available software for this. o [Zenmap] Consider a couple ideas from Norris Carden (http://seclists.org/nmap-dev/2010/q2/228): - remember last save and/or open location for new saves and/or opens - default save location option o [Nsock] Consider adding server support to Nsock so it can accept multiple connections and multiplex the SD's, like it does for clients. This could potentially be used by Ncat and Nping echo mode. Currently Ncat server doesn't use Nsock at all, while Nping echo mode basically polls, repeating a loop of 1s in nsock_loop followed by a nonblocking accept(). Then Nping gives the SD's to Nsock to manage. o Consider implementing both global and per-host congestion control in the IPv6 OS detection engine. Currently it handles congestion globally (one CWND and SSTHRESH shared by all hosts). This works fine but it may not be the most efficient approach: if the congestion is not in our network segment but in a target's and we are os-scanning hosts in different networks, then all hosts get "penalized" because there is congestion in another network, not in theirs. o [Nsock] Consider implementing a nsock_pcap_close() function or making nsp_delete() call pcap_close() when pcap IODs are used. Currently valgrind warns about a socket descriptor left opened (at least in Nping). ==10526== at 0x62F77A7: socket (syscall-template.S:82) ==10526== by 0x4E348A5: ??? (in /usr/lib/libpcap.so.1.0.0) ==10526== by 0x4E36819: pcap_activate (in /usr/lib/libpcap.so.1.0.0) ==10526== by 0x4E375FC: pcap_open_live (in /usr/lib/libpcap.so.1.0.0) ==10526== by 0x4311A9: nsock_pcap_open (nsock_pcap.c:64) ==10526== by 0x428078: ProbeMode::start() (ProbeMode.cc:329) o Consider rethinking Nmap's -s* syntax for specifing scan types o Current problems with this -s syntax: o We already use like 20 of the 26 letters, so we end up with things like SCTP scan using -sY o Can make Nmap command lines hard to read, particularly given that we often need to improvise to find a letter which isn't taken. o Problematic for scan types -sI and -b which require arguments o Inconsistencies. For example, -sC and -sV do script scan and version detection, respectively, and yet for OS detection we use -O. Also, control flow (-sP, -sL) is used with -s, which further overloads the options. o Possible solution: o We are enabling -Pn and -sn as preferred notations for -PN and -sP which mean "no ping" and "no port scan". Those match the already existing -n for "no DNS". The problem with -sP is that it implies "ping only", when what it really should mean is "disable port scan" because you may want to do NSE, OS detection, traceroute, etc. still. o We might want to just give them normal option strings, so you could do --maimon instead of -sM, for example. For extremely common options such as SYN scan, UDP scan, version detection, we could perhaps find good single letter options as an alias to the longer one. o Another idea is to use something like --scantype syn,udp,sctp, which is a lot longer for single-type scans, but shorter when you're combining mulitiple ones. Doesn't allow for individual scan arguments easily. I (Fyodor) think I prefer the idea above of just givem them top level arguments. o If we keep -s*, we could just give it one defined function, such as selecting port scan type, or control flow. o Obviously this will take some discussion/brainstorming on nmap-dev. o Do -p- Internet UDP scans. o [Ncat] This may sound ridiculous, but I'm starting to think that Ncat should offer a very simple built-in http server (e.g. for simply sharing files, etc.) And maybe a simple client too. o Scanning through proxies o Nmap should be able to scan through proxy servers, particularly now that we have an NSE script for detectiong open proxies and now that Ncat can act as proxy client or server. o Requirements: o Would be nice to be able to chain through multiple proxy servers of different types. o Would be nice to be able to spread the load amongst multiple proxies. o Should support port scanning, version detection, and NSE. In other words, nsock should support proxies. o Support IPv4 and v6 o Need to figure out how to get good performance. Pool of connections to proxy or proxies for concurrency? HTTP pipelining? o Support the different varieties of proxies: socks4, socks4a, socks5, HTTP GET (if possible), HTTP CONNECT. Note that GET proxies present some challenges since the error messages may not be standard, etc. o Maybe auto-detect the proxy type so that Nmap can try the most efficient scanning method first? o I've been asked to support basic, ntlm, and digest authentication if possible. o Implementation ideas: o There is a patch by Zoltan Panczel (http://nmap-dev.fw.hu) and it has been improved by Jacob Appelbaum in nmap-exp/ioerror/ . This patch doesn't handle things like parallelization, but it may be a good proof of concept. o This might not be appropriate for ultra_scan ... perhaps would be better to write a general scanning engine for abusing applications for port scanning purposes. This could handle scanning through proxies and the existing FTP bounce scan would also be ported to this engine (or, frankly, we could probably get away with removing FTP bounce). rembrandt at jpberlin.de tells me that you can also do this with the "forwarding" commands on IMAP servers. Whoever does this should probably start by reading the code for the main port scanning engine (ultra_scan()) and also the version detection code (service_scan()). And the version detection paper at http://nmap.org/book/vscan.html. If you understand all that, you may be ready for this project :). This is important, because it is easy to do poorly. The tough part is high performance and clean code which is general enough that all these different applications can be scanned through using the same basic engine. You should run your ideas by nmap-dev in as much detail as possible before starting. o David: I'm starting to think about building proxy support into Nsock and then implementing -sT with Nsock instead of ultra_scan. o [Web] Consider adding training/introduction videos to the Nmap site o Would be great to have a (5 minute or less) promotional video introduction to each tool (Nmap, Zenmap, Ncat, Ndiff) on its web page. o They need to be good to be useful--the sort of the quality you see in Laura Chappell's Wireshark videos or James Messer's Nmap videos or Irongeek's videos (http://www.irongeek.com). o Besides the promotional videos, users would probably enjoy more in-depth video instructions (e.g. covering the Nmap Network Scanning topics). o Here's an example product page with lots of videos (we may not go that far): http://www.splunk.com/product o The Zenmap translation system (http://nmap.org/book/zenmap-lang.html) has been pretty successful so far. We should consider doing the same for Nmap. After all, we already have the reference guide in 16 languages at http://nmap.org/docs.html. We should definitely try to use the same translation methods for Zenmap as we do for Nmap. In fact, maybe we can create a combined PO file Nmap, Zenmap, Ncat, and Ndiff so that they can all be translated and maintained together. Something to consider: calling setlocale can change the behavior of functions like isalpha. Locale-dependent functions need to be checked for security risks. o [NSE] Consider whether we should include some sort of NSE debugger. Or we could include something simpler. For example, Nmap now provides a traceback (with sufficient debugging/verbosity) when a script ends in error. For some inspiration/ideas, look at Diman's NSE debugger (http://seclists.org/nmap-dev/2008/q1/0228.html). o [NSE] Support routing http requests through proxies. o Consider offering a way to link Winpcap DLLs so that they start the service as needed rather than requiring explicitly installing Winpcap and having it start upon system boot. CACE has offered such a thing for many years as WinPcap Pro (http://www.cacetech.com/products/winpcap_pro.html). If we change WinPcap in this way, we'd presumably want to also change the symbol names as is done in WinPcap Pro. And it would mean that we have to build our Winpcap binaries ourselves (including 64-bit). We might even have to sign our drivers for 64-bit Windows. o [NSE] Would be great if NSE scripts could be made to NOT run as root if they don't have to. o [NSE] Security Review o Consider what, if any, vulnerabilities or security risks NSE has with respect to buffer overflows, format string bugs, any other maliciously formatted responses from target systems, etc. Maybe address the known risk of malicious scripts too. o Consider that NSE runs scripts as root o More security auditing of Nmap code (it never hurts to do more proactive security auditing). o Figure out and document (in at least the Ncat user's guide) the best way to use Ncat for chaining through proxies. One option is this sort of thing: ncat -l localhost 1234 --sh-exec "ncat --proxy A.A.A.A B.B.B.B" ncat --proxy localhost:1234 C.C.C.C If you had two proxies A.A.A.A and B.B.B.B, connecting to C.C.C.C. With another listener/--sh-exec pair for each additional proxy. But perhaps we can make it easier by adding it to the syntax. o Look into whether we should loosen/change the global congestion control system to address possible cases of one target host with many dropped packets slowing down the whole group. See http://seclists.org/nmap-dev/2008/q1/0096.html . * Related possibility: Fix --nogcc to gracefully handle ping scans. Right now it seems to go WAY TOO FAST (e.g. several thousand packets per second on my DSL line). * [12/22/09] David says: It still is in one case that I've documented on my wiki. I had an idea to fix it, but on testing it it didn't work. The idea was to treat the global congestion limit differently. Instead of dropping it down to the minimum level on a drop as is done currently, I thought about only dropping it by the amount that the individual host limit drops. For example, if a host had a drop and its limit fell from 25 to 1, then the global limit would change (if it was at 100 to begin with) to 76, not all the way down to 2 or whatever it is. The idea being that the global limit is most important at the beginning of a scan, when there's no information to set host limits, and every host wants to send all its first probes at once. See http://www.bamsoftware.com/wiki/Nmap/PerformanceNotesArchive2#global-cc. I am convinced, though, that some sort of global control is necessary. There's a reason that a web browser limits the number of connections it will make, and doesn't try to download every image file at once and count on the fairness of TCP to sort it out. o libnmap organization for UNIX and Windows o Then change Nmap and Zenmap to simply call this library o It is interesting to look at: http://www.gnupg.org/gpgme.html o Deal with UDP retransmission for version detection (I think I should just do a second run of all probes for UDP if it fails to match anything). The advantage there is that no retransmissions are neccessary if the service is found. Then again, per-probe retransmission would let us redo the most likely probes (the one(s) that match the port number) quickly. Lost packets should probably affect ideal_parallelism. o Make RPM relocatable (requires somehow avoiding storing paths in the binary) - That may be easier now that David has made some big improvements in detecting where the binary is cross-platform and then looking for data files based on that location. o Nmaprc-related - Create a system to store Nmap defaults/preferences in an nmaprc file. o nmaprc should be in ~/.nmap on UNIX o On Windows, we may need a registry key to find the .nmaprc o Perhaps Lua could be used as the format? o .nmaprc for keeping defaults, etc. o Nmaprc infrastructure, hook to new timing variables o Nmaprc man page o Default timing mode o Default NSE arguments, such as user agent o Maybe Default source IP (-S) argument o should be a way to specify your own .nmaprc o Maybe lets you add a directory and template for saving all scans. o Maybe let you define "scan profiles" like is done with Zenmap. There would then be a command-line option to select the profile used. o Get new Zenmap logo o consider putting back on top-right of command constructor wizard (there used to be umit logo there). o Maybe that can be done after the release by soliciting ideas. o Create or collect some great ./configure ascii art. o Look at all the pcap functions, there are some like pcap_findalldevs() which could be quite useful. There are mails to the Nmap list relating to suggested improvements -- http://seclists.org/lists/nmap-dev/2004/Apr-Jun/0024.html . Actually I do indirectly use that for Windows. I wonder if they work for UNIX? o perhaps each 'match' line in nmap-service-probes should have a maximum lines, bytes, and/or time by which a response should be available. Once that much time (or many bytes or lines) have passed, that match can be considered 'failed' and ignored in subsequent runs. Once all matches are considered failed, that probe is done. This could be a useful optimization and is arguably better than the less granular 'totalwaitms'. Or I could just have a simple function that looks at whether a given regex could possibly match something starting with the received data (not too hard since almost all of the current regexes are anchored). But before doing this, I should look long and hard at how many of the probes have every match capable of doing this. In particular, many of the softmatch lines don't offer many chars anchored at the front. o Separate nbase into its own Windows library in the same way as Andy did with iphlpapi . o Nmap / Nmap-hackers FAQ o random tip database