NSE proposal: New Rules Version 1: http://seclists.org/nmap-dev/2010/q2/952 Version 2: http://seclists.org/nmap-dev/2010/q2/966 Table: ------ 1) Introduction 2) More NSE Rules Background 3) Motivation 4) New NSE scripts details 5) New Rules 6) The Action function 7) Changes on Nmap/NSE code 8) Conclusion 1) Introduction: ---------------- In this proposal we will try to introduce new rules which can be used by a new class of NSE scripts. The current NSE rules are: o hostrule: specifies that the script should only run once against the host IP. o portrule: specifies that the script should only run against the ports list of the current target. The limitation of these rules is that the scripts will run only against a single host IP (hostrule or portrule), and this host is provided by Nmap which has already done a bunch of operations to get this information, so as an example, we can't run scripts against other targets which can be networks or domain names that do not depend on Nmap previous operations. * Network targets can be useful for host discovery and broadcast operations. * Domain name targets can be useful to do DNS brute force operations. Another limitation is that since NSE scripts depend on host and port tables information they must run after Nmap scan phases. A simple example: * Script: dns-zone-transfer.nse * Portrule: portrule = shortport.portnumber(53, 'tcp') Modify the dns-zone-transfer.nse script and add another rule to let the script run against the domain name to discover new targets. The current script will only run when Nmap finds a DNS server, so with a new added rule that script will run directly and does not depend on open ports and can find new subdomain targets for Nmap, in other words specify a domain name as a target and with the use of the results of this script, Nmap will scan all the newly discovered subdomains and hostnames. Please note that this is a first proposal about new rules. There will be other proposals to complete the New scripts class definition. 2) More NSE Rules Background: ----------------------------- At present there are only two types of scripts, host and port classes scripts. Each script has its own rule and action. o The action function is the heart of an NSE script [1]. o The rule is the Lua function which will be evaluated by NSE to true or false. The script's action will only be executed if its rule evalutes to true. NSE Rules: o Host rule: "Specifies that the script should be run only once against a target IP and only if the given conditions are met." [1] The Host rule accepts a host table which contains the IP or Hostname of the target as an argument. o Port rule: "Governs which ports of a target the scripts may run against" [1]. The Port rule accepts both host and port tables as arguments or any TCP or UDP port in the open, open|filtered, or unfiltered port states [1]. 3) Motivation: -------------- 1. Network operations: We can need scripts that perform different network operations in order to discover more hosts. In general these scripts will run once per scan and can feed Nmap with new targets, but we can have other use cases. o Use broadcast and multicast protocols as suggested in this thread http://seclists.org/nmap-dev/2010/q1/883 o We can also use AS entries to get IPs as suggested in this thread http://seclists.org/nmap-dev/2010/q2/101 o Perform DNS discovery: brute force, zone transfert etc. o Perform Whois requests. o Any other operation which can take a network or a simple input that can be used to produce a list of Nmap targets. o Scripts that run after Nmap scans to collect/report results. o Scripts which want to use Nmap/NSE API. 4) New NSE scripts details: ------------------------------- o The new scripts can run at different times: * Before any scanning: pre-scan scripts. * After Nmap scans: post-scan scripts. Before/after NSE hostgroup scan can be emulated by the current NSE dependency system. o Pre-scan scripts can be used for host discovery like DNS scripts. We can also have other cases of use like scripts which simply want to use Nmap/NSE/Lua API to do cool tricks. e.g: Ron's California Vanity License plate script :-) http://www.skullsecurity.org/blog/?p=723 o Post-scan scripts: can be used to report data or to perform other operations and take advantage of the Nmap/NSE/Lua API. e.g: A script which collects all the whois hostrule script contact details results and report the unique ones. o Since scripts can run at different times, then we need different rules. * prerule -- this rule will be used for pre-scan scripts. * postrule -- this rule will be used for post-scan scripts. o The new scripts will be identified by the presence of these new rules and by their categories. o Scripts must use a new NSE API to push new targets. o Hostrule and portrule scripts can have the ability to add new targets using the new NSE API, but perhaps we need a new Nmap option to allow this behaviour. o Any target filtering will be done by Nmap. The new scripts must care only about their arguments. o Each script can have its own arguments like any other script. e.g: script.arg1=x, script.arg2=x ... o Scripts will use new API functions to access network interfaces like eth0, and mac address source etc. A big point which will be discussed in the next proposal: If NSE host discovery scripts will expand the Nmap target list or narrow it down like the ping scan ? Please keep in mind that this point will be discussed in a next Nmap New Targets proposal. 5) New Rules: ------------- Since we can have different scripts which can run at different times or multiple times as stated in the [New NSE scripts details] section, the imprortant question will be: When should the scripts run ? For this we have proposed: o Different rules funtions: * prerule -- this rule will be used for pre-scan scripts. * postrule -- this rule will be used for post-scan scripts. o The new rules in most cases will evaluate to true. o The new rules will be evaluated when the user specifies their category with the --script option. o The new rules will facilitate the addition of the new host discovery and scanning scripts. e.g: prerule = function() return true end postrule = function() return true end 6) The Action function: ----------------------- o Since new NSE scripts can run at different times, the action function must know when it has been called, because some scripts can have different behaviours especially when they have multiple rules. e.g: A whois script which have a prerule and a postrule rules will run at the beginning and sets a registry value to tell the whois hostrule script to save the results onto the registry, after that the first whois script will run at the end (postrule) and collects/report the final results (e.g: unique contact details). Having two different scripts, a pre-scan one and a post-scan script to do this seems to me rediculous. o This feature of a script which can have multiple rules must not be abused, if the action code differes a lot from when the script is in the pre-scan mode and in the post-scan mode, then the best solution is to have two different scripts each one with a rule. Solutions to support scripts with both prerule and postrule: o Add a SCRIPT_TYPE or RULE_TYPE field to the script thread's global table that is the rule type of the running script: "portrule", "hostrule", "prerule", "postrule". The action function can then multiplex based on this global field. e.g: action = function() if SCRIPT_TYPE == "prerule" then ... elseif SCRIPT_TYPE == "postrule" then ... end end o The rule function can return an optional second value which is the action function to run. If they only return a true boolean, then use the global action function. This allows our current scripts to remain untouched and more complex scripts can still use this new functionality. 7) Changes on Nmap/NSE code: ---------------------------- o Since there will be three different script scan phases then new scan types must be defined. NSE needs also to know on which phase it is to activate the correct rules. NSE Scan Phases: * SCRIPT_PRE_SCAN: phase for prerule scripts, before any Nmap scan. * SCRIPT_SCAN: normal NSE scan for hostrule and portrule scripts. * SCRIPT_POST_SCAN: phase for postrule scripts, at the end of an Nmap scan. Other changes: o A new argument to the script_scan() to indicate the current script scan phase, this argument will be assigned to the current_scantype variable. e.g: void script_scan (std::vector &targets, int scantype) { o.current_scantype = scantype; ... } script_scan(Targets, SCRIPT_PRE_SCAN); // new call of script_scan() o A new string argument will be passed to the main NSE function, this string is the result of the scantype2str(o.current_scantype) call. The main NSE function can then setup prerule, hostrule, portrule and postrule scripts according to this argument. e.g (NSE main function): local NSE_PRE_SCAN = "NSE_PRE_SCAN"; local NSE_SCAN = "NSE_SCAN"; local NSE_POST_SCAN = "NSE_POST_SCAN"; ... return function (hosts, scantype) -- activate prerule scripts if (scantype == NSE_PRE_SCAN) then ... -- activate hostrule, portule and versionrule scripts elseif (scantype == NSE_SCAN) then ... -- activate postrule scripts elseif (scantype == NSE_POST_SCAN) then ... end -- run scripts end 8) Conclusion: -------------- The addition of these new rules will let the scripts take different targets not only Nmap IP hosts or port lists, and will offer them a way to run before or after Nmap scan phases. This idea about a new class of scripts was first brought by Ron [2]. [1] http://nmap.org/book/nse-script-format.html [2] http://seclists.org/nmap-dev/2010/q1/883