/*************************************************************************** * nsock_scan.cc -- Nmap's connect() scan implemented using nsock library. * * ***********************IMPORTANT NMAP LICENSE TERMS************************ * * * The Nmap Security Scanner is (C) 1996-2013 Insecure.Com LLC. Nmap is * * also a registered trademark of Insecure.Com LLC. This program is free * * software; you may redistribute and/or modify it under the terms of the * * GNU General Public License as published by the Free Software * * Foundation; Version 2 ("GPL"), BUT ONLY WITH ALL OF THE CLARIFICATIONS * * AND EXCEPTIONS DESCRIBED HEREIN. This guarantees your right to use, * * modify, and redistribute this software under certain conditions. If * * you wish to embed Nmap technology into proprietary software, we sell * * alternative licenses (contact sales@nmap.com). Dozens of software * * vendors already license Nmap technology such as host discovery, port * * scanning, OS detection, version detection, and the Nmap Scripting * * Engine. * * * * Note that the GPL places important restrictions on "derivative works", * * yet it does not provide a detailed definition of that term. To avoid * * misunderstandings, we interpret that term as broadly as copyright law * * allows. For example, we consider an application to constitute a * * derivative work for the purpose of this license if it does any of the * * following with any software or content covered by this license * * ("Covered Software"): * * * * o Integrates source code from Covered Software. * * * * o Reads or includes copyrighted data files, such as Nmap's nmap-os-db * * or nmap-service-probes. * * * * o Is designed specifically to execute Covered Software and parse the * * results (as opposed to typical shell or execution-menu apps, which will * * execute anything you tell them to). * * * * o Includes Covered Software in a proprietary executable installer. The * * installers produced by InstallShield are an example of this. Including * * Nmap with other software in compressed or archival form does not * * trigger this provision, provided appropriate open source decompression * * or de-archiving software is widely available for no charge. For the * * purposes of this license, an installer is considered to include Covered * * Software even if it actually retrieves a copy of Covered Software from * * another source during runtime (such as by downloading it from the * * Internet). * * * * o Links (statically or dynamically) to a library which does any of the * * above. * * * * o Executes a helper program, module, or script to do any of the above. * * * * This list is not exclusive, but is meant to clarify our interpretation * * of derived works with some common examples. Other people may interpret * * the plain GPL differently, so we consider this a special exception to * * the GPL that we apply to Covered Software. Works which meet any of * * these conditions must conform to all of the terms of this license, * * particularly including the GPL Section 3 requirements of providing * * source code and allowing free redistribution of the work as a whole. * * * * As another special exception to the GPL terms, Insecure.Com LLC grants * * permission to link the code of this program with any version of the * * OpenSSL library which is distributed under a license identical to that * * listed in the included docs/licenses/OpenSSL.txt file, and distribute * * linked combinations including the two. * * * * Any redistribution of Covered Software, including any derived works, * * must obey and carry forward all of the terms of this license, including * * obeying all GPL rules and restrictions. For example, source code of * * the whole work must be provided and free redistribution must be * * allowed. All GPL references to "this License", are to be treated as * * including the terms and conditions of this license text as well. * * * * Because this license imposes special exceptions to the GPL, Covered * * Work may not be combined (even as part of a larger work) with plain GPL * * software. The terms, conditions, and exceptions of this license must * * be included as well. This license is incompatible with some other open * * source licenses as well. In some cases we can relicense portions of * * Nmap or grant special permissions to use it in other open source * * software. Please contact fyodor@nmap.org with any such requests. * * Similarly, we don't incorporate incompatible open source software into * * Covered Software without special permission from the copyright holders. * * * * If you have any questions about the licensing restrictions on using * * Nmap in other works, are happy to help. As mentioned above, we also * * offer alternative license to integrate Nmap into proprietary * * applications and appliances. These contracts have been sold to dozens * * of software vendors, and generally include a perpetual license as well * * as providing for priority support and updates. They also fund the * * continued development of Nmap. Please email sales@nmap.com for further * * information. * * * * If you have received a written license agreement or contract for * * Covered Software stating terms other than these, you may choose to use * * and redistribute Covered Software under those terms instead of these. * * * * Source is provided to this software because we believe users have a * * right to know exactly what a program is going to do before they run it. * * This also allows you to audit the software for security holes (none * * have been found so far). * * * * Source code also allows you to port Nmap to new platforms, fix bugs, * * and add new features. You are highly encouraged to send your changes * * to the dev@nmap.org mailing list for possible incorporation into the * * main distribution. By sending these changes to Fyodor or one of the * * Insecure.Org development mailing lists, or checking them into the Nmap * * source code repository, it is understood (unless you specify otherwise) * * that you are offering the Nmap Project (Insecure.Com LLC) the * * unlimited, non-exclusive right to reuse, modify, and relicense the * * code. Nmap will always be available Open Source, but this is important * * because the inability to relicense code has caused devastating problems * * for other Free Software projects (such as KDE and NASM). We also * * occasionally relicense the code to third parties as discussed above. * * If you wish to specify special license conditions of your * * contributions, just say so when you send them. * * * * This program is distributed in the hope that it will be useful, but * * WITHOUT ANY WARRANTY; without even the implied warranty of * * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the Nmap * * license file for more details (it's in a COPYING file included with * * Nmap, and also available from https://svn.nmap.org/nmap/COPYING * * * ***************************************************************************/ /* $Id$ */ #include "nsock_scan.h" #include "nmap_error.h" #include "timing.h" #include "nmap_tty.h" #include "NmapOps.h" #include #include /* Returns the current time in a struct timeval instance. */ struct timeval get_now() { struct timeval ret; gettimeofday(&ret, NULL); return ret; } /* TODO: these belong to ultra_scan_performance_vals, which is declared in scan_engine.cc. This should probably be moved to some common header. */ static const long TIMING_CANARYTIME = 1250000; static const int TIMING_CANARY_MAGNIFIER = 3; extern NmapOps o; class NsockProbe; class NsockHostScanStats; /* The central, abstract class for Nsock scanning engine. Responsibilities: * Reacts to probe dropped/acknowledged events, * Handles various probe-sending related mechanisms: -> Timeouts, -> Retransmissions, -> Congestion control, -> Scan delay/min-rate/max-rate, -> Probe canaries. * Generates progress information, * Contains probe sending logic. */ class NsockScanEngine { protected: /* The "Targets", "portarray" and "numports" arguments that nsock_scan() was started with. */ const unsigned int targets_count; const u16 *portarray; const int numports; /* The target that we're going to send a next probe to. TODO: maybe it should be a local variable of sendNextProbes somehow? */ std::list::const_iterator next_nhss; /* This lets us pair a Target* to NsockHostScanStats. TODO: maybe store a pointer to NsockHostScanStats in Target instead and avoid some lookups? */ const std::list nhss_list; /* A list of probes we're currently waiting to come back. */ std::list probes_outstanding; int active_probes; /* Basically probes_outstanding.size(). */ /* Global timing structures. TODO: Maybe it would be better to factor them out to a NsockScanStats class that NsockHostScanStats would inherit from? */ struct timeout_info to; static scan_performance_vars perf; struct ultra_timing_vals timing; ScanProgressMeter *SPM; /* These two are related to --max-rate and --min-rate. */ struct timeval send_no_later_than; struct timeval send_no_earlier_than; inline int getProbeLimit() const; std::list populateNhssList( const std::vector &Targets_arg, struct timeval now); void printTimingStats(struct timeval *now) const; void printHostsTimingStats() const; int countDoneHosts(struct timeval *now) const; void handleAck(NsockProbe *probe, struct timeval *now); void handleDrop(NsockProbe *probe, struct timeval *now, int reason_id); void handleTimeouts(struct timeval *now); void sendProbe(NsockProbe *probe, struct timeval *now); void sendNewProbe(NsockHostScanStats *nhss, unsigned short portno, int tryno, bool is_canary, struct timeval *now = NULL); double getCompletionFraction() const; bool sendOK(unsigned int inactive_hosts); bool trySendAnyNextProbe(struct timeval *now, unsigned long *when_ok, unsigned int *inactive_hosts, unsigned int *complete_hosts, std::map *inactive_hosts_map); void initUltraTimingVals(ultra_timing_vals &timing); void stopTimeoutClocks(struct timeval &now); public: NsockScanEngine(const std::vector &Targets_arg, const u16 *portarray_arg, const int numports_arg); ~NsockScanEngine(); static scan_performance_vars &getScanPerformanceVars(); int getNumPorts() const; const struct ultra_timing_vals getTiming() const; const struct timeval getSendNoEarlierThan() const; const struct timeval getSendNoLaterThan() const; /* TODO: all the following members apart from run() are public only because it wasn't trivial to make these private, mostly because of the Nsock callbacks. This is NOT proper OO. */ nsock_pool nsp; void sendNextProbes(); virtual void sendProbeInternal(NsockProbe* probe) = 0; void handleProbeEvent(NsockProbe* probe, int reason_id, int port_state); void doSendNextProbe(struct timeval *now, NsockHostScanStats *nhss); void run(); }; scan_performance_vars NsockScanEngine::perf; /* Host-specific scan state. Responsibilities: * Tells NsockScanEngine which port we're going to scan, * Provides NsockScanEngine with information for various probe-sending mechanisms. */ class NsockHostScanStats { public: /* Tells Nmap what's the maximum number of attempts it can make before considering probe timeouts as firewall filtering our requests. This is increased when we notice that retried probes actually make it to the host, which might mean that the network is unreliable. */ int max_tryno; /* The index of port in NsockScanEngine::portarray we're going to scan next. */ int current_port_idx; /* The number of outstanding probes we have for this host. */ int active_probes; /* Per-host timing structures. */ struct ultra_timing_vals timing; struct timeout_info to; struct timeval last_sent; /* Variables related to "canary" probes, also called "ping probes". These are probes that are sent when we rarely get any respones that are supposed to tell us if we don't send probes too fast. */ struct timeval last_canary_sent; struct timeval last_received; int numprobes_sent; int lastping_sent_numprobes; Target* target; const NsockScanEngine &nsse; std::list retransmission_queue; int retransmission_queue_size; NsockHostScanStats(const NsockScanEngine &nsse, Target *target_arg, struct timeval *now); bool isDone(struct timeval *now); bool sendOK(struct timeval *now, unsigned long *when); bool shouldSendCanary(struct timeval *now); void getProperTiming(struct ultra_timing_vals *tmng); }; /* A class that contains information about a particular scanning probe. */ class NsockProbe { public: NsockHostScanStats *nhss; unsigned short portno; u8 tryno; nsock_iod sock_nsi; struct timeval sent; bool is_canary; }; NsockHostScanStats::NsockHostScanStats(const NsockScanEngine &nsse, Target *target_arg, struct timeval *now) : nsse(nsse) { last_canary_sent = *now; last_received = *now; numprobes_sent = 0; lastping_sent_numprobes = 0; last_sent = *now; retransmission_queue_size = 0; current_port_idx = 0; max_tryno = 1; active_probes = 0; timing.cwnd = nsse.getScanPerformanceVars().host_initial_cwnd; timing.ssthresh = nsse.getScanPerformanceVars().initial_ssthresh; timing.num_replies_expected = 0; timing.num_replies_received = 0; timing.num_updates = 0; initialize_timeout_info(&to); target = target_arg; } /* A destructor for NsockScanEngine. It has to be defined after the declaration of NsockHostScanStats in order to know how to free its memory. */ NsockScanEngine::~NsockScanEngine() { for (std::list::const_iterator it = nhss_list.begin(); it != nhss_list.end(); it++) { delete (*it); } } /* A getter for scan_performance_vars that is used in NsockHostScanStats's constructor. */ scan_performance_vars &NsockScanEngine::getScanPerformanceVars() { return perf; } /* The getter for numports that lets NsockHostScanStats::sendOK tell if scanning already completed for the given host. */ int NsockScanEngine::getNumPorts() const { return numports; } const struct ultra_timing_vals NsockScanEngine::getTiming() const { return timing; } const struct timeval NsockScanEngine::getSendNoEarlierThan() const { return send_no_earlier_than; } const struct timeval NsockScanEngine::getSendNoLaterThan() const { return send_no_earlier_than; } /* Returns the number of active probes we can have right now. */ inline int NsockScanEngine::getProbeLimit() const { int cwnd = floor(timing.cwnd); /* As scan_engine.cc says: > There are good arguments for limiting the number of probes sent > between waits even when we do get appropriate receive times. For > example, overflowing the pcap receive buffer with responses is no > fun. On one of my Linux boxes, it seems to hold about 113 > responses when I scan localhost. And half of those are the @#$# > sends being received. I think I'll put a limit of 50 sends per > wait */ return cwnd > 50 ? 50 : cwnd; } /* An equivalent of HostScanStats::getTiming() from scan_engine.cc. I hate this solution, but couldn't come up with something more readable. scan_engine.cc said: > This function provides the proper cwnd and ssthresh to use. It may > differ from versions in timing member var because when no responses > have been received for this host, may look at others in the group. > For CHANGING this host's timing, use the timing memberval > instead. */ void NsockHostScanStats::getProperTiming(struct ultra_timing_vals *tmng) { assert(tmng); /* Use the per-host value if a pingport has been found or very few probes have been sent */ if (target->pingprobe.type != PS_NONE || numprobes_sent < 80) { *tmng = timing; return; } /* Otherwise, use the global cwnd stats if it has sufficient responses */ if (nsse.getTiming().num_updates > 1) { *tmng = nsse.getTiming(); return; } /* Last resort is to use canned values */ tmng->cwnd = nsse.getScanPerformanceVars().host_initial_cwnd; tmng->ssthresh = nsse.getScanPerformanceVars().initial_ssthresh; tmng->num_updates = 0; return; } /* Returns the number of done hosts for the given moment. */ int NsockScanEngine::countDoneHosts(struct timeval *now) const { int num_done = 0; for (std::list::const_iterator it = nhss_list.begin(); it != nhss_list.end(); it++) { NsockHostScanStats *nhss_tI = (*it); if (nhss_tI->isDone(now)) num_done++; } return num_done; } /* Prints timing debugging statistics for individual hosts. */ void NsockScanEngine::printHostsTimingStats() const { for (std::list::const_iterator it = nhss_list.begin(); it != nhss_list.end(); it++) { NsockHostScanStats *nhss_tI = (*it); struct ultra_timing_vals timing; nhss_tI->getProperTiming(&timing); log_write(LOG_PLAIN, " %s: %d/%d/%d/*/*/* %.2f/%d/* %d/%d/%d\n", nhss_tI->target->targetipstr(), nhss_tI->active_probes, numports - nhss_tI->current_port_idx, nhss_tI->retransmission_queue_size, timing.cwnd, timing.ssthresh, nhss_tI->to.timeout, nhss_tI->to.srtt, nhss_tI->to.rttvar); } } /* Prints timing stats similar to those from scan_engine.cc for debugging purposes. */ void NsockScanEngine::printTimingStats(struct timeval *now) const { log_write(LOG_PLAIN, "**TIMING STATS** ""(%.4fs): IP, probes active/" "freshportsleft/retry_stack/outstanding/retranwait/onbench, " "cwnd/ssthresh/delay, timeout/srtt/rttvar/\n", o.TimeSinceStart()); log_write(LOG_PLAIN, " Groupstats (%d/%d incomplete): " "%d/*/*/*/*/* %.2f/%d/* %d/%d/%d\n", targets_count - countDoneHosts(now), targets_count, active_probes, timing.cwnd, timing.ssthresh, to.timeout, to.srtt, to.rttvar); if (o.debugging >= 4) printHostsTimingStats(); } /* A function called when we find out that a probe arrived. */ void NsockScanEngine::handleAck(NsockProbe *probe, struct timeval *now) { if (o.debugging > 4) log_write(LOG_PLAIN, "NsockScanEngine::handleAck(%s:%hu)\n", probe->nhss->target->targetipstr(), probe->portno); probe->nhss->last_received = *now; /* If we don't have a canary probe yet, let's set one. TODO: at this point we should see the port state and if it wasn't the same (we had a closed canary port, but now we found another that is open), consider switching. */ if (probe->nhss->target->pingprobe.type == PS_NONE) { probe->nhss->target->pingprobe.type = PS_CONNECTTCP; probe->nhss->target->pingprobe.pd.tcp.dport = probe->portno; probe->nhss->target->pingprobe.proto = IPPROTO_TCP; probe->nhss->target->pingprobe.pd.tcp.flags = TH_SYN; } active_probes--; probe->nhss->active_probes--; /* NOTE: this does not belong to NsockScanEngine - instead, this is a global procedure, declared in timing.cc. TODO: move adjust_timeouts2 to struct timeout_info. */ adjust_timeouts2(&probe->sent, now, &probe->nhss->to); adjust_timeouts2(&probe->sent, now, &to); if (probe->tryno > 0) { /* If it was a retried probe, let's note that the previous probe was dropped. */ timing.drop_group(active_probes, &perf, now); probe->nhss->timing.drop(active_probes, &perf, now); } /* Notify the congestion control mechanism that we received a response. */ int canary_magnifier = probe->is_canary ? TIMING_CANARY_MAGNIFIER : 1; timing.ack(&perf, canary_magnifier); probe->nhss->timing.ack(&perf, canary_magnifier); if (probe->tryno > probe->nhss->max_tryno && probe->tryno < o.getMaxRetransmissions()) { /* This is a retried probe that sets a new retransmission limit. */ probe->nhss->max_tryno = probe->tryno; log_write(LOG_STDOUT, "Increased max_tryno for %s to %d (packet drop).\n", probe->nhss->target->targetipstr(), probe->tryno); } /* The probe served its purpose. Let's free its memory. */ probes_outstanding.remove(probe); delete probe; } /* A function called when a drop was detected. Also takes care of scheduling retransmissions. */ void NsockScanEngine::handleDrop(NsockProbe *probe, struct timeval *now, int reason_id) { if (o.debugging > 4) log_write(LOG_PLAIN, "NsockScanEngine::handleDrop(%s:%hu)\n", probe->nhss->target->targetipstr(), probe->portno); /* The probe timed out. Let's see if we can retry it to make sure that its port is filtered. */ if (probe->tryno < probe->nhss->max_tryno + 1 && !probe->is_canary) { active_probes--; probe->nhss->active_probes--; probes_outstanding.remove(probe); probe->nhss->retransmission_queue.push_back(probe); probe->nhss->retransmission_queue_size++; } else { active_probes--; probe->nhss->active_probes--; probe->nhss->target->ports.setPortState(probe->portno, IPPROTO_TCP, PORT_FILTERED); probe->nhss->target->ports.setStateReason(probe->portno, IPPROTO_TCP, ER_NORESPONSE, 0, NULL); /* We can't retry the probe, so let's send next "normal" probe to keep the number of outstanding probes. */ if (o.debugging) log_write(LOG_STDOUT, "Giving up on %s:%d\n", probe->nhss->target->targetipstr(), probe->portno); probes_outstanding.remove(probe); delete probe; } } /* Now that the response is interpreted, react to the probe by sending retranmissions (if possible and necessary), updating scan, port and timing information and sending new probes. */ void NsockScanEngine::handleProbeEvent(NsockProbe* probe, int reason_id, int port_state) { Target *target = probe->nhss->target; if (port_state == PORT_HIGHEST_STATE) { active_probes--; probe->nhss->active_probes--; /* We didn't get a response, but don't want to see if we can retransmit, probably because of some error. Let's mark the port as filtered and try to move on. */ target->ports.setPortState(probe->portno, IPPROTO_TCP, PORT_FILTERED); target->ports.setStateReason(probe->portno, IPPROTO_TCP, reason_id, 0, NULL); probes_outstanding.remove(probe); delete probe; sendNextProbes(); return; } /* We'll need current time to signal drop to ultra_timing_vals. */ struct timeval now = get_now(); if (port_state != PORT_FILTERED) { if (!probe->is_canary) { target->ports.setPortState(probe->portno, IPPROTO_TCP, port_state); target->ports.setStateReason(probe->portno, IPPROTO_TCP, reason_id, 0, NULL); } handleAck(probe, &now); } else { handleDrop(probe, &now, reason_id); } sendNextProbes(); } class NsockTCPScanEngine : public NsockScanEngine { public: NsockTCPScanEngine(const std::vector &Targets_arg, const u16 *portarray_arg, const int numports_arg) : NsockScanEngine(Targets_arg, portarray_arg, numports_arg) {}; static void statusToPortState(enum nse_status status, int connect_errno, int* port_state, int *reason_id, bool *should_return); static void errnoToPortState(int connect_errno, int *port_state, int *reason_id); void silentlyRetransmit(nsock_iod nsi, NsockProbe *probe); void sendProbeInternal(NsockProbe* probe); static void connectHandlerStatic(nsock_pool nsp, nsock_event evt, void *data); void connectHandler(nsock_pool nsp, nsock_event evt, NsockProbe *probe); ~NsockTCPScanEngine(); }; NsockTCPScanEngine::~NsockTCPScanEngine() { if (SPM) delete SPM; } /* As scan_engine.cc said: > Sometimes we get false results when scanning localhost with -p- because we > scan localhost with src port = dst port and see our outgoing packet and > think it is a response. I call it "self-connect" or "ephemeral ports bug". This happens mostly under Linux and should go away after retransmitting the probe. Link to the related discussion: http://seclists.org/nmap-dev/2014/q1/136 */ static bool is_self_connect(int sd, int dport, Target *target) { struct sockaddr_storage local; socklen_t local_len = sizeof(struct sockaddr_storage); struct sockaddr_storage remote; size_t remote_len; if (getsockname(sd, (struct sockaddr*)&local, &local_len) == 0 && target->TargetSockAddr(&remote, &remote_len) == 0) { if (sockaddr_storage_cmp(&local, &remote) == 0 && ( (local.ss_family == AF_INET && ((struct sockaddr_in*)&local)->sin_port == htons(dport)) #if HAVE_IPV6 || (local.ss_family == AF_INET6 && ((struct sockaddr_in6*)&local)->sin6_port == htons(dport)) #endif )) { return true; } } else { gh_perror("getsockname or TargetSockAddr failed"); } return false; } /* Sets port_state and reason_id based on an nse_status. Will cause the program to exit with a fatal error if we run out of file descriptors. A lot of further behavior depends on the event's status. This is where we find out if we managed to connect, got a connection closed, timed out or some other kind of error, which is later most useful for congestion control subsystem. */ void NsockTCPScanEngine::statusToPortState(enum nse_status status, int connect_errno, int *port_state, int *reason_id, bool *should_return) { /* First, let's see if it's some kind of an error and if so, which one... */ if (status == NSE_STATUS_ERROR) { errnoToPortState(connect_errno, port_state, reason_id); /* Handle the TCP connection timeout. For now we just assume that it's dropped by the firewall, not congestion. */ } else if (status == NSE_STATUS_TIMEOUT) { *port_state = PORT_FILTERED; *reason_id = ER_NORESPONSE; /* This just says that we cancelled a pending connection due to handleTimeouts. Ignore it. */ } else if (status == NSE_STATUS_CANCELLED) { *should_return = true; } else { /* We managed to connect! */ assert(status == NSE_STATUS_SUCCESS); *port_state = PORT_OPEN; *reason_id = ER_SYNACK; } } /* Sets port_state and reason_id based on connect_errno. Will cause the program to exit with a fatal error if we run out of file descriptors. */ void NsockTCPScanEngine::errnoToPortState(int connect_errno, int *port_state, int *reason_id) { switch (connect_errno) { case ECONNREFUSED: *port_state = PORT_CLOSED; *reason_id = ER_CONREFUSED; break; case ENETUNREACH: if (o.debugging) log_write(LOG_STDOUT, "Got ENETUNREACH from %s connect()\n", __func__); *reason_id = ER_NETUNREACH; break; case EACCES: if (o.debugging) log_write(LOG_STDOUT, "Got EACCES from %s connect()\n", __func__); *reason_id = ER_ACCES; break; case EMFILE: /* Let's increase the maximum descriptor number... Haha, joking. Let's just panic and blame it all on the user. */ fatal("Got 'Too many open files' error. Perhaps you set too high" " --max-parallelism (-M)?"); break; /* break just to make static analysis happy. */ default: if (o.debugging) log_write(LOG_STDOUT, "Got \"%s\" [#%d] from %s connect()\n", strerror(connect_errno), connect_errno, __func__); *reason_id = ER_UNKNOWN; } } /* Retransmits the probe without affecting the congestion control subsystem. */ void NsockTCPScanEngine::silentlyRetransmit(nsock_iod nsi, NsockProbe *probe) { if (o.verbose) log_write(LOG_STDOUT, "Detected likely self-connect on %s:%d\n", probe->nhss->target->targetipstr(), probe->portno); /* Close the socket. */ /* FIXME: Henri debugged the code looking for problems related to proxy scanning and found that there are segfaults which are fixed by setting sock_nsi to NULL after nsi_delete. Investigate this. */ nsi_delete(nsi, NSOCK_PENDING_NOTIFY); /* Let's pretend that this probe didn't exist for the congestion control system. */ probe->nhss->timing.num_replies_expected--; probe->nhss->timing.num_updates--; /* Retransmit the probe. */ sendProbeInternal(probe); } /* This is there only to pass a reference to nsse to connectHandler, which is static because we need to pass a pointer to function to nsock and we can't really take a pointer to a method of an instantiated object. */ struct NsockTCPScanEngineAndProbe { NsockTCPScanEngine *nsse; NsockProbe *probe; NsockTCPScanEngineAndProbe(NsockTCPScanEngine *nsse, NsockProbe *probe) : nsse(nsse), probe(probe) {} }; /* nsock_connect_* callback. This is where we the connect() result is interpreted so that we can call handleProbeEvent. */ void NsockTCPScanEngine::connectHandlerStatic(nsock_pool nsp, nsock_event evt, void *data) { NsockTCPScanEngineAndProbe *handler_args = (NsockTCPScanEngineAndProbe *) data; /* Extract the pointer to the probe and target from the event arguments. */ NsockTCPScanEngine *nsse = handler_args->nsse; NsockProbe *probe = handler_args->probe; delete handler_args; nsse->connectHandler(nsp, evt, probe); } void NsockTCPScanEngine::connectHandler(nsock_pool nsp, nsock_event evt, NsockProbe *probe) { enum nse_status status = nse_status(evt); enum nse_type type = nse_type(evt); nsock_iod nsi = nse_iod(evt); assert(type == NSE_TYPE_CONNECT); int connect_errno = nse_errorcode(evt); /* ID of a "REASON" that will appear when --reason was enabled. */ int reason_id; /* The port state we'll later pass to setPortState. PORT_HIGHEST_STATE means that we'll set it to filtered, but won't even consider sending retranmissions. (TODO: actually, this is kludgy.) */ int port_state = PORT_HIGHEST_STATE; bool should_return = false; statusToPortState(status, connect_errno, &port_state, &reason_id, &should_return); if (should_return) return; if (is_self_connect(nsi_getsd(nsi), probe->portno, probe->nhss->target)) { /* This is a self-connect, "ephemeral" port. See is_self_connect() comments for details. */ silentlyRetransmit(nsi, probe); } else { /* This is a normal probe. Just close the connection and process data. */ nsi_delete(nsi, NSOCK_PENDING_NOTIFY); handleProbeEvent(probe, reason_id, port_state); } } /* Start an nsock connection to the target and port given in NsockProbe variables. Does not manipulate the other data structures, like tryno, sent or active_probes. */ void NsockTCPScanEngine::sendProbeInternal(NsockProbe *probe) { nsock_iod sock_nsi = nsi_new(nsp, NULL); if (sock_nsi == NULL) fatal("Failed to create nsock_iod."); probe->sock_nsi = sock_nsi; /* Set the socket lingering so we will RST connections instead of wasting bandwidth with the four-step close. */ struct linger l; l.l_onoff = 1; l.l_linger = 0; nsi_set_linger(sock_nsi, &l); /* Spoof the source IP address. The following is copied from l_connect from nse_nsock.cc. */ if (o.spoofsource) { struct sockaddr_storage ss; size_t sslen; o.SourceSockAddr(&ss, &sslen); nsi_set_localaddr(sock_nsi, &ss, sslen); } /* Set the IP TTL value. */ nsi_set_ttl(sock_nsi, o.ttl); /* Set any IP options user requested. */ if (o.ipoptionslen) nsi_set_ipoptions(sock_nsi, o.ipoptions, o.ipoptionslen); /* Translate target's IP to struct sockaddr_storage. */ struct sockaddr_storage targetss; size_t targetsslen; if (probe->nhss->target->TargetSockAddr(&targetss, &targetsslen) != 0) fatal("Failed to get target socket address in %s", __func__); NsockTCPScanEngineAndProbe* data = new NsockTCPScanEngineAndProbe(this, probe); /* Initialize the connection. */ nsock_connect_tcp(nsp, sock_nsi, connectHandlerStatic, 1000, /* timeout */ (void *)data, (struct sockaddr *)&targetss, targetsslen, probe->portno); } /* Returns an estimate completion fraction that ScanProgressMeter will use to notify the user about the scan progress. */ double NsockScanEngine::getCompletionFraction() const { double total = 0; for (std::list::const_iterator it = nhss_list.begin(); it != nhss_list.end(); it++) { NsockHostScanStats *host = (*it); int maxtries = (host->max_tryno + 1); double thishostpercdone; /* scan_engine.cc uses a bit different formula there that I didn't understand: thishostpercdone = host->ports_finished * (maxtries - 1) + host->numprobes_sent; thishostpercdone /= maxtries * gstats->numprobes; */ thishostpercdone = ((host->current_port_idx * maxtries) + host->active_probes) / (double)(numports * maxtries); /* This can happen when we're scanning the final port and there are retranmissions waiting to come back. */ if (thishostpercdone >= 0.9999) thishostpercdone = 0.9999; total += thishostpercdone; } return total / targets_count; } /* Actually start the connection for the given probe and perform post-send tasks. */ void NsockScanEngine::sendProbe(NsockProbe *probe, struct timeval *now) { sendProbeInternal(probe); if (now == NULL) probe->sent = get_now(); else probe->sent = *now; /* Notify the congestion control mechanism that we sent a probe. */ timing.num_replies_expected++; timing.num_updates++; active_probes++; probe->nhss->last_sent = probe->sent; if (o.max_packet_send_rate != 0.0) TIMEVAL_ADD(send_no_earlier_than, send_no_earlier_than, (time_t) (1000000.0 / o.max_packet_send_rate)); if (o.min_packet_send_rate != 0.0) { if (TIMEVAL_SUBTRACT(send_no_later_than, *now) > 0) { /* The next scheduled send is in the future. That means there's slack time during which the sending rate could drop. Pull the time back to the present to prevent that. */ send_no_later_than = *now; } TIMEVAL_ADD(send_no_later_than, send_no_later_than, (time_t) (1000000.0 / o.min_packet_send_rate)); } probe->nhss->active_probes++; probe->nhss->timing.num_replies_expected++; probe->nhss->timing.num_updates++; probe->nhss->numprobes_sent++; } /* Start an nsock connection to the given target on a given port. */ void NsockScanEngine::sendNewProbe(NsockHostScanStats *nhss, unsigned short portno, int tryno, bool is_canary, struct timeval *now) { /* Prepare NsockProbe and run nsock_connect_tcp. */ NsockProbe *probe = new NsockProbe(); probes_outstanding.push_back(probe); probe->nhss = nhss; probe->portno = portno; probe->tryno = tryno; probe->is_canary = is_canary; sendProbe(probe, now); } /* Signal drops on probes that are timed out. */ void NsockScanEngine::handleTimeouts(struct timeval *now) { /* We can't remove probes from a list we're iterating over, so let's create a temporary list. */ std::list timedout_probes; for (std::list::const_iterator it = probes_outstanding.begin(); it != probes_outstanding.end(); it++) { NsockProbe *probe = (*it); /* Pick the struct timeout_info that makes most sense. */ long timeout = 0; if (probe->nhss->target->to.srtt > 0) { /* We have at least one timing value to use. Good enough, I suppose */ timeout = probe->nhss->target->to.timeout; } else if (to.srtt > 0) { timeout = to.timeout; } else { timeout = probe->nhss->target->to.timeout; /* It comes with a default */ } if (TIMEVAL_SUBTRACT(*now, probe->sent) > timeout) { timedout_probes.push_back(probe); } } /* Now that we've built the temporary list, let's signal drops. */ for (std::list::const_iterator it = timedout_probes.begin(); it != timedout_probes.end(); it++) { NsockProbe *probe = (*it); /* Close the socket. */ /* FIXME: Henri debugged the code looking for problems related to proxy scanning and found that there are segfaults which are fixed by setting sock_nsi to NULL after nsi_delete. Investigate this. */ nsi_delete(probe->sock_nsi, NSOCK_PENDING_NOTIFY); handleDrop(probe, now, ER_NORESPONSE); } } /* The callback that is ran when we decide that sendNextProbes cannot send any probes right now and we need to schedule another call of it. */ static void send_next_probes_callback(nsock_pool nsp, nsock_event evt, void *data) { assert(nse_status(evt) == NSE_STATUS_SUCCESS); NsockScanEngine* nsse = (NsockScanEngine *)data; nsse->sendNextProbes(); } /* Tells whether it is okay to send any probes to this host at this moment. The argument when_ok is updated if it's not possible, but we've found a new soonest moment to fire a probe. */ bool NsockHostScanStats::sendOK(struct timeval *now, unsigned long *when_ok) { assert(when_ok != NULL); /* Let's calculate when it's okay to send a probe if the scan delay is in effect. */ long since_last_sent = TIMEVAL_SUBTRACT(*now, last_sent); unsigned long when_delay_ends = (o.scan_delay * 1000) - since_last_sent; assert(since_last_sent >= 0); bool waiting_for_delay = (o.scan_delay > 0) && (since_last_sent <= (o.scan_delay * 1000)); if (waiting_for_delay && (*when_ok == 0 || (when_delay_ends < *when_ok))) { *when_ok = when_delay_ends; } struct ultra_timing_vals usable_timing; getProperTiming(&usable_timing); bool host_is_congested = active_probes >= usable_timing.cwnd; if (o.debugging > 4) log_write(LOG_STDOUT, "NsockHostScanStats[%s]::sendOK - ", target->targetipstr()); if (isDone(now)) { if (o.debugging > 4) log_write(LOG_STDOUT, "host is done\n"); return false; } if (o.min_packet_send_rate != 0.0) { long since_send_no_later_than = TIMEVAL_SUBTRACT(nsse.getSendNoLaterThan(), *now); if (since_send_no_later_than) { *when_ok = since_send_no_later_than; } else { log_write(LOG_STDOUT, "returning true because of --min-rate\n"); return true; } } if (host_is_congested) { if (o.debugging > 4) log_write(LOG_STDOUT, "host is congested\n"); return false; } if (waiting_for_delay) { if (o.debugging > 4) log_write(LOG_STDOUT, "waiting for delay\n"); return false; } if (TIMEVAL_AFTER(nsse.getSendNoEarlierThan(), *now)) { if (o.debugging > 4) log_write(LOG_STDOUT, "too early to send\n"); /* TODO: I think I didn't confuse the units, but I should make extra sure. */ *when_ok = TIMEVAL_SUBTRACT(nsse.getSendNoEarlierThan(), *now); return false; } /* Otherwise, it's all okay. */ if (o.debugging > 4) log_write(LOG_STDOUT, "returning true\n"); return true; } /* Tells whether we should send a probe canary. */ bool NsockHostScanStats::shouldSendCanary(struct timeval *now) { bool ret = TIMEVAL_SUBTRACT(*now, last_received) > TIMING_CANARYTIME && TIMEVAL_SUBTRACT(*now, last_canary_sent) > TIMING_CANARYTIME && target->pingprobe.type != PS_NONE && numprobes_sent >= lastping_sent_numprobes + 10; return ret; } /* Tells whether we're done scanning the host. */ bool NsockHostScanStats::isDone(struct timeval *now) { return ((current_port_idx >= nsse.getNumPorts() && retransmission_queue_size == 0) || target->timedOut(now)); } /* Tells whether it is possible to send any new probes at the per-group level. TODO: inactive_hosts isn't directly relevant here - probably should be passed as an instance variable. */ bool NsockScanEngine::sendOK(unsigned int inactive_hosts) { if (inactive_hosts >= targets_count) { if (o.debugging > 4) log_write(LOG_STDOUT, "NsockScanEngine::sendNextProbes(): " "inactive_hosts >= targets_count\n"); return false; } if (active_probes >= getProbeLimit()) { if (o.debugging > 4) log_write(LOG_STDOUT, "NsockScanEngine::sendNextProbes(): " "active_probes[%d] >= getProbeLimit()[%d]\n", active_probes, getProbeLimit()); return false; } return true; } /* Actually sends a new probe for a given host. */ void NsockScanEngine::doSendNextProbe(struct timeval *now, NsockHostScanStats *nhss) { /* Let's see if we should send a canary, retransmission or an ordinary probe. */ if (nhss->shouldSendCanary(now)) { nhss->last_canary_sent = *now; nhss->lastping_sent_numprobes = nhss->numprobes_sent; sendNewProbe(nhss, nhss->target->pingprobe.pd.tcp.dport, 0, true, now); } else if (nhss->retransmission_queue_size > 0) { NsockProbe *probe = nhss->retransmission_queue.back(); nhss->retransmission_queue.pop_back(); nhss->retransmission_queue_size--; probe->tryno++; sendProbe(probe, now); } else { assert(nhss->retransmission_queue_size == 0); unsigned short portno = portarray[nhss->current_port_idx]; sendNewProbe(nhss, portno, 0, false, now); nhss->current_port_idx++; } } /* Performs a single attempt to send a probe. Returns true if the attempt succeeded and false if it failed. */ bool NsockScanEngine::trySendAnyNextProbe(struct timeval *now, unsigned long *when_ok, unsigned int *inactive_hosts, unsigned int *complete_hosts, std::map *inactive_hosts_map) { assert(now != NULL); if (!sendOK(*inactive_hosts)) return false; if (next_nhss == nhss_list.end()) next_nhss = nhss_list.begin(); NsockHostScanStats *nhss = (*next_nhss); assert(active_probes >= 0 && nhss->active_probes >= 0); /* Have we found a sooner when_ok? */ if (!nhss->sendOK(now, when_ok)) { if ((*inactive_hosts_map)[nhss] == false) { /* This host is either congested or complete. Mark it as inactive. */ (*inactive_hosts_map)[nhss] = true; (*inactive_hosts)++; if (nhss->isDone(now)) { (*complete_hosts)++; } } } else { doSendNextProbe(now, nhss); } next_nhss++; return true; } /* Sends new probes, respecting per-host and group congestion windows. */ void NsockScanEngine::sendNextProbes() { struct timeval now = get_now(); handleTimeouts(&now); /* We'll keep track of the number of hosts that can't be scanned anymore as the loop progresses. TODO: code review question: maybe something like: bool inactive_hosts_arr[targets_count]; would be better? */ unsigned int inactive_hosts = 0; unsigned int complete_hosts = 0; std::map inactive_hosts_map; /* If we can't send any packets, when can we expect to be able to? */ unsigned long when_ok = 0; if (keyWasPressed()) { SPM->printStats(getCompletionFraction(), NULL); log_flush(LOG_STDOUT); } if (o.debugging >= 3) printTimingStats(&now); while(true) { if(!trySendAnyNextProbe(&now, &when_ok, &inactive_hosts, &complete_hosts, &inactive_hosts_map)) break; } /* If there are no outstanding probes, but we're not done yet, we're need to to set up a timer that will send next probes. */ if (active_probes == 0 && complete_hosts < targets_count) { assert(when_ok > 0); nsock_timer_create(nsp, send_next_probes_callback, when_ok / 1000, this); } /* If all hosts are complete/timed out, we can take no action even though active_probes == 0. This will make the main loop return, effectively ending the scan. */ } /* Creates nhss_list from a vector of Target instances. */ std::list NsockScanEngine::populateNhssList( const std::vector &Targets_arg, struct timeval now) { std::list ret; for (std::vector::const_iterator tI = Targets_arg.begin(); tI != Targets_arg.end(); tI++) { Target *target = (*tI); NsockHostScanStats *nhss = new NsockHostScanStats(*this, target, &now); ret.push_back(nhss); } return ret; } /* Initialize ultra_timing_vals. */ void NsockScanEngine::initUltraTimingVals(ultra_timing_vals &timing) { /* Initialize the congestion control mechanism. */ timing.cwnd = perf.group_initial_cwnd; /* Will be reduced if any packets are dropped anyway: */ timing.ssthresh = perf.initial_ssthresh; timing.num_replies_expected = 0; timing.num_replies_received = 0; timing.num_updates = 0; timing.last_drop = get_now(); } /* NsockScanEngine constructor. Does most of the initialization work that could be done without actually signalling a scan start. */ NsockScanEngine::NsockScanEngine(const std::vector &Targets_arg, const u16 *portarray_arg, const int numports_arg) : targets_count(Targets_arg.size()), portarray(portarray_arg), numports(numports_arg), nhss_list(populateNhssList(Targets_arg, get_now())) { struct timeval now = get_now(); send_no_earlier_than = now; send_no_later_than = now; SPM = new ScanProgressMeter("Connect Scan"); active_probes = 0; if (o.debugging) log_write(LOG_STDOUT, "nsock_scan() begins.\n"); /* Initialize the Nsock pool. */ nsp = nsp_new(NULL); if (nsp == NULL) fatal("Failed to create nsock_pool."); /* Bind to the device specified with -e. */ nsp_setdevice(nsp, o.device); nsock_set_log_function(nsp, nmap_nsock_stderr_logger); nmap_adjust_loglevel(nsp, o.packetTrace()); initUltraTimingVals(timing); initialize_timeout_info(&to); } void NsockScanEngine::stopTimeoutClocks(struct timeval &now) { for (std::list::const_iterator it = nhss_list.begin(); it != nhss_list.end(); it++) { NsockHostScanStats *nhss = (*it); Target *target = nhss->target; target->stopTimeOutClock(&now); } } /* Main scanning function. Schedules the first probes and runs the nsock main loop. */ void NsockScanEngine::run() { o.current_scantype = CONNECT_SCAN; struct timeval now = get_now(); /* Store the information that we started scanning targets and reset their pingprobe types if necessary. */ for (std::list::const_iterator it = nhss_list.begin(); it != nhss_list.end(); it++) { NsockHostScanStats *nhss = (*it); Target *target = nhss->target; target->startTimeOutClock(&now); if (target->pingprobe.type != PS_CONNECTTCP) target->pingprobe.type = PS_NONE; } /* Schedule the first probes. Setting current_port_idx to -1 and next_nhss to Targets.end() will force a reset in the next send_next_probe() call. */ next_nhss = nhss_list.end(); sendNextProbes(); /* Jump into the main loop. Handle any unexpected errors. */ /* FIXME: the current design of the loop causes very deep stack traces and might be a cause of stack overflows. Redesign this by calling nsock_loop multiple times. Caught by Henri. */ enum nsock_loopstatus looprc = nsock_loop(nsp, -1); if (looprc == NSOCK_LOOP_ERROR) { int err = nsp_geterrorcode(nsp); fatal("nsock_scan: unexpected nsock_loop error. Error code %d (%s)", err, socket_strerror(err)); } /* We finished scanning the targets, so let's store this information in their data structures. */ stopTimeoutClocks(now); /* The main loop is done and so is nsock_scan. Destroy the pool. */ nsp_delete(nsp); } void nsock_scan(const std::vector &Targets, const u16 *portarray, const int numports) { NsockScanEngine::getScanPerformanceVars().init(); NsockTCPScanEngine(Targets, portarray, numports).run(); }