Fwd: kyxspam: scanners (was: Re: [Snort-users] Security scanning software)

Dragos Ruiu dr at ...50...
Sun Oct 15 22:05:07 EDT 2000

On Sun, 15 Oct 2000, Al Huger - Mail Account wrote:
> On Thu, 12 Oct 2000, Jerry Shenk wrote:
> > What do people here use for security scanning - that is, testing your
> > security setups to make sure you've got things covered?  I typically use
> > nmap, nessus and to a lesser degree Saint.  How do they compare with ISS,
> > CyberCop or other commercial apps?
> All those open source tools together are probably comparable to either ISS
> or CyberCop Scanner in terms of testing an IDS. If you are using it to
> test individual signatures I would suggest sticking with Nessus or another
> Open Source solution given that you have access to the code and can see
> precisely what the check is doing, a luxury you do not have with
> commercial scanners (except for CyberCop CASL modules). 
> In terms of how they compare for use in auditing etc. that's a religous
> issue. There was a thread on the securityfocus pen-test list but it
> mostly dealt with their effectiveness in auditing, IMO both CCS and ISS
> have up's and downs with neither being overall 'better'.

Thought this would be of interest....


----------  Forwarded Message  ----------
Subject: kyxspam: scanners
Date: Mon, 25 Sep 2000 20:41:54 -0700
From: Dragos Ruiu <dr at ...381...>

(Meaningful scanner reviews are too rare.... but here is one.  --dr)


Secure strategies A year-long series on the fundamentals of information
systems security Part 2

Audits, Assessments & Tests (Oh, My) 

On the surface, all vulnerability assessment scanners perform essentially the same way. Here's how to decide which one-if any-is right for your requirements. BY AL BERG 

EDITOR'S NOTE: The following is part two of a four-part series on information systems security testing. This installment explores the topic of vulnerability assessment, including a look at today's technologies, commercial and freeware products and best-practice methodologies. As with all Secure Strategies articles, this feature is targeted to beginner/novice infosec professionals, though more experienced practitioners may also find it useful as an update on new vulnerability assessment product releases. 

 You go to the doctor and he pokes your stomach with a stick. "Does this hurt?" he asks, poking your pancreas. "No," you reply through gritted teeth. "How about this?" he continues, poking your liver. As you scream out in pain, the doctor makes a note on your chart, and says something like,"Hmmm...guess we need to look more closely at that liver of yours." Congratulations. What you've just undergone closely resembles an information systems vulnerability assessment. Vulnerability scanners are, essentially, software that check the relative health of computers and other devices by probing for a finite number of problems that could leave them open to attack. Some examples include:

Buffer overflows, in which sending too much input to a program causes it to fail, allowing the attacker to execute rogue commands on the host system.

Back doors left in programs by vendors; these are meant to ease support, but if exposed, can give an attacker entry into a system.

Bugs that can be exploited to force a program to perform an unauthorized operation.

Large, complex pieces of software, such as operating systems, are likely to have such vulnerabilities. They tend to be written by many different people and must have the flexibility to operate under widely varied conditions. According to the Bug-Traq mailing list (a major Internet news source for software vulnerabilities), during 1999 and the first half of 2000, some 130 vulnerabilities were identified in Windows NT 4.0, 21 in Windows 2000 and 84 in various Linux versions. Most vulnerabilities, however, are introduced to systems as a result of simple, easily correctable misconfigurations by systems administrators.

Knowing which vulnerabilities may lurk in your systems' software should be an important part of your security strategy. In many cases, major security problems can be cured with the simple application of a patch or new software version-assuming you know the fix is needed in the first place. Scanning systems for vulnerabilities is a popular activity among crackers looking for easy entryways to corporate computers. "Every second, there are 80 port scans being done on computers around the world," says Ronald Van Geijn, a product manager at Axent Corp. That's 80 attempted intrusions per second, folks. Using vulnerability scanners, systems admins can close the holes that bad guys look for before they attack.

In this article, we'll lay out some of the key issues you need to consider when deciding whether or not to use a security scanner. We'll also outline criteria for selecting the right product to use, and discuss how to interpret the results of your scans. While there are lots of scanners on the market today, for this article we'll concentrate on six of the leading products in terms of reputation and market share.

 The Numbers Game
Like many antivirus products, the vulnerability scanner relies on a "signatures database" to identify potential problems. As the security community identifies new problems, the database must be updated before the scanner product can detect its newest targets. For this reason, both the frequency of database updates and the number of detected vulnerabilities are important factors in evaluating these products.

When you try to compare vulnerability scanners to each other, an obvious problem arises. Scanner vendor A says it can spot 1,345 different security problems, while vendor B's product claims to pinpoint "over 1,800" separate vulnerabilities. Is scanner B that much better than scanner A? Not necessarily; it all depends on how you define a vulnerability.

For example, let's say that Program X is known to crash and provide a remote user with a root shell if he enters a parameter of more than 254 characters on the command line. Let's say that Program X will also crash to a root shell if you answer one of its prompts with a response of more than 65,535 characters. Is this one vulnerability, or two? It depends on how you name them. You could lump both vulnerabilities under "Program X has multiple buffer overflows," or you could describe the two vulnerabilities separately-"Pro-gram X command-line buffer overflow" and "Program X input-field buffer overflow." It's possible to make a case for either scheme. Multiply this by every possible case and it quickly becomes apparent how comparing the number of vulnerabilities detected in Scanner A to that in Scanner B is like comparing apples to oranges.

One organization trying to bring some order to the world of security vulnerabilities is the Common Vulnerabilities and Expo-sures (CVE) project (http://cve.mitre.org). CVE's sponsoring organization, MITRE Corp., is a nonprofit corporation that works closely with the federal government on technology projects. The CVE project is an effort to standardize the names of security vulnerabilities so that users, vendors and security professionals can use a common language when discussing them. For instance, CVE-2000-0162 describes a problem in IE 4.x and 5.x that allows an attacker to read files on a target machine via a malicious Java applet. Before a security problem is defined and added to the CVE dictionary, it is reviewed by an editorial board of industry experts, vendors and educators. 

When using the CVE list, it is important to understand what CVE isn't. The CVE list does not provide fix or impact information about the problems it describes-that type of information can be found in other databases, which use the CVE identifiers to enhance their content (examples of such databases include those at www.secu rityfocus.com or various vendors' sites).

Many vendors are working to make their products CVE-compatible (an updated list of these vendors and their products' compatibility can be found on the cve.mitre. org site). In order to be declared CVE-compatible, a product must allow users to search for vulnerabilities with the CVE identifiers, output information using CVE names and IDs, and map vulnerabilities to the CVE database.

 Buying Expertise
When you buy a vulnerability scanner, you are buying expertise. This knowledge base includes databases of vulnerabilities, the automatic-repair capabilities (if any) of the products and explanatory materials that help you deal with the problems found. You are also buying updates of new secu-rity problems discovered at some future point. Hence, before choosing a vulnerability-scanning product, you should take a careful look at the team supporting it.

First, and most important, determine if the team is doing original research or merely reading through Internet mailing lists looking for reports of problems discovered elsewhere. A team doing its own R&D is usually more skilled and innovative than one that simply trolls the message boards. A good indicator of the tech- nical savvy of a vendor's team is the number and quality of papers, advisories and tools it has authored. 

The Web site for BindView's RAZOR team (http://razor.bindview.com) lists 10 original security advisories that the team has issued since February 1999. The site also includes some excellent papers on vulnerability classification and distributed denial-of-service (DDoS) attacks. RAZOR's "Tools" page includes NT and Unix tools that can help systems administrators neutralize DDoS "zombie" machines, as well as a group of useful utilities for security professionals. The site's "Links" section is comprehensive, with listings of many online security resources and presentations made by team members at industry events. What's key here is that almost all of this material was written by RAZOR team members, indicating a high level of R&D activity and knowledge.

Network Associates' security team is called COVERT (COmputer Vulnerability Emergency Response Team) (www.pgp. com/asp_set/covert/default.asp). COVERT has authored 11 security advisories since the beginning of 1999 (41 since 1996), all of which are comprehensive and easy to understand. COVERT does not offer much in the way of other information on its site, however, relying instead on links to outside sources of information like Carnegie Mellon University's Computer Emergency Response Team (CERT). Unlike RAZOR's site, there are no security tools available for download on the COVERT site. Internet Security Systems's formidable X-Force (http://xforce.iss.net) has been quite busy this year, issuing 40 security alerts through six months. Their Web site offers a lot of security content-a searchable database of alerts, links to useful sites, presentations, ISS product manuals and mailing lists to join. However, the tools section (called ProtoWorx) was a bit of a disappointment; there was n
othing to see except a blurb about a forthcoming tool and an SQL password cracker that had been absorbed into the company's database scanning product. 

Axent's security group is called the SWAT team (www2.axent.com/swat). The SWAT home page gets right down to business by listing security advisories-most of which are in the form of updates to their products-that address new threats. Axent has listed 35 security advisories since the beginning of the year, but a number of these are program updates and notes on advisories issued by other organizations. I saw very little evidence of original research on the SWAT site.

Symantec's security team seems to be keeping a very low profile, and I was not able to find a specific page showcasing their research. According to the company, the Symantec team focuses on collecting vulnerability information from open sources rather than performing its own research.1 All in all, I found the BindView site to be the most impressive of the lot. I got the feeling that the RAZOR team eats, sleeps and breathes security, and really knows its stuff. The RAZOR team includes "Simple Nomad," a "white hat" hacker who has an impressive background uncovering vulnerabilities in products from Microsoft, Novell and other vendors.

 The Products: Lots of Similarities
Each of the market-leading products has the same basic feature set: They will automatically scan one or more hosts to spot vulnerabilities contained in their databases, and present the user with a series of reports listing problems found and suggestions for corrective action. While the vendors squabble about whose product is able to find the largest number of vulnerabilities, the reality is that 95 percent of the vulnerabilities these products scan for were harvested from publicly available sources such as BugTraq.

The value that the vendors add to this information-and the features that differentiate one from another-are contained in the products' user interfaces, reports and extra ease-of-use features. Fortunately, all the vendors offer downloadable evaluation versions of their products, allowing potential purchasers to get an idea of how the scanners work before plunking down cash.

 The Scanning Process
While the basic scanning process for each product is the same, some of the products add useful bells and whistles. NAI's Cyber-Cop, for instance, offers a multithreaded scanning engine that's able to test 100 hosts at the same time, which speeds up enterprise assessments (see screen, p. 61). CyberCop can also detect the operating system of the device being scanned and will thus perform only the tests appropriate to that OS-also accelerating an assessment. Finally, CyberCop offers a "probe-only" mode, which is useful for mapping networks and devices when you're not ready to test for vulnerabilities.

CyberCop also offers a scripting language called CASL (Custom Attack Script-ing Language), which allows users to build tests for new attacks without waiting for the vendor's updates. With CASL's GUI interface, users can manipulate the contents of IP packets in ways that simulate attacker tactics, such as setting option bits and fragmenting packets in an attempt to bypass firewalls. (ISS offers a similar facility, called FlexCheck, while Axent requires you to purchase a separate product (Enterprise Security Manager, or ESM) for customized testing.) The scanning engine in Axent's NetRecon (see screen, above) uses data from parallel scans of many systems to enhance its assaults upon additional systems-a heuristic capability that the company calls "capability progressive scanning." Axent claims that this technique allows their product to uncover larger numbers of potential problems by exploiting cross-system weaknesses. For those of you with NetWare servers on your networks, it's worth
 noting that Axent's NetRecon offers a number of checks specifically designed to locate known problems on the Novell platform.

ISS's Internet Scanner, part of the company's RealSecure suite, is able to query network routers to determine the IP structure of the target networks, freeing the administrator from the task of specifying the networks to be scanned. The ISS product (see screen, p. 65) is compatible with Windows 2000 (as a scanner platform as well as a scanned target), and can route scanner alerts in real time to the ISS Real-Secure console. Internet Scanner integrates with the rest of the RealSecure suite, including System Scanner, which allows admins to enforce system-configuration policies; and Database Scanner, which checks databases for security problems.

Symantec's Network Security (SNS) product2 takes a different approach to scanning compared to other products. SNS is designed for the conservative network administrator; its positively Hippo-cratic philosophy is "first, do no harm," according to Corey King, Symantec's director of business development. Rather than scanning networks by attempting to exploit vulnerabilities, Retriever/SNS uses its database of vulnerabilities to gain a high level of confidence that a problem exists. This allows the administrator to perform more rigorous testing on a nonproduction network, causing less downtime, according to King.

 Making the Fix
NAI's CyberCop includes an additional layer of checks, allowing administrators to verify that security policies are being implemented properly at the desktop level. These checks allow the comparison of system-configuration values (e.g., those in the Windows 9x/NT/2000 registry) against "acceptable" ranges. In addition to reporting on systems that are misconfigured, CyberCop can automatically restore the errant configurations to their proper state. BindView's HackerShield also has the ability to automatically perform corrective actions on computers with security problems. This "AutoFix" feature (see screen, p. 66) can save the network administrator a lot of time and shoe leather.

 Reporting In
The reports generated by the vulnerability scanner are, obviously, vital; they provide a description of the problems found and a roadmap to fixing them. Clear, detailed and easy-to-understand reports are also key to evaluating these products, and each of the scanners discussed here offers a variety of reports detailing the security issues they find.

ISS's Internet Scanner offers reports that can be tailored to the needs of technicians (who need the most detail, since they have to fix the holes found by the scanner); line managers (who need enough detail to track the technicians' progress); and executives (who need a quick, easy-to-understand snapshot of the security issues their organizations face). These multi-level reports render the task of disseminating information easier, and make the business case for security that much stronger.

Axent's NetRecon offers the administrator a nifty feature: an inside look at how attackers would exploit the vulnerabilities they uncover to access the organization's systems. Double-clicking on a particular vulnerability will show the exact steps-down to the commands typed-that an attacker can use to gain entry. NetRecon's report data are stored as Access databases and can be viewed with Crystal Reports, exported to HTML or accessed from user-written custom report programs. The product also offers integration with major network-management platforms, including BMC Patrol, HP OpenView and Tivoli NetView, allowing managers to make vulnerability-assessment reports part of their normal net-management data.

For overall technical detail, though, BindView's HackerShield reports get my vote. The RAZOR team's in-depth knowledge of vulnerabilities, their ramifications and what you'll need to do to fix them really comes through in these reports. On the downside, techies will need to distill this information themselves to make it readable for their managers and executives.

 Getting the Latest 411
Except for Axent and ISS, all of the commercial products reviewed here offer automated updates of their vulnerability databases via FTP or e-mail; NetRecon and Internet Scanner updates must be retrieved and installed from Axent's or ISS's Web site. Database updates are digitally signed to prevent an attacker from inserting new "vulnerability checks," which could elicit denial-of-service attacks from the very tool that is supposed to help you avoid them. And, while all of the vendors offer regularly scheduled updates, they also issue emergency updates if particularly widespread or threatening problems suddenly arise.

Symantec Network Security version 2.0 (due out before the end of the year) is supposed to offer the "LiveUpdate" technology that is already present in the company's antivirus products. Its update engine would allow SNS 2.0 to receive new vulnerability signatures on an almost-real-time basis-a very attractive feature, given the dynamic nature of the threat landscape.

An Open-Source Alternative
If you're up to braving the difficulties of the open-source world, a Unix scanner called Nessus is worth a look. The goals of the Nessus Project are to provide the Internet community with a "free, powerful, up-to-date and easy-to-use remote secu- rity scanner." The Nessus scanner is a client/server application. The actual scanning engine, called "nessusd," can run on a separate computer from the front-end client. This means that a security professional can run a Nessus server in one location, access it from a front-end somewhere else, and scan a third machine at yet another location. Access to the scanning engine can be protected using passwords or public-key cryptography, and the system administrator can configure the server to limit the hosts or subnets that can be scanned from a server. Because Nessus is a Unix/Linux-based product, there is no Windows 95 or NT version of the Nessus server, although there is a front-end client for 32-bit Win-dows environments. There is also a Jav
a-based client, which allows browser-based access to the Nessus server.

As an open-source product, Nessus has the same potential pitfalls and advantages as other products developed by the Web community (see box, p. 66). To begin with, there is no company that stands behind and supports Nessus. Distribution can be a pain as well. "Open source" generally means you get only the source code, so in many cases you'll have to compile the product yourself-a time-consuming, confusing and intimidating process for "Windows Weenies," who are accustomed to nice, shrink-wrapped software with GUI install programs.

However, Nessus has a lot of advantages to recommend it. First of all, you can see the source code, and not only that of the product itself, but also of all of the secu-rity-checking scripts. Also, scripts (plugins) can be written in C or in the Nessus Attack Security Language (NASL). Plus, there's a large community of users writing scripts for Nessus, and the list of attacks detected gets longer on a daily basis. It's a pretty good bet that, as new vulnerabilities are discovered, new scripts will be made quickly available on the Nessus site. (Nessus includes an agent program, which will download the latest attack scripts automatically if you wish.)

Once you get Nessus up and running, it is very easy to use; the client side is graphical, and reports are detailed and easy to interpret. In addition, Nessus reports can be saved in a number of formats, including text and HTML. Predictably, the quality of data in Nessus reports will vary, since each attack script (and its integrated mitigation information) is written by a different contributor to the project. Ac-cordingly, Nessus reports are more appropriate for the techie than the business manager in your organization. Nessus has developed quite a following in the security community. Indeed, the U.S. mirror of the Nessus download site is hosted by the Treasury Department's Computer Investigative Specialist Program. Nessus was also the number-one rated tool in a recent survey run by insecure.org, a security Web site. Nessus is definitely worth a look, either as a standalone tool or as an adjunct to other commercial security scanners you may already use.

The Crystal Ball
What does the future hold for vulnerability-scanner technology? According to the vendors I spoke to, more chaos and confusion...and a bit of hope. First, chaos and confusion. The number of potential vulnerabilities, exposures and threats is on the rise (as always), and (despite the CVE project) the industry hasn't yet agreed on definitions for the most common problems. This means that the war of numbers between vendors will continue: the numbers of vulnerabilities that vendors claim to check and/or fix will continue to be difficult or impossible to compare, and tool selection will continue to be a challenge for users.

This problem also means that the reports presented by these tools will continue to include a lot of "noise," or false positives. Vendors tend to err on the side of caution when reporting vulnerabilities, giving users more information rather than less, but forcing them to interpret a lot of it. On the hopeful side, efforts to classify vulnerabilities and exposures according to the true threat they pose to systems will help make scanners more useful. A recent step in this direction, the SANS "Top Ten Threat List" (www.sans.org/topten.htm) provides a guide to the vulnerabilities and exploits that account for the most commonly reported security breaches. Weaknesses in the BIND domain name service, faulty CGI programs and problems with Remote Procedure Calls (RPCs) top this list, which SANS complied with the help of 30 security agencies and organizations. Other prominent threats included Microsoft RDS vulnerabilities, Sendmail attacks and flaws in POP and IMAP configurations. 

Tool vendors are looking at the SANS list and others to help provide reports focused on problems systems administrators need to fix first. However, Axent's technical product manager Harold Toomey cautions that the Top Ten list may give end users a false sense of security. "They think that, by fixing these issues, their systems are safe," says Toomey, "but in truth they are only as safe as their weakest link." I think we'll also see a lot more integration between vulnerability assessment tools and other parts of the security toolset, such as intrusion-detection monitors. One of the issues facing users of IDSes is the sheer flood of data they provide. By coupling vulnerability-assessment tools with IDS tools, vendors can help to prioritize alerts, so that attacks on machines that are likely to be vulnerable get a higher priority than attacks that are bound to fail.

 Buyer Beware
Before you run out and scan your networks for vulnerabilities, be aware that some of the tests performed by these products have the potential to crash the host being scanned. It's important to understand what tests you are running and the effect they may have on your network; all of the products mentioned in this article provide the user with pre-scan warnings when potentially disruptive tests are going to be performed. As a rule, DON'T run these tests during mission-critical periods (like year-ends).

Like other security tools, vulnerability scanners are just one part of an overall enterprise security plan. They should be complemented with firewalls, antivirus tools and, for some organizations, intrusion detection tools. And remember that no tool is a replacement for an understanding of basic security concepts and your knowledge of the value of your systems and data.

 AL BERG (aberg at ...644...) is a New York City-based consultant with the Security Consult-ing Practice of Mentor Technologies Inc. (formerly Chesapeake Network Solutions), an IT training and consulting firm.


Network Associates Inc. 

Internet Scanner 
Internet Security Systems Inc. 

BindView Development Corp. 

Axent Technologies Inc. 

Retriever (Symantec Network Security) 
Symantec Corp. 

The Nessus Project 

Dragos Ruiu <dr at ...50...>   dursec.com ltd. / kyx.net - we're from the future 
gpg/pgp key on file at wwwkeys.pgp.net

More information about the Snort-users mailing list