[Snort-devel] unofficial testing

Martin Roesch roesch at ...48...
Fri Feb 23 01:12:54 EST 2001


Ok, I've made a couple of tweaks to fix most of the stuff covered here. 
Rules 1-10 are rejected properly now (number 5 isn't, that's actually a
valid Snort rule, it'll pop the default alert message if it detects that
activity).  Suprisingly, a lot of the tests were properly handled in the
1.7 code (tests 6-10).  I fixed up the depth/offset problem as well. 
Case 20 is a bit more of a problem and I haven't taken the time to
address that one.

The -DDEBUG code has been working well for as long as I can remember....

I'm all for setting up a testing regimen for 2.0, testing is one of the
things that we're most lacking in Snort currently.  I do the testing I
can and let the users check out what they're interested in, but it
doesn't always translate to a full test set for the system.

   -Marty

Todd Lewis wrote:
> 
> I was just reading Keith Bostic's keynote address to BSDCon 2000.  He
> said:
> 
>         As I may have already mentioned, I think it's a demonstrable
>         fact that, if a line of software hasn't been tested, it almost
>         certainly doesn't work.  Software reliability depends on testing,
>         and therefore, on modularity, because the software code base
>         in most projects is too big to depend on anything other than
>         module testing.
> 
> So, in the spirit of our rules discussion, I came up with some examples
> of bogus rules; they are attached.  Here are the results for 1.6.3.
> 
> First for ten ways to truncate a rule:
> 
> 1) Segmentation fault
> 2) Segmentation fault
> 3) Segmentation fault
> 4) Segmentation fault
> 5) (Mistakenly loads faulty rule)
> 6) Segmentation fault
> 7) Segmentation fault
> 8) Segmentation fault
> 9) Segmentation fault
> 10) Segmentation fault
> 
> That's right, out of 10 ways to truncate a rule, none of the ten are
> properly handled by snort in its present form.
> 
> Now for some intra-rule mischief:
> 
> 11) (properly tries to resolve name)
> 12) (properly detects bad port number)
> 13) (properly detects bad IP)
> 14) (properly detects bum mask)
> 15) (properly detects negative mask)
> 16) (appears to detect escaped quotes)
> 17) (accepts absurdly-large offset; arguably a bug)
> 18) (accepts absurdly-large depth; arguable a bug)
> 19) (accepts capital case)
> 20) (rejects valid but long line with bogus error)
> 
> This was actually a pretty good performance.
> 
> By the way, compiling snort with "-DDEBUG" has not worked for as
> long as I've been hacking snort, which admittedly has not been
> very long.  Also, you can pretty much forget about using the
> following flags to gcc:
> 
>         -Wstrict-prototypes
>         -ansi
>         -pedantic-errors
> 
> All of this leads me to a question: would anyone be averse to some sort
> of testing regimen and a release checklist being contributed for 2.0?
> The regimen would include test rule files to make sure the parser
> is not broken as well as, maybe, unit tests of internal components.
> The checklist would include:
> 
>         - running the tests
>         - compiling and testing with debugging enabled
>         - compiling with strict compiler checking
> 
> Would anyone be interested in helping with such a project?
> 
> --
> Todd Lewis                                       tlewis at ...120...
> 
>   God grant me the courage not to give up what I think is right, even
>   though I think it is hopeless.          - Admiral Chester W. Nimitz
> 
>   ------------------------------------------------------------------------
>                         Name: snort-test.tar.gz
>    snort-test.tar.gz    Type: Unix Tape Archive (application/x-tar)
>                     Encoding: BASE64

--
Martin Roesch
roesch at ...48...
http://www.snort.org




More information about the Snort-devel mailing list