[Snort-devel] unofficial testing
roesch at ...48...
Fri Feb 23 01:12:54 EST 2001
Ok, I've made a couple of tweaks to fix most of the stuff covered here.
Rules 1-10 are rejected properly now (number 5 isn't, that's actually a
valid Snort rule, it'll pop the default alert message if it detects that
activity). Suprisingly, a lot of the tests were properly handled in the
1.7 code (tests 6-10). I fixed up the depth/offset problem as well.
Case 20 is a bit more of a problem and I haven't taken the time to
address that one.
The -DDEBUG code has been working well for as long as I can remember....
I'm all for setting up a testing regimen for 2.0, testing is one of the
things that we're most lacking in Snort currently. I do the testing I
can and let the users check out what they're interested in, but it
doesn't always translate to a full test set for the system.
Todd Lewis wrote:
> I was just reading Keith Bostic's keynote address to BSDCon 2000. He
> As I may have already mentioned, I think it's a demonstrable
> fact that, if a line of software hasn't been tested, it almost
> certainly doesn't work. Software reliability depends on testing,
> and therefore, on modularity, because the software code base
> in most projects is too big to depend on anything other than
> module testing.
> So, in the spirit of our rules discussion, I came up with some examples
> of bogus rules; they are attached. Here are the results for 1.6.3.
> First for ten ways to truncate a rule:
> 1) Segmentation fault
> 2) Segmentation fault
> 3) Segmentation fault
> 4) Segmentation fault
> 5) (Mistakenly loads faulty rule)
> 6) Segmentation fault
> 7) Segmentation fault
> 8) Segmentation fault
> 9) Segmentation fault
> 10) Segmentation fault
> That's right, out of 10 ways to truncate a rule, none of the ten are
> properly handled by snort in its present form.
> Now for some intra-rule mischief:
> 11) (properly tries to resolve name)
> 12) (properly detects bad port number)
> 13) (properly detects bad IP)
> 14) (properly detects bum mask)
> 15) (properly detects negative mask)
> 16) (appears to detect escaped quotes)
> 17) (accepts absurdly-large offset; arguably a bug)
> 18) (accepts absurdly-large depth; arguable a bug)
> 19) (accepts capital case)
> 20) (rejects valid but long line with bogus error)
> This was actually a pretty good performance.
> By the way, compiling snort with "-DDEBUG" has not worked for as
> long as I've been hacking snort, which admittedly has not been
> very long. Also, you can pretty much forget about using the
> following flags to gcc:
> All of this leads me to a question: would anyone be averse to some sort
> of testing regimen and a release checklist being contributed for 2.0?
> The regimen would include test rule files to make sure the parser
> is not broken as well as, maybe, unit tests of internal components.
> The checklist would include:
> - running the tests
> - compiling and testing with debugging enabled
> - compiling with strict compiler checking
> Would anyone be interested in helping with such a project?
> Todd Lewis tlewis at ...120...
> God grant me the courage not to give up what I think is right, even
> though I think it is hopeless. - Admiral Chester W. Nimitz
> Name: snort-test.tar.gz
> snort-test.tar.gz Type: Unix Tape Archive (application/x-tar)
> Encoding: BASE64
roesch at ...48...
More information about the Snort-devel