[Snort-devel] unofficial testing

Todd Lewis tlewis at ...255...
Mon Feb 12 16:12:20 EST 2001


I was just reading Keith Bostic's keynote address to BSDCon 2000.  He
said:

	As I may have already mentioned, I think it's a demonstrable
	fact that, if a line of software hasn't been tested, it almost
	certainly doesn't work.  Software reliability depends on testing,
	and therefore, on modularity, because the software code base
	in most projects is too big to depend on anything other than
	module testing.

So, in the spirit of our rules discussion, I came up with some examples
of bogus rules; they are attached.  Here are the results for 1.6.3.

First for ten ways to truncate a rule:

1) Segmentation fault
2) Segmentation fault
3) Segmentation fault
4) Segmentation fault
5) (Mistakenly loads faulty rule)
6) Segmentation fault
7) Segmentation fault
8) Segmentation fault
9) Segmentation fault
10) Segmentation fault

That's right, out of 10 ways to truncate a rule, none of the ten are
properly handled by snort in its present form.

Now for some intra-rule mischief:

11) (properly tries to resolve name)
12) (properly detects bad port number)
13) (properly detects bad IP)
14) (properly detects bum mask)
15) (properly detects negative mask)
16) (appears to detect escaped quotes)
17) (accepts absurdly-large offset; arguably a bug)
18) (accepts absurdly-large depth; arguable a bug)
19) (accepts capital case)
20) (rejects valid but long line with bogus error)

This was actually a pretty good performance.

By the way, compiling snort with "-DDEBUG" has not worked for as
long as I've been hacking snort, which admittedly has not been
very long.  Also, you can pretty much forget about using the
following flags to gcc:

	-Wstrict-prototypes
	-ansi
	-pedantic-errors

All of this leads me to a question: would anyone be averse to some sort
of testing regimen and a release checklist being contributed for 2.0?
The regimen would include test rule files to make sure the parser
is not broken as well as, maybe, unit tests of internal components.
The checklist would include:

	- running the tests
	- compiling and testing with debugging enabled
	- compiling with strict compiler checking

Would anyone be interested in helping with such a project?

--
Todd Lewis                                       tlewis at ...120...

  God grant me the courage not to give up what I think is right, even
  though I think it is hopeless.          - Admiral Chester W. Nimitz
-------------- next part --------------
A non-text attachment was scrubbed...
Name: snort-test.tar.gz
Type: application/octet-stream
Size: 1243 bytes
Desc: 
URL: <https://lists.snort.org/pipermail/snort-devel/attachments/20010212/0ae96228/attachment.obj>


More information about the Snort-devel mailing list