[Snort-devel] draft rfc - reengineering the snort packet matching system

Todd Lewis tlewis at ...255...
Sat Apr 7 04:04:22 EDT 2001

On Fri, 6 Apr 2001, Simon Attwell wrote:

> Let say for example that the core is keeping track of these fragments,
> or better yet a pe that is designed explicitly for handling fragments :)

If it's IP fragmentation, then I probably the IP pe is a good place
to handle it, although despite my crystal ball the whole area is still
murky to me.

> On receipt of a fragment, the pe can determine the rules that should be
> applied for that fragment dependant upon the information contained in the
> fragment. It should then put that fragment in a list of known fragments
> with the list of rules that have already been applied to that fragment,
> when another fragment is received by the pe it should first check to see
> if it is the first fragment or a continuation of a previous fragment by
> examing the overlap, the pe can determine if it should reapply rules
> that were previously applied or only apply rules that would match the
> known assembled fragment minus the rules previously applied.

A problem is that the pe doesn't examine entire rules or sets of rules.
The pe is given a set of criteria from the matcher and returns the subset
of those criteria that are matched by the packet.  All decision-making
is done in the matcher.

You know, now that I think about it, the potential for duplicate
matches, where both (f1) and (f1&f2) ding the same rule, is no different
the the same rule being triggered twice by duplicate IP datagrams!
Nolo contendere, ipso facto, hocus pocus, pater familias, since the
problem is no worse than a problem that is already tolerated, then the
problem must be tolerable!

Ok, I am officially not worried about fragments anymore; just keep
submitting the aggregate as fragments keep coming in and, if you're
a firewall, when you match a bad rule, reject that fragment.  If the
rule is well-formed, then the attack will be stopped and world peace
preserved.  If someone else wants to worry about it, then feel free to
send in your ideas.

Damn, am I a good programmer or what?  That problem was solved with 0
lines of code!  8^)  Seriously, a message to all of the young hackers
out there, when you do it right, the best and most important programming
is done before the first line of code is ever written.  By the time
we're done discussing this proposal, if we decide to go with it, the
implementation should be very easy and the suprises minimal, and this
mini-discussion is a good example of that.

> Of course, with strict memory management and limits set on how long one 
> maintains a fragment before disposal.

Yeah.  With disks being free, you could just mmap() a GB of disk and throw
them there.  That way, the kernel automatically takes care of the problem
for you when there's memory pressure.  Plus, you schedule a sweep every
(1.5 x normal IP fragment discard interval) to toss the oldies.

> >         8. portscan detection
> >
> >Add some random TCP/UDP port matching rules and then do detection in
> >post-processing.
> this i think will be a contentious issue, since as i understand currently, 
> the portscan
> preprocesser simply watches the output from the PA, prior to the core, does 
> simple decode
> for ip header information and watches for destination ranges with similar 
> source.
> Passing all this information through the core, the matcher and a pe would 
> require somewhat more processing time would it not ?

Yes.  Look, if people want to put portscan detection in the TCP pe, it's
not going to bother me one whit.  My rulesets won't use it, and so the
only thing it costs me is a tiny code footprint, which is no different
than the hundreds of other pe functionalities that I won't use.  Hell,
if you have the pe's demand-load their implementations of different
operations, which ain't a bad idea now that I think about it, especially
for heavyweight operations, then it won't even cost me the footprint.

Given the literature from the NRL (I think) documenting stuff like
longstanding, coordinated, distributed portscan against military targets,
I think you're a fool to try to do portscan detection in a lightweight
way, but hey, it's a free country.

In fact, considering how little I actually want to do this implementation
work, I hope that it's a contentious issue.  I hereby announce my
intention, if I have to write the TCP or UDP protocol engine, never to
support portscan detection in them and to publish them under a license
ensuring that it never will be supported.  If you don't like that,
then you'll have to do the pe's for these protocols.  8^)

> >                 + build adaptor for existing rules
> >
> >This should be a straightforward piece of code once the mapping of matching
> >options to new pe's is performed.
> The rule engine itself should be developed separately, it should be able to 
> handle the many and
> varietal types of input required by the userbase. It should be able to 
> inform pe's of changes in the
> rules applicable to that pe, or be able to notify the pe that a new rule 
> has been added and let the pe
> check to see if the new rule is applicable :)

Actually, I was thinking that the matcher, the pe's, and the
post-processors would all have APIs for submitting rule fragments into
them (and zapping them later) and the rule handlers would juggle all of
the balls and make sure that none end up on the floor.

> Basically this approach provides the ability to do on-the-fly updates of 
> the ruleset, and reduce processor
> load by having to reparse the entire ruleset every time a change is made.

Unless rereading the entire ruleset is the implementation strategy you
use for your rule handler, a decision which is in no way impinged by
this design.  Still, it should be possible to do better than that if
you're so inclined.

> Great document btw, we have a long road ahead but hopefully we can only 
> make things better :)

Hey, thanks a lot, and thanks for your feedback.  One of my big concerns
about my proposal is that it is such a big change that it will represent a
whole lot of work and time.  I'm still eagerly awaiting Marty's sentiments
on the idea.

> The idea of course of a modular approach provides the opportunity for 
> multiple paengines (multiple interfaces)
> to provide input to the core in a threaded fashion. By using a pcap 
> instance per paengine/interface one eliminates
> the need for pcap to be able to handle the multiple interfaces itself.
> Ok... i need another beer :)

Me too.  8^)

Todd Lewis
tlewis at ...255...

More information about the Snort-devel mailing list