[Snort-devel] snort-2.0 and paengines

Todd Lewis tlewis at ...120...
Thu Jan 18 22:58:11 EST 2001


On Thu, 18 Jan 2001, Martin Roesch wrote:

> > 2) Will the 1.7 code be the starting point?
> 
> Yes, no reason to throw out the 34000+ lines of code we have so far. :) 

8^)

> > 6) Would there be any objection to my picking up the threading code and
> > getting it working properly?  I want to get some serious performance
> > out of this release via the use of threading on MP machines.
> 
> Not from me.  Fyodor?  (The original threading code was Fyodor's, he's
> the guy to ask about that.  As we discussed, we should be very careful
> to avoid negatively impacting the performance of the uniprocessor code
> with threads.

Again, it will all be #def'd away if you compile the code without
-DUSE_THREADS (or whatever).

> > 7) I notice that the present threading code assigns one thread per
> > interface.  I am curious as to why you could not just run one instance
> > of snort per interface instead.  Wouldn't the performance be the same?
> > If I do take over the threading stuff, my disposition would be to move
> > away from this model to a worker-pool model, where worker threads queue
> > up to receive the next available packet from the paengine, serialized
> > by a mutex; this would spread the workload even via packet acquisition
> > mechanisms with a single entry point, like netfilter.  Would there be
> > any objections to this approach?
> 
> So you're proposing having one thread per packet and handling packet
> asynchronously?  That could present certain complications with things
> that expect to see them in a synchronous fashion (defrag, stream, etc). 
> What about threading the output stage?  Having the output section of
> Snort be "non-blocking" (not really, but you know what I mean) would be
> really nice, especially when we're doing slow DB inserts.

No, I'm not suggesting that.  I would have a small number of threads,
equal or slightly greater than the number of CPUs, executing the packet
handling loop.  (Viz. my mail from earlier today for that loop.)

When a packet comes in, one of the threads would receive it, process
it, and then get at the back of the line and wait its turn to go again.
This should allow -> 100% CPU utilization on MP machines, with the only
limit being any serialization that the packet acquisition mechanism
imposes.  (E.g., pcap in its present form can not be parallelized very
aggressively.)

When I am ready, I'll have actual code.  That code change will be very
small, but it's important that we start thinking about reentrancy issues
and make the snort code, if not threaded, then at least thread-friendly.

--
Todd Lewis                                       tlewis at ...120...

  God grant me the courage not to give up what I think is right, even
  though I think it is hopeless.          - Admiral Chester W. Nimitz





More information about the Snort-devel mailing list