[Snort-devel] Threaded snort

tlewis at ...255... tlewis at ...255...
Fri Jun 15 00:15:35 EDT 2001


Take a look at glib v2's threading library.  With that, your 20
threaded cases + 1 non-threaded case, or 21 cases, collapse to
2 cases: glib threads and no threads.  If you don't like glib, then
you can use the apache portable runtime library; it's the same thing.

Then, when you need to lock something, use a macro that, when snort
is compiled with threading support, expands to:

if(global_config->threading_mode){
	... /* abstracted thread stuff */
}

When compiled without threading support, the thread macros expand to
nothing.

If this is too complicated, then you can still get decent threading
support by this expedient:

	- break snort into well-abstracted subsystems.
	- have each implementation of a subsystem export a capability set.
	- have one of these capabilities represent reentrancy.  
	- when the core of snort v2, which supports threading, gives
	control to a non-reentrant subsystems, it just grabs a mutex.
	- a reentrant subsystem implementation is written by a programmer
	willing to include thread operation macros in his code.  This
	means a little more work, but potentially significant speedups
	for the threaded case.

E.g.,

	/* thread loop */
	while(1){

		LOCK(pae);
		p=pae->get_packet()){
		UNLOCK(pae);

		LOCK(matcher);
		s=matcher->get_matches(p);
		UNLOCK(matcher);

		LOCK(processor);
		processor->process_matches(s, p);
		UNLOCK(processor);
	}

This would expand to, in part:

		if(!(pae->cap&REENTRANT)){
			lock(pae->lock);
		}
		p=pae->get_packet()){
		if(!(pae->cap&REENTRANT)){
			unlock(pae->lock);
		}

So, with three major subsystems, you could utilize three processors,
saturating at least one of them, without using threading code anywhere
outside of these 6 macro instances.  Of course, this might not be
the ideal threading mode, but it's guaranteed to be faster than
single-threaded mode.

There's lots of stuff that you can do.  If snort is ever threaded, then
it will continue fully to support single-threaded platforms, and thread
support will not be onerous to programmers who want to ignore threading.

	We can rebuild it. We have the technology.  We have the capability
	to make the world's first Bionic IDS.  Snort will be that IDS.
	Better than it was before.  Better... stronger... faster.

--
Todd Lewis
tlewis at ...255...

On Fri, 15 Jun 2001 agetchel at ...358... wrote:

> Hey Todd,
> 	Pardon my ignorance, like I've said before I'm no programmer, but
> how could this be accomplished with one source tree?  It was my
> understanding that various OS's have drastically different threading models,
> and thus the code would have to be drastically different depending on the OS
> it's being compiled on.  Is this not true?  I know that writing
> multithreaded apps isn't hard, I've done it before, but I can't imagine
> trying and take all of the threading idiosyncrasies of the over twenty OS's
> that Snort runs on and put them all into one pretty package.
> 
> Thanks,
> Abe
> 
> Abe L. Getchell - Security Engineer
> Division of System Support Services
> Kentucky Department of Education
> Voice   502-564-2020x225
> E-mail  agetchel at ...358...
> Web     http://www.kde.state.ky.us/
> 
> 
> 
> > -----Original Message-----
> > From: tlewis at ...255... [mailto:tlewis at ...255...]
> > Sent: Thursday, June 14, 2001 2:41 PM
> > To: agetchel at ...358...
> > Cc: sjsnort at ...398...; snort-devel at lists.sourceforge.net
> > Subject: RE: [Snort-devel] Threaded snort
> > 
> > 
> > On Thu, 14 Jun 2001 agetchel at ...358... wrote:
> > 
> > > Hi Siddhartha,
> > > 	No one ever said there wasn't an advantage to having 
> > Snort threaded,
> > > it was just stated that it probably wasn't going to happen. =)
> > 
> > On the contrary, I think that it will happen.
> > 
> > > I think
> > > everyone agrees that threading Snort to allow it to take 
> > advantage of
> > > multiple processors would help performance, but the real 
> > question is can
> > > this be done without hurting the portability of the code?  
> > The answer was a
> > > resounding 'no'.
> > 
> > It might have been resounding, but it was not unanimous.
> > 
> > > I think that the Snort developers are on the right track
> > > when it comes to releasing Snort 2.0 to have the _capability_ to be
> > > threaded, but I sure wouldn't want to be the one to manage all those
> > > different source trees... =)
> > 
> > It will be one source tree and a simple one at that.  
> > Threading is not hard.
> > 
> > --
> > Todd Lewis
> > tlewis at ...255...
> > 
> 





More information about the Snort-devel mailing list