[Snort-users] SnortSnarf performance

James Hoagland hoagland at ...47...
Wed Mar 28 01:48:43 EST 2001


Thanks Tony, for your patch.  If people are pretty anxious to speed 
their SnortSnarf up, then they might want to try Tony's patch.

At 2:25 PM -0500 3/27/01, Tony Lill wrote:
>There are three big problems that make that version of snortsnarf
>slow:
>1) DNS lookups - a lot of alerts are generated by sites without
>addr to name mappings. Each of these failures causes a 90s stall,
>although I presume you have these turned off.

Yeah, SnortSnarf should do some DNS caching at some point.  I might 
be wrong but I don't think the majority of SnortSnarf users use this, 
so this is not the highest priority for me.  This is something that 
should be done somewhat carefully due to wanting expire entries in 
the cache after an appropriate duration.  What I'd really like to see 
is an external module to do this.  Something that will provide a 
"lookup" function and where the main part of SnortSnarf need not 
worry about it.  Actually I wouldn't be surprised if there is 
something already to do this on CPAN or somewhere.

>2) Time comparisons - the timestamps on alerts are compared field by
>field instead of converted to a time() value. This cause a
>surprisingly large slowdown in sorting lists (and is just broken when
>the year rolls over).

It was surprising to me how much of a slowdown that was when you 
first told me about it.  Fortunately, with the modularized 
SnortSnarf, things are done differently (alerts are discrete items 
and its Unix time is stored), so it does something comparable to what 
you have in your patch.

I'm hoping to get modularized SnortSnarf out soon.  Wanting to 
develop from the modularized code is actually one of the things 
holding up the #1 requested enhancement (having SnortSnarf somehow 
make links to arachNIDS for rules without a IDSxxx in their msg 
field, such as is the case in the new snort.org ruleset).  I'm hoping 
to be able to work on this next week.

I was actually working on the modularized version on my bumpy flight 
into New Orleans tonight (going there for a meeting with our research 
sponsors), getting text4sel.pl to grab alerts from arbitrary input 
modules.  Just a little bit more code conversion and adding requested 
feature enhancements and updating the docs and we should be good to 
go.  For those of you what don't know, modularizing SnortSnarf was a 
major rewrite of the code but will be worth it I'm sure.

>3) Memory - if the footprint of snortsnarf forces you to swap, you're
>doomed.

Well, doomed only if you have time restrictions.  We've had reports 
of SnortSnarf successfully completing on a 200Mb alert file from 
DefCon8's Capture the Flag by using about a gig of swap space.  (It 
took a while to complete.)

Fortunately, with modularized SnortSnarf, there will be a few 
approaches that can be taken to reduce SnortSnarf's memory footprint. 
This is something that a motivated third party could undertake in a 
way that is easy to plug in.

Best regards,

   Jim



-- 
|*   Jim Hoagland, Associate Researcher, Silicon Defense    *|
|*               hoagland at ...47...                *|
|*              http://www.silicondefense.com/              *|
|*      Silicon Defense - Technical Support for Snort       *|
|*  Voice: (530) 756-7317              Fax: (530) 756-7297  *|




More information about the Snort-users mailing list