[Snort-devel] Latest cvs updates

Marc Norton marc.norton at ...402...
Thu Oct 10 10:42:08 EDT 2002


Use the wu-manber (mwm) pattern matcher, it uses less memory.  There
will be a significant increase in memory usage for set based pattern
matching.  Aho-Corasick is the classic algorithm, and uses 2x-3x the
memory of wu-manber.  For very large sets, it may outperform wu-manber
when pattern sizes are small.  As the minimum pattern size gets bigger,
the wu-manber speeds up due to it's Boyer-Moore bad character shift.
However, Aho Corasick is pretty insensitive to the minimum pattern size,
by comparison.  There are many issues here, use the wu-manber in
general.  Overall performance is relatively insensitive to which one you
use. Be sure to use the httpflow preprocessor for maximum performance.

-----Original Message-----
From: snort-devel-admin at lists.sourceforge.net
[mailto:snort-devel-admin at lists.sourceforge.net] On Behalf Of
Kreimendahl, Chad J
Sent: Thursday, October 10, 2002 12:38 PM
To: Daniel Roelker; snort-devel at lists.sourceforge.net
Subject: RE: [Snort-devel] Latest cvs updates


That helps quite a great deal.  I'm going to have to read up more on the
two different multi-pattern matching algorithms... See which one would
more likely be of value to us... As I'm not that familiar with any... 

A quick test on one of our production systems show it uses slightly less
cpu (15%) with the mwm method, and the same (or sometimes less) with the
ac method, as compared to snort-1.9.  Should there be in increase in
used CPU and memory?  Also, should the memory usage when using ac be so
much greater (2x) than mwm?  When running with the ac method, I see
upwards of 200M used, whereas the same config file changed to mwm uses
only 68M.

Documentation on the internet has gone greatly over my head in
understanding the pattern matching algorithms, so is there an english
way to explain what the benefits/downfalls of each of these?  Maybe some
docs on the net somewhere that someone could RTFM me to.

-woot
--cjk

-----Original Message-----
From: Daniel Roelker [mailto:droelker at ...402...] 
Sent: Thursday, October 10, 2002 12:20 PM
To: Kreimendahl, Chad J; snort-devel at lists.sourceforge.net
Subject: Re: [Snort-devel] Latest cvs updates


There are three major changes to the Snort detection engine, two of
those
have to do with performance, the other one deals with event queuing.

The first change is a rule optimizer.  This basically reads all of the
rules
from the RTN/OTN lists and group those into unique rule sets based on
protocol parameters.  This insures that each packet that is analyzed can
select one group of rules and that's it.  Since each packet deals with
one
rule group, set inspection methodologies can be utilized.  Obviously
this
increases performance drastically.

The second change is a multi-rule detection engine.  This is where the
rules
in a group are actually processed using set analysis.  As you picked up
on
in your next post, there are two types of multi-pattern matchers you can
use:  one is a wu-manber algorithm and the other is a classic
aho-corasick
algorithm.  You should note though, that what makes this whole detection
philosophy different is the way in which the rule optimizer creates rule
subsets based on unique parameters.  This allows the multi-rule
detection
engine to work much better than previous attempts at speeding up Snort.

The third change is in event queuing.  Snort now detects all occurrences
of
alerts in a packet/stream and queues them for a final selection process.
Currently, this is based on the longest content match, which should give
you
a more specific alert.  For example, instead of picking up an HTTP
directory
traversal, you would see a cmd.exe alert.

This code is contained in the following files:

fpcreate.[ch]
fpdetect.[ch]
mwm.[ch]
acsmx.[ch]
mpse.[ch]
pcrm.[ch]

To answer your question about how to enable mwm or ac, here's how:

config detection: search-method [ac or mwm]

some of the other options are:

debug
* this turns on a lot of information for you to see when building rule
sets
and post processing.

no_stream_inserts
* this is a little performance boost by telling the detection engine to
not
inspect stream inserts, since they will (hopefully :) ) be flushed
through
the detection engine.  You might want to run some tests turning this on
so
you can see the difference.

max_queue_events
* this allows you to tell the detection engine how many events to queue
for
a packet before selecting one.

Hope this helps a little.

Dan

On 10/10/02 6:52 AM, "Kreimendahl, Chad J" <Chad.Kreimendahl at ...1167...>
wrote:

> 
> I noticed the massive amount of changes, coupled with a change in
build
> number to '1'.   Anyone care to enlighten those of us who chose to
test
> this baby out as to what all the wonderous new features are?... My
guess
> from the cvs log... Performance increase, change to output stuffs...
> 
> -----Original Message-----
> From: Chris Green [mailto:chrisgreen at ...64...]
> Sent: Thursday, October 10, 2002 12:06 AM
> To: snort-cvsinfo at lists.sourceforge.net
> Subject: [snort-cvs] CVS: snort - chrisgreen
> 
> 
> CVSROOT:    /cvsroot/snort
> Module name:    snort
> Changes by:    chrisgreen at ...1161...    2002/10/09 22:06:00
> 
> Modified files:
> .              : ChangeLog config.h.in configure configure.in
> contrib        : Makefile.in
> src            : Makefile.am Makefile.in checksum.h decode.c
>                 decode.h detect.c detect.h log.c parser.c
>                 parser.h plugbase.c plugbase.h signature.c
>                 snort.c snort.h util.c
> src/detection-plugins: sp_clientserver.c sp_icmp_code_check.c
>                       sp_icmp_id_check.c sp_icmp_seq_check.c
>                       sp_icmp_type_check.c sp_icmp_type_check.h
> 
>                       sp_ip_proto.c sp_ip_proto.h
>                       sp_ip_same_check.c sp_ipoption_check.c
>                       sp_pattern_match.c sp_pattern_match.h
>                       sp_react.c sp_react.h sp_respond.c
>                       sp_rpc_check.c sp_session.c
>                       sp_tcp_ack_check.c sp_tcp_flag_check.c
>                       sp_tcp_seq_check.c sp_tcp_win_check.c
> src/output-plugins: spo_alert_smb.c spo_alert_syslog.c
>                    spo_alert_unixsock.c spo_csv.c
>                    spo_log_ascii.c spo_log_tcpdump.c
>                    spo_unified.c spo_xml.c
> src/preprocessors: Makefile.am Makefile.in spp_arpspoof.c
>                   spp_asn1.c spp_bo.c spp_conversation.c
>                   spp_fnord.c spp_frag2.c spp_frag2.h
>                   spp_http_decode.c spp_http_decode.h
>                   spp_perfmonitor.c spp_perfmonitor.h
>                   spp_portscan.c spp_portscan2.c spp_stream4.c
>                   spp_stream4.h spp_telnet_negotiation.c
>                   spp_telnet_negotiation.h
> src/win32      : Makefile.in
> src/win32/WIN32-Prj: snort.dsp
> Added files:
> src            : acsmx.c acsmx.h bitop.h fpcreate.c fpcreate.h
>                 fpdetect.c fpdetect.h mpse.c mpse.h mwm.c mwm.h
> 
>                 pcrm.c pcrm.h smalloc.h
> src/preprocessors: http-resp.c http-resp.h perf-base.c
>                   perf-base.h perf-event.c perf-event.h
>                   perf-flow.c perf-flow.h perf.c perf.h
>                   sfprocpidstats.c sfprocpidstats.h
>                   spp_httpflow.c spp_httpflow.h
> Removed files:
> src            : perf-base.c perf-base.h perf-event.c
>                 perf-event.h perf-flow.c perf-flow.h perf.c
>                 perf.h
> 
> Log message:
> * merged in Sourcefire perfmods, httpflow, newer perfmonitor
> * merged with snort-1.9.0 head
> 
> will work on win32 proj/docs soon
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Snort-cvsinfo mailing list
> Snort-cvsinfo at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/snort-cvsinfo
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Snort-devel mailing list
> Snort-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/snort-devel
> 

-- 
Daniel Roelker
Software Engineer
droelker at ...402...

www.sourcefire.com
www.snort.org





-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
Snort-devel mailing list
Snort-devel at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/snort-devel





More information about the Snort-devel mailing list