[Snort-devel] Re: [Snort-users] Stream4 and other stuff

Martin Roesch roesch at ...402...
Fri Jun 29 09:28:39 EDT 2001


I've had another report of segfaults on RH 7, is this build 30 that's
crashing?  Can you give me a traceback (read the BUGS file) so I can see
where the problem is?

     -Marty

"Mayers, Philip J" wrote:
> 
> >From the CVS of an hour or two ago, I get a Segmentation Fault on startup -
> the system is an intel RedHat 7.0 box. Commenting out the stream4
> preprocessor lines allows snort to run OK.
> 
> (Note - we do have a rather high-traffic site, currently doing 45megabits,
> although we skip all port 80 traffic.)
> 
> Regards,
> Phil
> 
> +----------------------------------+
> | Phil Mayers, Network Support     |
> | Centre for Computing Services    |
> | Imperial College                 |
> +----------------------------------+
> 
> -----Original Message-----
> From: Martin Roesch [mailto:roesch at ...402...]
> Sent: 28 June 2001 18:01
> To: snort-users; snort-dev
> Subject: [Snort-users] Stream4 and other stuff
> 
> Hi everyone,
>      Sorry I've been quiet lately, I'm busier than a one legged man at
> an ass-kicking contest lately and I haven't been very good about
> answering my email, but I have a good reason.
>      Those of you who have been tracking CVS lately may have noticed a
> new module go into the mix within the last week: spp_stream4.  I just
> checked what I believe to be stable version of the new plugin into CVS,
> so it's available and turned on by default in version 1.8-beta8 (build
> 29).  Stream4 is an entirely new preprocessor that preforms two
> functions:
> 
> 1) Stateful inspection of TCP sessions
> 2) TCP stream reassembly
> 
> I implemented stream4 out of the desire to have more robust stream
> reassembly capabilities and the desire to defeat the latest "stateless
> attacks" that have been coming out against Snort (c.f. stick and snot).
> Stream4 is written with the intent to let Snort be able to handle
> performing stream reassembly for "enterprise class" users, people who
> need to track and reassemble more than 256 streams simultaneously.  I've
> optimized the code fairly extensively to be robust, stable, and fast.
> The testing and calculations I've performed lead me to be fairly
> confident that stream4 can provide full stream reassembly for several
> thousand simultaneous connections and stateful inspection for upwards of
> 64,000 simultaneous sessions.
> 
> Stream4 is a large and complex piece of code (almost 2000 lines) and
> there are a lot of options associated with its runtime configuration, so
> I'll go over them here.
> 
> preprocessor stream4: [noinspect], [keepstats], [timeout <seconds>],
> [memcap <bytes>], [noalerts]
> 
> noinspect - disable stateful inspection
> 
> keepstats - record session summary information in <logdir>/session.log
> 
> timeout <seconds> - amount of time to keep an inactive stream in the
> state table, sessions that are flushed will automatically be picked up
> again if more activity is seen, default is 30 seconds
> 
> memcap <bytes> - number of bytes to set the memory cap at, if this limit
> is exceeded stream4 will aggressively prune inactive sessions, default
> is 8MB
> 
> noalerts - turns off alerts for stream events of note, such as evasive
> RST packets, data on the SYN packet, and out of window sequence numbers
> 
> preprocessor stream4_reassemble: [clientonly], [serveronly], [noalerts],
> [ports <portlist>]
> 
> clientonly - provide reassembly for the client side of a connection only
> 
> serveronly - provide reassembly for the server side of a connection only
> 
> noalerts - don't alert on events that may be insertion or evasion
> attacks
> 
> ports <portlist> - a whitespace separated lit of ports to perform
> reassembly for, "all" provides reassembly for all ports, "default"
> provides reassembly for ports 21 23 25 53 80 110 111 143 and 513
> 
> Just setting the stream4 and stream4_reassemble directives without
> arguments in the snort.conf file will set them up in their default
> configuration
> 
> [*] stream4 defaults:
> Session timeout: 30 seconds
> Session memory cap: 8388608 bytes
> Stateful Inspection: ACTIVE
> Stream Reassembly: INACTIVE
> Stream Stats: INACTIVE
> State Alerts: ACTIVE
> 
> [*] stream4_reassemble defaults:
> Reassemble client: ACTIVE
> Reassemble server: INACTIVE
> Reassemble ports: 21 23 25 53 80 143 110 111 513
> Reassembly alerts: ACTIVE
> 
> There is a new command line switch that is used in concert with the
> stream4 code, "-z".  The -z switch can take one of two arguments: "est"
> and "all".  The "all" argument is the default if you don't specify
> anything and tells Snort to alert normally.  If the -z switch is
> specified with the "est" argument, Snort will only alert (for TCP
> traffic) on streams that have been established via a three way handshake
> or streams where cooperative bidirectional activity has been observed
> (i.e. where some traffic went one way and something other than a RST or
> FIN was seen going back to the originator).  With "-z est" turned on,
> Snort completely ignores TCP-based stick/snot "attacks".
> 
> Besides that, there is another new command line switch, "-k".  The -k
> switch allows you to modify the checksum tests that Snort performs in
> the decoder stage.  It can take the following arguments:
> 
> "noip" - turn off IP checksum verification
> 
> "notcp" - turn off TCP checksum verification
> 
> "noudp" - turn off UDP checksums
> 
> "noicmp" - turn off ICMP checksums
> 
> "none" - turn off all checksums
> 
> Ths is done as a mechanism to let people have control over a subsystem
> in Snort that can eat a lot of CPU cycles if you're not careful.  For
> example, in many networks today, packets with bad IP checksums never
> make it past the router or switch, and so IP checksum verification is a
> waste of time for Snort on these networks.  Allowing people to turn that
> subsystem off (or tune it) lets them get better performance without
> having to make code tweaks.
> 
> I've also added some new "stuff" to the output stage of Snort recently.
> If you've noticed lately, there's some new data in the alerts that looks
> something like "[1:523:1]" just before the alert message.  This is the
> new Snort unique rule ID information that we've recently integrated into
> the rule set that will allow much better correlation of Snort rules to
> other vulnerability databases (CVE, Bugtraq, etc) and provide much
> easier machine processing of Snort output as well.
> 
> The fields in the block are [generator:sid:revision].  Generator is the
> piece of Snort that generated the alert.  The generators.h file contains
> the list of generators and the SIDs that they can generate, except for
> the primary detection engine (generator 1).  The SID is the "Snort ID"
> of the rule, a unique rule ID assigned to the rule by myself.  The
> partitioning of the SID space is as follows:
> 
> 0-100:       Reserved for Marty
> 101-999,999: Reserved for "official" Snort rules
> 1,000,000+:  User land SID space
> 
> Each there is a full set of SIDs available per generator, and the SID
> space is a 32-bit unsigned integer inside Snort, so the values can go up
> to 4 billion or so.
> 
> The last number in the block is the revision number of the rule.  As
> rules change and their messages change (today's "SYN FIN scan" can
> become tomorrow's "SCAN - SYN FIN") we want to be able to track the
> revision history of the rules so that people who are doing machine
> processing of Snort rules (like ARIS, incidents.org, etc) can
> effectively perform their own back-end tasks.  The revision number
> updating depends on the ruleset maintainer, but in general anytime you
> tweak a rule, you should update the revision number.  If you modify a
> rule that ships with Snort, please set the revision number to "0" so
> that any machine processors will know you're running modified standard
> rules.
> 
> Snort will also print out the transport protocol designation in
> fast/syslog alerts now as well.
> 
> In output land, we have a new output plugin called spo_unified.  This
> plugin is used for both alerting and logging, and writes alerts and logs
> to two separate binary formatted files.  The alert file contains the
> following information of interest:
> 
> Event data (generator, sid, rev, classification, priority,
> event_reference)
> timestamp
> source IP
> dest IP
> source port
> dest port
> protocol
> tcp flags (if any)
> 
> The log file contains the event data, flags that indicate the nature of
> the stored packet (reassembled frag, etc), and the raw binary packet
> (like in the log_tcpdump plugin).
> 
> Using this system we will be able to write Snort output to high speed
> binary spool files and then use a second process to reformat the data
> for consumption by the user.  I'm currently writing a program to perform
> just that task called 'barnyard'.  Barnyard can read the binary file
> formats of the unified alert and log spool files and present that data
> to output plugins, which can then format the data for output to whatever
> reporting and storage system is desired (database, XML, pcap, syslog,
> etc).  Once this program is ready I will go into more detail on using
> the integrated system to enable higer performance implementations of
> Snort.
> 
> That's it for right now.  If you're feeling sufficiently motivated,
> please check out Snort version 1.8-beta8 (build 29) from CVS and test it
> out for performance, memory leaks, and stability.  If all goes well,
> this is the codebase that 1.8 will be based on.
> 
>      -Marty
> 
> --
> Martin Roesch
> roesch at ...402...
> http://www.sourcefire.com - http://www.snort.org
> 
> _______________________________________________
> Snort-users mailing list
> Snort-users at lists.sourceforge.net
> Go to this URL to change user options or unsubscribe:
> http://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://www.geocrawler.com/redir-sf.php3?list=snort-users
> 
> _______________________________________________
> Snort-users mailing list
> Snort-users at lists.sourceforge.net
> Go to this URL to change user options or unsubscribe:
> http://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://www.geocrawler.com/redir-sf.php3?list=snort-users

--
Martin Roesch
roesch at ...402...
http://www.sourcefire.com - http://www.snort.org




More information about the Snort-devel mailing list