[Snort-users] Poor performance with Snort 2.9.4.6 under OpenBSD 5.3

Victor Roemer vroemer at ...1935...
Wed Jun 5 13:08:53 EDT 2013


Martinez, as Joel already mentioned, we'll want to see your Snort
configuration. Shutdown stats would also be useful, but perfmon data would
be better; if those can be provided.

You mentioned that OpenBSD configured the network sysctl parameters "on the
fly"; could you direct us to some documentation about this?

You also mentioned that Snort was listening on em3, however the startup
information in your email indicates that Snort is listening on em4, could
you elaborate on this setup?


Regarding Suricata, I personally do not have any experience in deploying or
configuring it. That said, are you using, relatively, the same
configurations? (e.g., any rules enabled, acquiring packets via libpcap,
etc..)

Also, why are "tcp.reassembly_gap" and "tcp.invalid_checksum" relevant?


On Wed, Jun 5, 2013 at 11:06 AM, Joel Esler <jesler at ...1935...> wrote:

> Can you post your snort.conf somewhere?
>
>
> On May 31, 2013, at 2:51 AM, C. L. Martinez <carlopmart at ...11827...> wrote:
>
> > On Thu, May 30, 2013 at 12:45 PM, C. L. Martinez <carlopmart at ...11827...>
> wrote:
> >> Hi all,
> >>
> >> According to the following stats:
> >>
> >> May 30 11:46:22 nsm01 snort[30096]:
> >>
> ===============================================================================
> >> May 30 11:46:22 nsm01 snort[30096]: Packet Performance Summary:
> >> May 30 11:46:22 nsm01 snort[30096]:    max packet time       : 10000
> usecs
> >> May 30 11:46:22 nsm01 snort[30096]:    packet events         : 654
> >> May 30 11:46:22 nsm01 snort[30096]:    avg pkt time          : 27.1384
> usecs
> >> May 30 11:46:22 nsm01 snort[30096]: Rule Performance Summary:
> >> May 30 11:46:22 nsm01 snort[30096]:    max rule time         : 4096
> usecs
> >> May 30 11:46:22 nsm01 snort[30096]:    rule events           : 20
> >> May 30 11:46:22 nsm01 snort[30096]:    avg rule time         : 1.046
> usecs
> >> May 30 11:46:22 nsm01 snort[30096]:
> >>
> ===============================================================================
> >> May 30 11:46:22 nsm01 snort[30096]: Packet I/O Totals:
> >> May 30 11:46:22 nsm01 snort[30096]:    Received:     69971576
> >> May 30 11:46:22 nsm01 snort[30096]:    Analyzed:     22427618 ( 32.052%)
> >> May 30 11:46:22 nsm01 snort[30096]:     Dropped:     41532168 ( 37.247%)
> >> May 30 11:46:22 nsm01 snort[30096]:    Filtered:            0 (  0.000%)
> >> May 30 11:46:22 nsm01 snort[30096]: Outstanding:     47543958 ( 67.948%)
> >> May 30 11:46:22 nsm01 snort[30096]:    Injected:            0
> >> May 30 11:46:22 nsm01 snort[30096]:
> >>
> ===============================================================================
> >> May 30 11:46:22 nsm01 snort[30096]: Breakdown by protocol (includes
> >> rebuilt packets):
> >> May 30 11:46:22 nsm01 snort[30096]:         Eth:     22436767 (100.000%)
> >> May 30 11:46:22 nsm01 snort[30096]:        VLAN:            0 (  0.000%)
> >> May 30 11:46:22 nsm01 snort[30096]:         IP4:     22436767 (100.000%)
> >> May 30 11:46:22 nsm01 snort[30096]:        Frag:           12 (  0.000%)
> >> May 30 11:46:22 nsm01 snort[30096]:        ICMP:       110634 (  0.493%)
> >> May 30 11:46:22 nsm01 snort[30096]:         UDP:       752816 (  3.355%)
> >> May 30 11:46:22 nsm01 snort[30096]:         TCP:     19433478 ( 86.614%)
> >>
> >> using snort under OpenBSD 5.3 doesn't returns good performance. Host
> >> is a Intel(R) Xeon(R) CPU E5620 @ 2.40GHz, with 8 GiB RAM and four
> >> e1000 interfaces.
> >>
> >> In this sensor, I only use so_rules:
> >>
> >> # dynamic library rules
> >> # include $SO_RULE_PATH/bad-traffic.rules
> >> # include $SO_RULE_PATH/chat.rules
> >> include $SO_RULE_PATH/dos.rules
> >> include $SO_RULE_PATH/exploit.rules
> >> # include $SO_RULE_PATH/icmp.rules
> >> # include $SO_RULE_PATH/imap.rules
> >> include $SO_RULE_PATH/misc.rules
> >> include $SO_RULE_PATH/multimedia.rules
> >> include $SO_RULE_PATH/netbios.rules
> >> # include $SO_RULE_PATH/nntp.rules
> >> include $SO_RULE_PATH/p2p.rules
> >> include $SO_RULE_PATH/smtp.rules
> >> # include $SO_RULE_PATH/snmp.rules
> >> include $SO_RULE_PATH/specific-threats.rules
> >> include $SO_RULE_PATH/web-activex.rules
> >> include $SO_RULE_PATH/web-client.rules
> >> include $SO_RULE_PATH/web-iis.rules
> >> include $SO_RULE_PATH/web-misc.rules
> >>
> >> and monitored network is a 1GiB network.
> >>
> >> Any ideas why??
> >
> >
> > More info:
> >
> >
> > top:
> > load averages:  0.69,  0.65,  0.53
> > 31 processes: 30 idle, 1 on processor
> > CPU0 states:  2.8% user,  0.0% nice,  0.4% system, 20.4% interrupt,
> 76.4% idle
> > CPU1 states:  2.2% user,  0.0% nice,  0.8% system,  0.0% interrupt,
> 97.0% idle
> > CPU2 states:  3.0% user,  0.0% nice,  3.4% system,  0.0% interrupt,
> 93.6% idle
> > CPU3 states:  6.0% user,  0.0% nice,  5.0% system,  0.0% interrupt,
> 89.0% idle
> > Memory: Real: 587M/2947M act/tot Free: 5012M Cache: 2213M Swap: 0K/6142M
> >
> >  PID USERNAME PRI NICE  SIZE   RES STATE     WAIT      TIME    CPU
> COMMAND
> > 14655 root       4    0  393M  183M sleep/1   bpf       8:44 14.26% snort
> > 25669 root       4    0 1132K 1740K sleep/2   bpf       0:06  3.52%
> daemonlogger
> >
> > systat ifstat (snort process is listening in em3)
> >
> >    3 users    Load 0.89 0.71 0.56                     Fri May 31
> 06:23:13 2013
> >
> > IFACE            STATE  DESC
> >                     IPKTS   IBYTES    IERRS    OPKTS   OBYTES
> > OERRS    COLLS
> > em0              up
> >                         2      132        0        0      261
> > 0        0
> > em1              up
> >                         0      126        0        0      131
> > 0        0
> > em2              up
> >                     10348  3425952        0        0        0
> > 0        0
> > em3              up
> >                     10346  3425044        0        0        0
> > 0        0
> >
> >
> > systat mbufs
> >
> >
> > IFACE             LIVELOCKS  SIZE ALIVE   LWM   HWM   CWM
> > System                    0   256   185          56
> >                               2k   171         435
> > lo0
> > em0                            2k     6     4   256     6
> > em1                            2k     6     4   256     4
> > em2                            2k    66     4   256    66
> > em3                            2k    65     4   256    65
> >
> >
> > Stats with ALL so_rules disabled (5 min, more or less):
> >
> > Rule application order:
> > activation->dynamic->pass->drop->sdrop->reject->alert->log
> > Verifying Preprocessor Configurations!
> > ICMP tracking disabled, no ICMP sessions allocated
> > IP tracking disabled, no IP sessions allocated
> > 0 out of 1024 flowbits in use.
> >
> > Packet Performance Monitor Config:
> >  ticks per usec  : 2417 ticks
> >  max packet time : 10000 usecs
> >  packet action   : fastpath-expensive-packets
> >  packet logging  : log
> >  debug-pkts      : disabled
> >
> > Rule Performance Monitor Config:
> >  ticks per usec  : 2417 ticks
> >  max rule time   : 4096 usecs
> >  rule action     : suspend-expensive-rules
> >  rule threshold  : 5
> >  suspend timeout : 10 secs
> >  rule logging    : log
> > pcap DAQ configured to passive.
> > Acquiring network traffic from "em4".
> > Reload thread starting...
> > Reload thread started, thread 0xc100dbb8f00 (18056)
> > Decoding Ethernet
> >
> >        --== Initialization Complete ==--
> >
> >   ,,_     -*> Snort! <*-
> >  o"  )~   Version 2.9.4.6 GRE (Build 73)
> >   ''''    By Martin Roesch & The Snort Team:
> > http://www.snort.org/snort/snort-team
> >           Copyright (C) 1998-2013 Sourcefire, Inc., et al.
> >           Using libpcap version 1.3.0
> >           Using PCRE version: 8.31 2012-07-06
> >           Using ZLIB version: 1.2.3
> >
> >           Rules Engine: SF_SNORT_DETECTION_ENGINE  Version 1.17  <Build
> 18>
> >           Rules Object: web-misc  Version 1.0  <Build 1>
> >           Rules Object: web-iis  Version 1.0  <Build 1>
> >           Rules Object: web-client  Version 1.0  <Build 1>
> >           Rules Object: web-activex  Version 1.0  <Build 1>
> >           Rules Object: specific-threats  Version 1.0  <Build 1>
> >           Rules Object: snmp  Version 1.0  <Build 1>
> >           Rules Object: smtp  Version 1.0  <Build 1>
> >           Rules Object: p2p  Version 1.0  <Build 1>
> >           Rules Object: nntp  Version 1.0  <Build 1>
> >           Rules Object: netbios  Version 1.0  <Build 1>
> >           Rules Object: multimedia  Version 1.0  <Build 1>
> >           Rules Object: misc  Version 1.0  <Build 1>
> >           Rules Object: imap  Version 1.0  <Build 1>
> >           Rules Object: icmp  Version 1.0  <Build 1>
> >           Rules Object: exploit  Version 1.0  <Build 1>
> >           Rules Object: dos  Version 1.0  <Build 1>
> >           Rules Object: chat  Version 1.0  <Build 1>
> >           Rules Object: bad-traffic  Version 1.0  <Build 1>
> >           Preprocessor Object: SF_DNP3  Version 1.1  <Build 1>
> >           Preprocessor Object: SF_MODBUS  Version 1.1  <Build 1>
> >           Preprocessor Object: SF_GTP  Version 1.1  <Build 1>
> >           Preprocessor Object: SF_REPUTATION  Version 1.1  <Build 1>
> >           Preprocessor Object: SF_SIP  Version 1.1  <Build 1>
> >           Preprocessor Object: SF_SDF  Version 1.1  <Build 1>
> >           Preprocessor Object: SF_DCERPC2  Version 1.0  <Build 3>
> >           Preprocessor Object: SF_SSLPP  Version 1.1  <Build 4>
> >           Preprocessor Object: SF_DNS  Version 1.1  <Build 4>
> >           Preprocessor Object: SF_SSH  Version 1.1  <Build 3>
> >           Preprocessor Object: SF_SMTP  Version 1.1  <Build 9>
> >           Preprocessor Object: SF_IMAP  Version 1.0  <Build 1>
> >           Preprocessor Object: SF_POP  Version 1.0  <Build 1>
> >           Preprocessor Object: SF_FTPTELNET  Version 1.2  <Build 13>
> > Commencing packet processing (pid=18056)
> > ^C*** Caught Int-Signal
> >
> ===============================================================================
> > Run time for packet processing was 421.51287 seconds
> > Snort processed 630885 packets.
> > Snort ran for 0 days 0 hours 7 minutes 1 seconds
> >   Pkts/min:        90126
> >   Pkts/sec:         1498
> >
> ===============================================================================
> > Packet Performance Summary:
> >   max packet time       : 10000 usecs
> >   packet events         : 0
> >   avg pkt time          : 5.9247 usecs
> > Rule Performance Summary:
> >   max rule time         : 4096 usecs
> >   rule events           : 0
> >
> ===============================================================================
> > Packet I/O Totals:
> >   Received:      1863847
> >   Analyzed:       630885 ( 33.849%)
> >    Dropped:       601452 ( 24.397%)
> >   Filtered:            0 (  0.000%)
> > Outstanding:      1232962 ( 66.151%)
> >   Injected:            0
> >
> ===============================================================================
> >
> > Not really good numbers ....
> >
> >
> > Stats with only misc.rules and multimedia.rules (5 min, more or less):
> >
> > Rule application order:
> > activation->dynamic->pass->drop->sdrop->reject->alert->log
> > Verifying Preprocessor Configurations!
> > ICMP tracking disabled, no ICMP sessions allocated
> > IP tracking disabled, no IP sessions allocated
> > WARNING: flowbits key 'file.vqf' is checked but not ever set.
> > WARNING: flowbits key 'file.wmp_playlist' is checked but not ever set.
> > 8 out of 1024 flowbits in use.
> >
> > [ Port Based Pattern Matching Memory ]
> > +- [ Aho-Corasick Summary ] -------------------------------------
> > | Storage Format    : Full-Q
> > | Finite Automaton  : DFA
> > | Alphabet Size     : 256 Chars
> > | Sizeof State      : Variable (1,2,4 bytes)
> > | Instances         : 27
> > |     1 byte states : 26
> > |     2 byte states : 1
> > |     4 byte states : 0
> > | Characters        : 1562
> > | States            : 1446
> > | Transitions       : 16926
> > | State Density     : 4.6%
> > | Patterns          : 90
> > | Match States      : 88
> > | Memory (KB)       : 562.24
> > |   Pattern         : 10.08
> > |   Match Lists     : 19.52
> > |   DFA
> > |     1 byte states : 261.06
> > |     2 byte states : 225.67
> > |     4 byte states : 0.00
> > +----------------------------------------------------------------
> > [ Number of patterns truncated to 20 bytes: 4 ]
> >
> > Packet Performance Monitor Config:
> >  ticks per usec  : 2422 ticks
> >  max packet time : 10000 usecs
> >  packet action   : fastpath-expensive-packets
> >  packet logging  : log
> >  debug-pkts      : disabled
> >
> > Rule Performance Monitor Config:
> >  ticks per usec  : 2422 ticks
> >  max rule time   : 4096 usecs
> >  rule action     : suspend-expensive-rules
> >  rule threshold  : 5
> >  suspend timeout : 10 secs
> >  rule logging    : log
> > pcap DAQ configured to passive.
> > Acquiring network traffic from "em4".
> > Reload thread starting...
> > Reload thread started, thread 0x4aa997dc00 (32237)
> > Decoding Ethernet
> >
> >        --== Initialization Complete ==--
> >
> >   ,,_     -*> Snort! <*-
> >  o"  )~   Version 2.9.4.6 GRE (Build 73)
> >   ''''    By Martin Roesch & The Snort Team:
> > http://www.snort.org/snort/snort-team
> >           Copyright (C) 1998-2013 Sourcefire, Inc., et al.
> >           Using libpcap version 1.3.0
> >           Using PCRE version: 8.31 2012-07-06
> >           Using ZLIB version: 1.2.3
> >
> >           Rules Engine: SF_SNORT_DETECTION_ENGINE  Version 1.17  <Build
> 18>
> >           Rules Object: web-misc  Version 1.0  <Build 1>
> >           Rules Object: web-iis  Version 1.0  <Build 1>
> >           Rules Object: web-client  Version 1.0  <Build 1>
> >           Rules Object: web-activex  Version 1.0  <Build 1>
> >           Rules Object: specific-threats  Version 1.0  <Build 1>
> >           Rules Object: snmp  Version 1.0  <Build 1>
> >           Rules Object: smtp  Version 1.0  <Build 1>
> >           Rules Object: p2p  Version 1.0  <Build 1>
> >           Rules Object: nntp  Version 1.0  <Build 1>
> >           Rules Object: netbios  Version 1.0  <Build 1>
> >           Rules Object: multimedia  Version 1.0  <Build 1>
> >           Rules Object: misc  Version 1.0  <Build 1>
> >           Rules Object: imap  Version 1.0  <Build 1>
> >           Rules Object: icmp  Version 1.0  <Build 1>
> >           Rules Object: exploit  Version 1.0  <Build 1>
> >           Rules Object: dos  Version 1.0  <Build 1>
> >           Rules Object: chat  Version 1.0  <Build 1>
> >           Rules Object: bad-traffic  Version 1.0  <Build 1>
> >           Preprocessor Object: SF_DNP3  Version 1.1  <Build 1>
> >           Preprocessor Object: SF_MODBUS  Version 1.1  <Build 1>
> >           Preprocessor Object: SF_GTP  Version 1.1  <Build 1>
> >           Preprocessor Object: SF_REPUTATION  Version 1.1  <Build 1>
> >           Preprocessor Object: SF_SIP  Version 1.1  <Build 1>
> >           Preprocessor Object: SF_SDF  Version 1.1  <Build 1>
> >           Preprocessor Object: SF_DCERPC2  Version 1.0  <Build 3>
> >           Preprocessor Object: SF_SSLPP  Version 1.1  <Build 4>
> >           Preprocessor Object: SF_DNS  Version 1.1  <Build 4>
> >           Preprocessor Object: SF_SSH  Version 1.1  <Build 3>
> >           Preprocessor Object: SF_SMTP  Version 1.1  <Build 9>
> >           Preprocessor Object: SF_IMAP  Version 1.0  <Build 1>
> >           Preprocessor Object: SF_POP  Version 1.0  <Build 1>
> >           Preprocessor Object: SF_FTPTELNET  Version 1.2  <Build 13>
> > Commencing packet processing (pid=32237)
> > ^C*** Caught Int-Signal
> >
> ===============================================================================
> > Run time for packet processing was 368.552024 seconds
> > Snort processed 643495 packets.
> > Snort ran for 0 days 0 hours 6 minutes 8 seconds
> >   Pkts/min:       107249
> >   Pkts/sec:         1748
> >
> ===============================================================================
> > Packet Performance Summary:
> >   max packet time       : 10000 usecs
> >   packet events         : 0
> >   avg pkt time          : 8.95128 usecs
> > Rule Performance Summary:
> >   max rule time         : 4096 usecs
> >   rule events           : 0
> >   avg rule time         : 1.96408 usecs
> >
> ===============================================================================
> > Packet I/O Totals:
> >   Received:      2121798
> >   Analyzed:       643495 ( 30.328%)
> >    Dropped:       618918 ( 22.582%)
> >   Filtered:            0 (  0.000%)
> > Outstanding:      1478303 ( 69.672%)
> >   Injected:            0
> >
> ===============================================================================
> >
> > About tunning sysctl options, if I am not wrong, OpenBSD tunes them
> > "on the fly" according to network load.
> >
> > And more info: I have installed suricata in this host also to do more
> > tests, and suricata returns me best perfomance without losing many
> > packets:
> >
> > -------------------------------------------------------------------
> > Counter                   | TM Name                   | Value
> > -------------------------------------------------------------------
> > capture.kernel_packets    | RxPcapem51                | 3052575199
> > capture.kernel_drops      | RxPcapem51                | 143259
> > capture.kernel_ifdrops    | RxPcapem51                | 0
> > decoder.pkts              | RxPcapem51                | 19561319
> > decoder.bytes             | RxPcapem51                | 15561225326
> > decoder.ipv4              | RxPcapem51                | 19561319
> > decoder.ipv6              | RxPcapem51                | 0
> > decoder.ethernet          | RxPcapem51                | 19561319
> > decoder.raw               | RxPcapem51                | 0
> > decoder.sll               | RxPcapem51                | 0
> > decoder.tcp               | RxPcapem51                | 19561139
> > decoder.udp               | RxPcapem51                | 0
> > decoder.sctp              | RxPcapem51                | 0
> > decoder.icmpv4            | RxPcapem51                | 180
> > decoder.icmpv6            | RxPcapem51                | 0
> > decoder.ppp               | RxPcapem51                | 0
> > decoder.pppoe             | RxPcapem51                | 0
> > decoder.gre               | RxPcapem51                | 0
> > decoder.vlan              | RxPcapem51                | 0
> > decoder.teredo            | RxPcapem51                | 0
> > decoder.ipv4_in_ipv6      | RxPcapem51                | 0
> > decoder.ipv6_in_ipv6      | RxPcapem51                | 0
> > decoder.avg_pkt_size      | RxPcapem51                | 796
> > decoder.max_pkt_size      | RxPcapem51                | 1506
> > defrag.ipv4.fragments     | RxPcapem51                | 0
> > defrag.ipv4.reassembled   | RxPcapem51                | 0
> > defrag.ipv4.timeouts      | RxPcapem51                | 0
> > defrag.ipv6.fragments     | RxPcapem51                | 0
> > defrag.ipv6.reassembled   | RxPcapem51                | 0
> > defrag.ipv6.timeouts      | RxPcapem51                | 0
> > defrag.max_frag_hits      | RxPcapem51                | 0
> > tcp.sessions              | Detect                    | 66702
> > tcp.ssn_memcap_drop       | Detect                    | 0
> > tcp.pseudo                | Detect                    | 7500
> > tcp.invalid_checksum      | Detect                    | 2
> > tcp.no_flow               | Detect                    | 0
> > tcp.reused_ssn            | Detect                    | 0
> > tcp.memuse                | Detect                    | 36175872
> > tcp.syn                   | Detect                    | 131466
> > tcp.synack                | Detect                    | 129929
> > tcp.rst                   | Detect                    | 56046
> > tcp.segment_memcap_drop   | Detect                    | 0
> > tcp.stream_depth_reached  | Detect                    | 306
> > tcp.reassembly_memuse     | Detect                    | 69060696
> > tcp.reassembly_gap        | Detect                    | 3214
> > detect.alert              | Detect                    | 38
> > flow_mgr.closed_pruned    | FlowManagerThread         | 78944
> > flow_mgr.new_pruned       | FlowManagerThread         | 3978
> > flow_mgr.est_pruned       | FlowManagerThread         | 2390
> > flow.memuse               | FlowManagerThread         | 3852512
> > flow.spare                | FlowManagerThread         | 10000
> > flow.emerg_mode_entered   | FlowManagerThread         | 0
> > flow.emerg_mode_over      | FlowManagerThread         | 0
> >
> > Relevant data here are tcp.reassembly_gap and tcp.invalid_checksum
> numbers.
> >
> > Any idea please??
> >
> >
> ------------------------------------------------------------------------------
> > Get 100% visibility into Java/.NET code with AppDynamics Lite
> > It's a free troubleshooting tool designed for production
> > Get down to code-level detail for bottlenecks, with <2% overhead.
> > Download for free and get started troubleshooting in minutes.
> > http://p.sf.net/sfu/appdyn_d2d_ap2
> > _______________________________________________
> > Snort-users mailing list
> > Snort-users at lists.sourceforge.net
> > Go to this URL to change user options or unsubscribe:
> > https://lists.sourceforge.net/lists/listinfo/snort-users
> > Snort-users list archive:
> > http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users
> >
> > Please visit http://blog.snort.org to stay current on all the latest
> Snort news!
>
>
>
> ------------------------------------------------------------------------------
> How ServiceNow helps IT people transform IT departments:
> 1. A cloud service to automate IT design, transition and operations
> 2. Dashboards that offer high-level views of enterprise services
> 3. A single system of record for all IT processes
> http://p.sf.net/sfu/servicenow-d2d-j
> _______________________________________________
> Snort-users mailing list
> Snort-users at lists.sourceforge.net
> Go to this URL to change user options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users
>
> Please visit http://blog.snort.org to stay current on all the latest
> Snort news!
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.snort.org/pipermail/snort-users/attachments/20130605/c1859095/attachment.html>


More information about the Snort-users mailing list