[Snort-users] Poor performance with Snort 2.9.4.6 under OpenBSD 5.3

C. L. Martinez carlopmart at ...11827...
Thu Jun 6 03:36:11 EDT 2013


On Wed, Jun 5, 2013 at 5:08 PM, Victor Roemer <vroemer at ...1935...> wrote:
> Martinez, as Joel already mentioned, we'll want to see your Snort
> configuration. Shutdown stats would also be useful, but perfmon data would
> be better; if those can be provided.
>
> You mentioned that OpenBSD configured the network sysctl parameters "on the
> fly"; could you direct us to some documentation about this?
>
> You also mentioned that Snort was listening on em3, however the startup
> information in your email indicates that Snort is listening on em4, could
> you elaborate on this setup?
>
>
> Regarding Suricata, I personally do not have any experience in deploying or
> configuring it. That said, are you using, relatively, the same
> configurations? (e.g., any rules enabled, acquiring packets via libpcap,
> etc..)
>
> Also, why are "tcp.reassembly_gap" and "tcp.invalid_checksum" relevant?
>

Ok, first of all sorry for this long post. I will try to answer to all
questions. First, my environment:

Host: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz, 8 GiB RAM
Snort: release 2.9.4.6 compiled against libpcap 1.3.0
Suricata: release 1.4.2 compiled against libpcap 1.3.0

Snort is listening in em4 interface and suricata in em5 (there was a
typo error previously).

Performance data using my actual snort.conf (with few rules and
without so_rules or IP based rules):

Packet Performance Monitor Config:
  ticks per usec  : 2405 ticks
  max packet time : 10000 usecs
  packet action   : fastpath-expensive-packets
  packet logging  : log
  debug-pkts      : disabled

Rule Performance Monitor Config:
  ticks per usec  : 2405 ticks
  max rule time   : 4096 usecs
  rule action     : suspend-expensive-rules
  rule threshold  : 5
  suspend timeout : 10 secs
  rule logging    : log
pcap DAQ configured to passive.
Acquiring network traffic from "em4".
Reload thread starting...
Reload thread started, thread 0x1ee8c530a500 (4050)
Decoding Ethernet

        --== Initialization Complete ==--

   ,,_     -*> Snort! <*-
  o"  )~   Version 2.9.4.6 GRE (Build 73)
   ''''    By Martin Roesch & The Snort Team:
http://www.snort.org/snort/snort-team
           Copyright (C) 1998-2013 Sourcefire, Inc., et al.
           Using libpcap version 1.3.0
           Using PCRE version: 8.31 2012-07-06
           Using ZLIB version: 1.2.3

           Rules Engine: SF_SNORT_DETECTION_ENGINE  Version 1.17  <Build 18>
           Preprocessor Object: SF_DNP3  Version 1.1  <Build 1>
           Preprocessor Object: SF_MODBUS  Version 1.1  <Build 1>
           Preprocessor Object: SF_GTP  Version 1.1  <Build 1>
           Preprocessor Object: SF_REPUTATION  Version 1.1  <Build 1>
           Preprocessor Object: SF_SIP  Version 1.1  <Build 1>
           Preprocessor Object: SF_SDF  Version 1.1  <Build 1>
           Preprocessor Object: SF_DCERPC2  Version 1.0  <Build 3>
           Preprocessor Object: SF_SSLPP  Version 1.1  <Build 4>
           Preprocessor Object: SF_DNS  Version 1.1  <Build 4>
           Preprocessor Object: SF_SSH  Version 1.1  <Build 3>
           Preprocessor Object: SF_SMTP  Version 1.1  <Build 9>
           Preprocessor Object: SF_IMAP  Version 1.0  <Build 1>
           Preprocessor Object: SF_POP  Version 1.0  <Build 1>
           Preprocessor Object: SF_FTPTELNET  Version 1.2  <Build 13>
Commencing packet processing (pid=4050)
===============================================================================
Run time for packet processing was 368.552024 seconds
Snort processed 643495 packets.
Snort ran for 0 days 0 hours 6 minutes 8 seconds
   Pkts/min:       107249
   Pkts/sec:         1748
===============================================================================
Packet Performance Summary:
   max packet time       : 10000 usecs
   packet events         : 0
   avg pkt time          : 8.95128 usecs
Rule Performance Summary:
   max rule time         : 4096 usecs
   rule events           : 0
   avg rule time         : 1.96408 usecs
===============================================================================
Packet I/O Totals:
   Received:      2121798
   Analyzed:       643495 ( 30.328%)
    Dropped:       618918 ( 22.582%)
   Filtered:            0 (  0.000%)
Outstanding:      1478303 ( 69.672%)
   Injected:            0
===============================================================================

And here they are some statistics outputs:

top output:

load averages:  0.18,  0.18,  0.13
21 processes: 20 idle, 1 on processor
CPU0 states:  0.0% user,  0.0% nice,  0.0% system,  0.6% interrupt, 99.4% idle
CPU1 states:  1.0% user,  0.0% nice,  0.0% system,  0.0% interrupt, 99.0% idle
CPU2 states:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
CPU3 states:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
Memory: Real: 506M/2942M act/tot Free: 5018M Cache: 2319M Swap: 0K/6142M

  PID USERNAME PRI NICE  SIZE   RES STATE     WAIT      TIME    CPU COMMAND
 4050 root       4    0  417M  151M sleep/0   bpf       0:06  2.83% snort
31522 root      10    0  356M  328M sleep/0   nanosle  35:09  0.73% suricata

Status interfaces (systat ifstat):

    2 users    Load 0.17 0.17 0.13

IFACE            STATE  DESC
                     IPKTS   IBYTES    IERRS    OPKTS   OBYTES
OERRS    COLLS
em0              up
                         1       72        0        0       71
0        0
em1              up
                         0        0        0        0        0
0        0
em2              up
                         0        0        0        0        0
0        0
em3              up
                         1      334        0        2     1732
0        0
em4              up
                      6773  4771755        0        0        0
0        0
em5              up
                      6773  4771755        0        0        0
0        0

Interfaces configruation options:

em4: flags=8bc3<UP,BROADCAST,RUNNING,NOARP,PROMISC,ALLMULTI,SIMPLEX,MULTICAST>
mtu 1514
        lladdr 00:50:56:17:fc:cd
        priority: 0
        media: Ethernet autoselect (1000baseT full-duplex,master)
        status: active
        inet6 fe80::250:56ff:fe17:fccd%em4 prefixlen 64 scopeid 0x5
em5: flags=8bc3<UP,BROADCAST,RUNNING,NOARP,PROMISC,ALLMULTI,SIMPLEX,MULTICAST>
mtu 1514
        lladdr 00:50:56:18:79:51
        priority: 0
        media: Ethernet autoselect (1000baseT full-duplex,master)
        status: active
        inet6 fe80::250:56ff:fe18:7951%em5 prefixlen 64 scopeid 0x6

Checking mbuffers (systat mbufs):

    2 users    Load 0.18 0.17 0.13

IFACE             LIVELOCKS  SIZE ALIVE   LWM   HWM   CWM
System                    0   256   361          59
                               2k   351         457
lo0
em0                            2k     7     4   256     7
em1                            2k     6     4   256     6
em2                            2k     4     4   256     4
em3                            2k     7     4   256     7
em4                            2k    69     4   256    69
em5                            2k    73     4   256    73

sysctl.conf options enabled:
kern.bufcachepercent=80
net.bpf.bufsize=65536
net.bpf.maxbufsize=2097152

Ok, now some Suricata stats:

5/6/2013 -- 14:29:09 - <Info> - stream "max-sessions": 262144
5/6/2013 -- 14:29:09 - <Info> - stream "prealloc-sessions": 32768
5/6/2013 -- 14:29:09 - <Info> - stream "memcap": 33554432
5/6/2013 -- 14:29:09 - <Info> - stream "midstream" session pickups: disabled
5/6/2013 -- 14:29:09 - <Info> - stream "async-oneside": disabled
5/6/2013 -- 14:29:09 - <Info> - stream "checksum-validation": enabled
5/6/2013 -- 14:29:09 - <Info> - stream."inline": disabled
5/6/2013 -- 14:29:09 - <Info> - stream.reassembly "memcap": 67108864
5/6/2013 -- 14:29:09 - <Info> - stream.reassembly "depth": 1048576
5/6/2013 -- 14:29:09 - <Info> - stream.reassembly "toserver-chunk-size": 2560
5/6/2013 -- 14:29:09 - <Info> - stream.reassembly "toclient-chunk-size": 2560
5/6/2013 -- 14:29:09 - <Info> - all 7 packet processing threads, 3
management threads initialized, engine started.
5/6/2013 -- 14:29:13 - <Info> - No packets with invalid checksum,
assuming checksum offloading is NOT used
6/6/2013 -- 06:27:20 - <Info> - Signal Received.  Stopping engine.
6/6/2013 -- 06:27:20 - <Info> - 0 new flows, 0 established flows were
timed out, 0 flows in closed state
6/6/2013 -- 06:27:21 - <Info> - time elapsed 57492.090s
6/6/2013 -- 06:27:21 - <Info> - (RxPcapem51) Packets 7317731, bytes 5500227933
6/6/2013 -- 06:27:21 - <Info> - (RxPcapem51) Pcap Total:2974089715
Recv:2974089715 Drop:0 (0.0%).
6/6/2013 -- 06:27:21 - <Info> - AutoFP - Total flow handler queues - 6
6/6/2013 -- 06:27:21 - <Info> - AutoFP - Queue 0  - pkts: 2754800
flows: 19921
6/6/2013 -- 06:27:21 - <Info> - AutoFP - Queue 1  - pkts: 2158501
flows: 15267
6/6/2013 -- 06:27:21 - <Info> - AutoFP - Queue 2  - pkts: 1276685
flows: 9932
6/6/2013 -- 06:27:21 - <Info> - AutoFP - Queue 3  - pkts: 678835
flows: 5843
6/6/2013 -- 06:27:21 - <Info> - AutoFP - Queue 4  - pkts: 358226
flows: 3109
6/6/2013 -- 06:27:21 - <Info> - AutoFP - Queue 5  - pkts: 127487
flows: 1488

-------------------------------------------------------------------
Date: 6/6/2013 -- 06:27:21 (uptime: 0d, 15h 58m 44s)
-------------------------------------------------------------------
Counter                   | TM Name                   | Value
-------------------------------------------------------------------
capture.kernel_packets    | RxPcapem51                | 2974088271
capture.kernel_drops      | RxPcapem51                | 0
capture.kernel_ifdrops    | RxPcapem51                | 0
decoder.pkts              | RxPcapem51                | 7317731
decoder.bytes             | RxPcapem51                | 5500227933
decoder.ipv4              | RxPcapem51                | 7317731
decoder.ipv6              | RxPcapem51                | 0
decoder.ethernet          | RxPcapem51                | 7317731
decoder.raw               | RxPcapem51                | 0
decoder.sll               | RxPcapem51                | 0
decoder.tcp               | RxPcapem51                | 7317660
decoder.udp               | RxPcapem51                | 0
decoder.sctp              | RxPcapem51                | 0
decoder.icmpv4            | RxPcapem51                | 71
decoder.icmpv6            | RxPcapem51                | 0
decoder.ppp               | RxPcapem51                | 0
decoder.pppoe             | RxPcapem51                | 0
decoder.gre               | RxPcapem51                | 0
decoder.vlan              | RxPcapem51                | 0
decoder.teredo            | RxPcapem51                | 0
decoder.ipv4_in_ipv6      | RxPcapem51                | 0
decoder.ipv6_in_ipv6      | RxPcapem51                | 0
decoder.avg_pkt_size      | RxPcapem51                | 752
decoder.max_pkt_size      | RxPcapem51                | 1506
defrag.ipv4.fragments     | RxPcapem51                | 0
defrag.ipv4.reassembled   | RxPcapem51                | 0
defrag.ipv4.timeouts      | RxPcapem51                | 0
defrag.ipv6.fragments     | RxPcapem51                | 0
defrag.ipv6.reassembled   | RxPcapem51                | 0
defrag.ipv6.timeouts      | RxPcapem51                | 0
defrag.max_frag_hits      | RxPcapem51                | 0
tcp.sessions              | Detect                    | 41460
tcp.ssn_memcap_drop       | Detect                    | 0
tcp.pseudo                | Detect                    | 5705
tcp.invalid_checksum      | Detect                    | 12
tcp.no_flow               | Detect                    | 0
tcp.reused_ssn            | Detect                    | 0
tcp.memuse                | Detect                    | 36175872
tcp.syn                   | Detect                    | 80827
tcp.synack                | Detect                    | 80035
tcp.rst                   | Detect                    | 33845
tcp.segment_memcap_drop   | Detect                    | 0
tcp.stream_depth_reached  | Detect                    | 148
tcp.reassembly_memuse     | Detect                    | 67770240
tcp.reassembly_gap        | Detect                    | 2222
detect.alert              | Detect                    | 29
flow_mgr.closed_pruned    | FlowManagerThread         | 49711
flow_mgr.new_pruned       | FlowManagerThread         | 3558
flow_mgr.est_pruned       | FlowManagerThread         | 1974
flow.memuse               | FlowManagerThread         | 3884096
flow.spare                | FlowManagerThread         | 10001
flow.emerg_mode_entered   | FlowManagerThread         | 0
flow.emerg_mode_over      | FlowManagerThread         | 0

As you can see, packets dropped counter shows 0% (well, in fact this
is not always the case. Suricata may lost between 1-2% of packets).
And this suricata sensor is configured to use IP based rules, when
snort not.
But there are two counters that can shows if some problems exists:
tcp.reassembly_gap and tcp.invalid_checksum. But as you can see, are
minimal according to Suricata docs:

https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Statistics

When I say: "OpenBSD tunes them "on the fly" according to network
load", I would say that OpenBSD 5.1 and later will dynamically adjust
the TCP send and receive window sizes. Please, refer to:

http://marc.info/?l=openbsd-misc&m=130804020930481&w=2
http://marc.info/?l=openbsd-misc&m=128904951710762&w=2
http://www.openbsd.org/faq/faq6.html#Tuning
http://quigon.bsws.de/papers/2013/AsiaBSDcon/cksums.pdf

Do you need to add more info or perform more tests?

And many thanks for your help.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: snort.conf
Type: application/octet-stream
Size: 26525 bytes
Desc: not available
URL: <https://lists.snort.org/pipermail/snort-users/attachments/20130606/ce221d2c/attachment.obj>


More information about the Snort-users mailing list