[Snort-devel] Snort++: how to get multithreading to work?

Russ rucombs at ...3461...
Wed Jun 17 13:55:20 EDT 2015


Try the -z or --max-packet-threads command line options.

Also:  snort -? | grep thread will help you discover such things.

On 6/17/15 1:41 PM, Prude, Terrell (SCC) wrote:
>
> Hello folks,
>
> This is my first post.  We’ve been running “regular” Snort since the 
> 2.9.5.x days and thought we’d give the new Snort 3.0.0 Alpha a whirl.  
> For us, the major attraction to Snort++ is the multithreading for 
> reasons of capacity.
>
> Unfortunately, I’m having some trouble figuring out how to get that to 
> work.  So far, the Snort process looks like it’s still using only one 
> CPU.    Snort itself seems to start right up and is “snorting” 
> packets, and we are getting output in the Unified2 format.
>
> Could someone point me in the right direction as to what I’m missing?
>
> Platform:
>
> ------------------------------------
>
> Processor:  Intel 4GHz quad-core w/ hyperthreading
>
> DRAM:  32 GB
>
> Disk space:  2TB, with about 1.9TB free
>
> NIC for Snorting:  Intel X520-SR2 10Gbit fiber Ethernet
>
> NIC for management:  Realtek 8169 built-in 1Gbit copper Ethernet
>
> OS:  CentOS 7.1
>
> Snort version:  3.0.0-a1-155
>
> LuaJIT version:  2.0.4
>
> DAQ version:  2.0.5
>
> ------------------------------------
>
> All the ./configure stuff uses the default paths, i. e. the /usr/local 
> tree.  I tried to stay as plain-vanilla as I reasonably could that way.
>
> The configure statement:
>
> ------------------------------------
>
> ./configure --disable-silent-rules --enable-ppm 
> --enable-perf-profiling --enable-large-pcap
>
> ------------------------------------
>
> I then ran the make statement with -j8, per the directions, followed 
> by “make install”.  That looked good.
>
> Command line to invoke Snort:
>
> ------------------------------------
>
> /usr/local/bin/snort -D -i enp1s0f0 -c /usr/local/etc/snort/snort.lua 
> -l /var/log/snort -z 8
>
> ------------------------------------
>
> The log output from when Snort starts:
>
> ------------------------------------
>
> Jun 17 04:07:47 p-its-idssnort2 snort[2984]: 
> --------------------------------------------------
>
> Jun 17 04:07:47 p-its-idssnort2 snort[2984]: o")~   Snort++ 3.0.0-a1-155
>
> Jun 17 04:07:47 p-its-idssnort2 snort[2984]: 
> --------------------------------------------------
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: Loading 
> /usr/local/etc/snort/snort.lua:
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: ips
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: active
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: classifications
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: rpc_decode
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: stream_tcp
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: binder
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: unified2
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: stream
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: stream_ip
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: event_queue
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: detection
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: network
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: normalizer
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: references
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: stream_udp
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: search_engine
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: Finished 
> /usr/local/etc/snort/snort.lua.
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: Loading rules:
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: Loading 
> /usr/local/etc/snort/rules/local.rules:
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: Finished 
> /usr/local/etc/snort/rules/local.rules.
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: Finished rules.
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: 
> --------------------------------------------------
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: rule counts
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: total rules loaded: 2304
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: text rules: 2304
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: option chains: 2304
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: chain headers: 2304
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: 
> --------------------------------------------------
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: rule port counts
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: tcp     udp    icmp      ip
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: any    2304    2304    
> 2304    2304
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: nc       0       0       
> 0    2304
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: 
> --------------------------------------------------
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: pcap DAQ configured to 
> passive.
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2984]: Initializing daemon mode
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2993]: Daemon initialized, 
> signaled parent pid: 2984
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2993]: Writing PID "2993" to 
> file "/var/log/snort/snort.pid"
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2993]: Commencing packet processing
>
> Jun 17 04:07:48 p-its-idssnort2 snort[2993]: ++ [0] enp1s0f0
>
> ------------------------------------
>
> The log output after I kill the Snort process:
>
> ------------------------------------
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: ** caught term signal
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: == stopping
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: -- [0] enp1s0f0
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: 
> --------------------------------------------------
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: Packet Statistics
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: 
> --------------------------------------------------
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: daq
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: pcaps: 1
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: received: 1120415147
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: analyzed: 23673620
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: dropped: 1096740235
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: outstanding: 1096741527
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: allow: 23673620
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: idle: 1
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: 
> --------------------------------------------------
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: codec
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: total: 
> 23673624            (100.000%)
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: other: 
> 555                 (  0.002%)
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: discards: 
> 2609430          ( 11.023%)
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: auth: 769          (  0.003%)
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: esp: 211987        (  0.895%)
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: eth: 23673624      (100.000%)
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: gre: 8574          (  0.036%)
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: icmp4: 
> 2671                (  0.011%)
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: icmp4_ip: 
> 1277             (  0.005%)
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: icmp6: 
> 26                  (  0.000%)
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: ipv4: 23673624     (100.000%)
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: ipv6: 58           (  0.000%)
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: ipv6_no_next: 
> 31           (  0.000%)
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: ppp_encap: 
> 8574            (  0.036%)
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: tcp: 16224198      ( 68.533%)
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: teredo: 
> 58                 (  0.000%)
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: udp: 4838675       ( 20.439%)
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: 
> --------------------------------------------------
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: Module Statistics
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: 
> --------------------------------------------------
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: tcp
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: bad checksum (ip4): 10418
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: 
> --------------------------------------------------
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: binder
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: packets: 4399149
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: inspects: 4399149
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: 
> --------------------------------------------------
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: stream
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: ip flows: 2800
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: tcp flows: 4296173
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: tcp prunes: 4165102
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: udp flows: 100176
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: 
> --------------------------------------------------
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: stream_ip
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: fragments: 220
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: reassembled: 4
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: trackers added: 216
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: trackers freed: 216
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: nodes inserted: 220
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: nodes deleted: 220
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: 
> --------------------------------------------------
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: stream_tcp
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: sessions: 4296173
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: discards: 170170
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: events: 3999594
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: syn trackers: 247856
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: syn-ack trackers: 6903
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: data trackers: 109476
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: trackers created: 364235
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: trackers released: 364235
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: segs queued: 335355
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: segs released: 335355
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: segs split: 227
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: segs used: 56291
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: rebuilt packets: 22755
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: rebuilt buffers: 42889
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: overlaps: 28
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: gaps: 73264
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: max segs: 15128
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: max bytes: 137382
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: client cleanups: 73351
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: server cleanups: 66854
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: 
> --------------------------------------------------
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: stream_udp
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: sessions: 100176
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: created: 100176
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: released: 100176
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: 
> --------------------------------------------------
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: Summary Statistics
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: 
> --------------------------------------------------
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: detection
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: analyzed: 23673620
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: 
> --------------------------------------------------
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: process
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: signals: 1
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: 
> --------------------------------------------------
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: timing
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: runtime: 01:25:16
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: seconds: 5116.16403
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: packets: 23673620
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: pkts/sec: 4627
>
> Jun 17 05:33:04 p-its-idssnort2 snort[2993]: o")~   Snort exiting
>
> ------------------------------------
>
> The “top” output while Snort++ is running:
>
> ------------------------------------
>
> top - 05:20:58 up  2:15,  3 users,  load average: 1.00, 1.01, 0.99
>
> Tasks:*201 *total,*   1 *running,*200 *sleeping,*   0 *stopped,*   0 * 
> zombie
>
> %Cpu0  :*  0.0 *us,*  0.0 *sy,*  0.0 *ni,*100.0 *id,*0.0 *wa,*  0.0 * 
> hi,*  0.0 *si,*  0.0 *st
>
> %Cpu1  :*  0.0 *us,*  0.0 *sy,*  0.0 *ni,*99.7 *id,*0.3 *wa,*  0.0 * 
> hi,*  0.0 *si,*  0.0 *st
>
> %Cpu2  :*  0.0 *us,*  0.0 *sy,*  0.0 *ni,*100.0 *id,*0.0 *wa,*  0.0 * 
> hi,*  0.0 *si,*  0.0 *st
>
> %Cpu3  :*  0.0 *us,*  0.0 *sy,*  0.0 *ni,*100.0 *id,*0.0 *wa,*  0.0 * 
> hi,*  0.0 *si,*  0.0 *st
>
> %Cpu4  :*  0.0 *us,*  0.0 *sy,*  0.0 *ni,*100.0 *id,*0.0 *wa,*  0.0 * 
> hi,*  0.0 *si,*  0.0 *st
>
> %Cpu5  :*  0.0 *us,*  0.0 *sy,*  0.0 *ni,*100.0 *id,*0.0 *wa,*  0.0 * 
> hi,*  0.0 *si,*  0.0 *st
>
> %Cpu6  :*  0.0 *us,*  0.0 *sy,*  0.0 *ni,*100.0 *id,*0.0 *wa,*  0.0 * 
> hi,*  0.0 *si,*  0.0 *st
>
> %Cpu7  :*97.3 *us,*  0.0 *sy,*  0.0 *ni,*  0.0 *id,*0.0 *wa,*  0.0 * 
> hi,*  2.7 *si,*  0.0 *st
>
> KiB Mem :*32703168 *total,*31865964 *free,*   659032 *used,*178172 
> *buff/cache
>
> KiB Swap:*     4092 *total,*     4092 *free,*        0 *used.*31846588 
> *avail Mem
>
> PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM TIME+ COMMAND
>
> 2993 root      20   0  379776 327360   4040 S 100.0  1.0  73:23.35 snort
>
>     1 root      20   0   56652   6728   3908 S   0.0  0.0   0:00.76 
> systemd
>
>     2 root      20   0       0      0      0 S   0.0  0.0   0:00.00 
> kthreadd
>
>     3 root      20   0       0      0      0 S   0.0  0.0   0:00.02 
> ksoftirqd/0
>
>     5 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 
> kworker/0:0H
>
>     7 root      rt   0       0      0      0 S   0.0  0.0   0:00.00 
> migration/0
>
>     8 root      20   0       0      0      0 S   0.0  0.0   0:00.00 rcu_bh
>
>     9 root      20   0       0      0      0 S   0.0  0.0   0:00.00 
> rcuob/0
>
>    10 root      20   0       0      0      0 S   0.0  0.0   0:00.00 
> rcuob/1
>
>    11 root      20   0       0      0      0 S   0.0  0.0   0:00.00 
> rcuob/2
>
>    12 root      20   0       0      0      0 S   0.0  0.0   0:00.00 
> rcuob/3
>
>    13 root      20   0       0      0      0 S   0.0  0.0   0:00.00 
> rcuob/4
>
>    14 root      20   0       0      0      0 S   0.0  0.0   0:00.00 
> rcuob/5
>
>    15 root      20   0       0      0      0 S   0.0  0.0   0:00.00 
> rcuob/6
>
>    16 root      20   0       0      0      0 S   0.0  0.0   0:00.00 
> rcuob/7
>
>    17 root      20   0       0      0      0 S   0.0  0.0   0:00.22 
> rcu_sched
>
>    18 root      20   0       0      0      0 S   0.0  0.0   0:00.09 
> rcuos/0
>
>    19 root      20   0       0      0      0 S   0.0  0.0   0:00.08 
> rcuos/1
>
>    20 root      20   0       0      0      0 S   0.0  0.0   0:00.01 
> rcuos/2
>
>    21 root      20   0       0      0      0 S   0.0  0.0   0:00.01 
> rcuos/3
>
>    22 root      20   0       0      0      0 S   0.0  0.0   0:00.03 
> rcuos/4
>
>    23 root      20   0       0      0      0 S   0.0  0.0   0:00.01 
> rcuos/5
>
>    24 root      20   0       0      0      0 S   0.0  0.0   0:00.03 
> rcuos/6
>
>    25 root      20   0       0      0      0 S   0.0  0.0   0:00.01 
> rcuos/7
>
>    26 root      rt   0       0      0      0 S   0.0  0.0   0:00.00 
> watchdog/0
>
>    27 root      rt   0       0      0      0 S   0.0  0.0   0:00.00 
> watchdog/1
>
>    28 root      rt   0       0      0      0 S   0.0  0.0   0:00.00 
> migration/1
>
>    29 root      20   0       0      0      0 S   0.0  0.0   0:00.01 
> ksoftirqd/1
>
> ------------------------------------
>
> And finally, what the NIC itself is reporting for traffic that it’s 
> seeing.  We’re seeing it come in, all right. J  So far, no errors, 
> collisions, or any other apparent nasties.
>
> ------------------------------------
>
> $ ip -s link show enp1s0f0
>
> 3: enp1s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state 
> UP mode DEFAULT qlen 1000
>
>     link/ether 90:e2:ba:85:28:74 brd ff:ff:ff:ff:ff:ff
>
>     RX: bytes  packets  errors  dropped overrun mcast
>
>     1865322070123 1892842032 0       0 0       8445
>
>     TX: bytes  packets  errors  dropped carrier collsns
>
>     0          0        0       0 0       0
>
> ------------------------------------
>
>
>
> ------------------------------------------------------------------------------
>
>
> _______________________________________________
> Snort-devel mailing list
> Snort-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/snort-devel
> Archive:
> http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel
>
> Please visit http://blog.snort.org for the latest news about Snort!

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.snort.org/pipermail/snort-devel/attachments/20150617/09ed03b6/attachment.html>


More information about the Snort-devel mailing list