[Snort-devel] Stream5 reload bug

Bhagya Bantwal (bbantwal) bbantwal at ...3461...
Tue Aug 12 12:29:36 EDT 2014


Hello Eugenio,

I just wanted to follow up with you and let you know that this issue has
been fixed in the upcoming snort release. Thanks again for reporting this
issue.

Thanks!
On 6/27/14 8:13 AM, "Eugenio Perez" <eugenio at ...3500...> wrote:

>Hi everyone.
>
>I think I've found the cause of Stream5 reloading fail.
>
>Stream5 preprocessor is added to preprocessor list under
>Stream5ReloadVerify
>function. However, in our configuration (and probably in many others
>one), this function is never called.
>
>It has suppose to be called under VerifyReloadedPreprocessors, called
>by ReloadConfigThread. Currently, this function return if some
>preprocessor fails checking it's own configuration, leaving all the
>others preprocessor with no verification function called.
>
>In fact, the failed preprocessors was the ones thats depends on
>stream5: ssh, smtp, and so ones. If you look carefully, these ones are
>loaded before S5 in out snort.conf, so it fails it own preprocessor
>verify.
>
>We have adopted two different solutions:
>
>- Reorder snort.conf so stream5 is loaded just after frag3, and first
>of all others preprocessors
>- Delete the return inside of VerifyReloadedPreprocessors. The
>particular preprocessor prints the error anyway, and it looks like it
>works properly after reload (the FTP rule gid:1,sid:13360 was the one
>tested). We don't know what action to take in this case, so we just
>ignore it and relay in preprocessor verify function to print the
>error.
>
>> 2014-06-23 18:31 GMT+02:00 Juan Jesus Prieto <jjprieto at ...3500...>:
>> Hi all,
>
>>   I am running some tests in snort-2.9.6.1 and I have discovered that
>> when I execute a reload for snort service via HUP signal, most of events
>> fail to sent, never appear again in the unified2 file, all of them
>> related to Stream5 preprocessor (events from other preprocessores work
>> fine after reload).
>
>>   I have make the test in different forms, all of them fails. To
>> reproduce it you can execute snort with this configuration:
>
>>   /usr/local/etc/snort/snort.
>>
>> conf:
>>
>> var BASE_PATH /usr/local/etc/snort
>> include $BASE_PATH/classification.config
>> include $BASE_PATH/reference.config
>> config disable_decode_alerts
>> config disable_tcpopt_experimental_alerts
>> config disable_tcpopt_obsolete_alerts
>> config disable_tcpopt_ttcp_alerts
>> config disable_tcpopt_alerts
>> config disable_ipopt_alerts
>> config checksum_mode: all
>> config pcre_match_limit: 1500
>> config pcre_match_limit_recursion: 1000
>> config detection: search-method ac-split search-optimize
>>max-pattern-len 20
>> config event_queue: max_queue 8 log 3 order_events content_length
>> config ppm: max-pkt-time 150, \
>>     fastpath-expensive-packets
>> config ppm: max-rule-time 100, \
>>     threshold 3, \
>>     suspend-expensive-rules, \
>>     suspend-timeout 20
>> dynamicpreprocessor directory /usr/local/lib/snort_dynamicpreprocessor
>> dynamicengine /usr/local/lib/snort_dynamicengine/libsf_engine.so
>> output unified2: filename snort.log, limit 128
>> preprocessor frag3_global: max_frags 65536, memcap 71303168
>> preprocessor frag3_engine: policy windows detect_anomalies
>>overlap_limit 10 min_fragment_length 100 timeout 180
>> preprocessor sfportscan: proto  { all } memcap { 10000000 } sense_level
>>{ medium }
>> preprocessor smtp: ports { 25 465 587 691 } \
>>      inspection_type stateful \
>>      b64_decode_depth 0 \
>>      qp_decode_depth 0 \
>>      bitenc_decode_depth 0 \
>>      uu_decode_depth 0 \
>>      log_mailfrom \
>>      log_rcptto \
>>      log_filename \
>>      log_email_hdrs \
>>      normalize cmds \
>>      normalize_cmds { ATRN AUTH BDAT CHUNKING DATA DEBUG EHLO EMAL ESAM
>>ESND ESOM ETRN EVFY } \
>>      normalize_cmds { EXPN HELO HELP IDENT MAIL NOOP ONEX QUEU QUIT
>>RCPT RSET SAML SEND SOML } \
>>      normalize_cmds { STARTTLS TICK TIME TURN TURNME VERB VRFY X-ADAT
>>X-DRCP X-ERCP X-EXCH50 } \
>>      normalize_cmds { X-EXPS X-LINK2STATE XADR XAUTH XCIR XEXCH50 XGEN
>>XLICENSE XQUE XSTA XTRN XUSR } \
>>      max_command_line_len 512 \
>>      max_header_line_len 1000 \
>>      max_response_line_len 512 \
>>      alt_max_command_line_len 260 { MAIL } \
>>      alt_max_command_line_len 300 { RCPT } \
>>      alt_max_command_line_len 500 { HELP HELO ETRN EHLO } \
>>      alt_max_command_line_len 255 { EXPN VRFY ATRN SIZE BDAT DEBUG EMAL
>>ESAM ESND ESOM EVFY IDENT NOOP RSET } \
>>      alt_max_command_line_len 246 { SEND SAML SOML AUTH TURN ETRN DATA
>>RSET QUIT ONEX QUEU STARTTLS TICK TIME TURNME VERB X-EXPS X-LINK2STATE
>>XADR XAUTH XCIR XEXCH50 XGEN XLICENSE XQUE XSTA XTRN XUSR } \
>>      valid_cmds { ATRN AUTH BDAT CHUNKING DATA DEBUG EHLO EMAL ESAM
>>ESND ESOM ETRN EVFY } \
>>      valid_cmds { EXPN HELO HELP IDENT MAIL NOOP ONEX QUEU QUIT RCPT
>>RSET SAML SEND SOML } \
>>      valid_cmds { STARTTLS TICK TIME TURN TURNME VERB VRFY X-ADAT
>>X-DRCP X-ERCP X-EXCH50 } \
>>      valid_cmds { X-EXPS X-LINK2STATE XADR XAUTH XCIR XEXCH50 XGEN
>>XLICENSE XQUE XSTA XTRN XUSR } \
>>      xlink2state { enabled }
>> preprocessor ssh: server_ports { 22 } \
>>                    autodetect \
>>                    max_client_bytes 19600 \
>>                    max_encrypted_packets 20 \
>>                    max_server_version_len 100 \
>>                    enable_respoverflow enable_ssh1crc32 \
>>                    enable_srvoverflow enable_protomismatch
>> preprocessor dns: ports { 53 } enable_rdata_overflow
>> preprocessor imap: \
>>     ports { 143 } \
>>     b64_decode_depth 0 \
>>     qp_decode_depth 0 \
>>     bitenc_decode_depth 0 \
>>     uu_decode_depth 0
>> preprocessor pop: \
>>     ports { 110 } \
>>     b64_decode_depth 0 \
>>     qp_decode_depth 0 \
>>     bitenc_decode_depth 0 \
>>     uu_decode_depth 0
>> dynamicdetection directory /usr/local/etc/snort/dynamicrules
>> ipvar HOME_NET any
>> ipvar EXTERNAL_NET any
>> ipvar HTTP_SERVERS any
>> ipvar SMTP_SERVERS any
>> ipvar SQL_SERVERS any
>> ipvar DNS_SERVERS any
>> ipvar TELNET_SERVERS any
>> ipvar AIM_SERVERS
>>64.12.24.0/23,64.12.28.0/23,64.12.161.0/24,64.12.163.0/24,64.12.200.0/24,
>>205.188.3.0/24,205.188.5.0/24,205.188.7.0/24,205.188.9.0/24,205.188.153.0
>>/24,205.188.179.0/24,205.188.248.0/24
>> ipvar SIP_SERVERS any
>> ipvar DNP3_SERVER any
>> ipvar DNP3_CLIENT any
>> ipvar MODBUS_CLIENT any
>> ipvar MODBUS_SERVER any
>> ipvar ENIP_CLIENT any
>> ipvar ENIP_SERVER any
>> portvar HTTP_PORTS
>>80,81,311,591,593,901,1220,1414,1830,2301,2381,2809,3128,3702,4343,5250,7
>>001,7145,7510,7777,7779,8000,8008,8014,8028,8080,8088,8118,8123,8180,8181
>>,8243,8280,8800,8888,8899,9080,9090,9091,9443,9999,11371,55555
>> portvar ORACLE_PORTS 1024:
>> portvar SHELLCODE_PORTS !80
>> portvar SSH_PORTS 22
>> portvar FILE_DATA_PORTS
>>80,81,311,591,593,901,1220,1414,1830,2301,2381,2809,3128,3702,4343,5250,7
>>001,7145,7510,7777,7779,8000,8008,8014,8028,8080,8088,8118,8123,8180,8181
>>,8243,8280,8800,8888,8899,9080,9090,9091,9443,9999,11371,55555,110,143
>> portvar FTP_PORTS 21,2100,3535
>> portvar SIP_PORTS 5060,5061,5600
>> portvar DNP3_PORTS 20000
>> preprocessor normalize_ip4
>> preprocessor normalize_tcp: ips ecn stream
>> preprocessor normalize_icmp4
>> preprocessor normalize_ip6
>> preprocessor normalize_icmp6
>> preprocessor perfmonitor: time 25 file /var/log/snort/snort.stats
>>pktcnt 10000 max_file_size 1048576
>> preprocessor rpc_decode: 111 32770 32771 32772 32773 32774 32775 32776
>>32777 32778 32779 no_alert_multiple_requests no_alert_large_fragments
>>no_alert_incomplete
>> preprocessor bo
>> preprocessor ftp_telnet: global inspection_type stateful
>>encrypted_traffic no
>> preprocessor ftp_telnet_protocol: telnet \
>>      ayt_attack_thresh 20 \
>>      normalize ports { 23 } \
>>      detect_anomalies
>> preprocessor ftp_telnet_protocol: ftp server default \
>>      def_max_param_len 100 \
>>      ports { 21 2100 3535 } \
>>      telnet_cmds yes \
>>      ignore_telnet_erase_cmds yes \
>>      ftp_cmds { ABOR ACCT ADAT ALLO APPE AUTH CCC CDUP } \
>>      ftp_cmds { CEL CLNT CMD CONF CWD DELE ENC EPRT } \
>>      ftp_cmds { EPSV ESTA ESTP FEAT HELP LANG LIST LPRT } \
>>      ftp_cmds { LPSV MACB MAIL MDTM MIC MKD MLSD MLST } \
>>      ftp_cmds { MODE NLST NOOP OPTS PASS PASV PBSZ PORT } \
>>      ftp_cmds { PROT PWD QUIT REIN REST RETR RMD RNFR } \
>>      ftp_cmds { RNTO SDUP SITE SIZE SMNT STAT STOR STOU } \
>>      ftp_cmds { STRU SYST TEST TYPE USER XCUP XCRC XCWD } \
>>      ftp_cmds { XMAS XMD5 XMKD XPWD XRCP XRMD XRSQ XSEM } \
>>      ftp_cmds { XSEN XSHA1 XSHA256 } \
>>      alt_max_param_len 0 { ABOR CCC CDUP ESTA FEAT LPSV NOOP PASV PWD
>>QUIT REIN STOU SYST XCUP XPWD } \
>>      alt_max_param_len 200 { ALLO APPE CMD HELP NLST RETR RNFR STOR
>>STOU XMKD } \
>>      alt_max_param_len 256 { CWD RNTO } \
>>      alt_max_param_len 400 { PORT } \
>>      alt_max_param_len 512 { SIZE } \
>>      chk_str_fmt { ACCT ADAT ALLO APPE AUTH CEL CLNT CMD } \
>>      chk_str_fmt { CONF CWD DELE ENC EPRT EPSV ESTP HELP } \
>>      chk_str_fmt { LANG LIST LPRT MACB MAIL MDTM MIC MKD } \
>>      chk_str_fmt { MLSD MLST MODE NLST OPTS PASS PBSZ PORT } \
>>      chk_str_fmt { PROT REST RETR RMD RNFR RNTO SDUP SITE } \
>>      chk_str_fmt { SIZE SMNT STAT STOR STRU TEST TYPE USER } \
>>      chk_str_fmt { XCRC XCWD XMAS XMD5 XMKD XRCP XRMD XRSQ } \
>>      chk_str_fmt { XSEM XSEN XSHA1 XSHA256 } \
>>      cmd_validity ALLO < int [ char R int ] > \
>>      cmd_validity EPSV < [ { char 12 | char A char L char L } ] > \
>>      cmd_validity MACB < string > \
>>      cmd_validity MDTM < [ date nnnnnnnnnnnnnn[.n[n[n]]] ] string > \
>>      cmd_validity MODE < char ASBCZ > \
>>      cmd_validity PORT < host_port > \
>>      cmd_validity PROT < char CSEP > \
>>      cmd_validity STRU < char FRPO [ string ] > \
>>      cmd_validity TYPE < { char AE [ char NTC ] | char I | char L [
>>number ] } >
>> preprocessor ftp_telnet_protocol: ftp client default \
>>      max_resp_len 256 \
>>      bounce yes \
>>      ignore_telnet_erase_cmds yes \
>>      telnet_cmds yes
>> preprocessor stream5_global: track_tcp yes, \
>>     track_udp yes, \
>>     track_icmp no, \
>>     max_tcp 1048576, \
>>     max_udp 1048576, \
>>     memcap 1073741824, \
>>     max_active_responses 2, \
>>     min_response_seconds 5
>> preprocessor stream5_tcp: policy windows, detect_anomalies,
>>require_3whs 180, \
>>     overlap_limit 10, small_segments 3 bytes 150, timeout 180, \
>>      ports client 21 22 23 25 42 53 79 109 110 111 113 119 135 136 137
>>139 143 \
>>          161 445 513 514 587 593 691 1433 1521 2100 3306 6070 6665 6666
>>6667 6668 6669 \
>>          7000 8181 32770 32771 32772 32773 32774 32775 32776 32777
>>32778 32779, \
>>      ports both 80 81 311 443 465 563 591 593 636 901 989 992 993 994
>>995 1220 1414 1830 2301 2381 2809 3128 3702 5250 7907 7001 7802 7777
>>7779 \
>>          7801 7900 7901 7902 7903 7904 7905 7906 7908 7909 7910 7911
>>7912 7913 7914 7915 7916 \
>>          7917 7918 7919 7920 8000 8008 8028 8080 8088 8118 8123 8180
>>8243 8280 8888 9090 9091 9443 9999 11371
>> preprocessor stream5_udp: timeout 180
>> preprocessor dcerpc2: memcap 102400, events [co ]
>> preprocessor dcerpc2_server: default, policy WinXP, \
>>      detect [smb [139,445], tcp 135, udp 135, rpc-over-http-server
>>593], \
>>      autodetect [tcp 1025:, udp 1025:, rpc-over-http-server 1025:], \
>>      smb_max_chain 3, smb_invalid_shares ["C$", "D$", "ADMIN$"]
>> preprocessor http_inspect: global iis_unicode_map unicode.map 1252
>>compress_depth 65535 decompress_depth 65535
>> preprocessor http_inspect_server: server default \
>>      chunk_length 500000 \
>>      server_flow_depth 300 \
>>      client_flow_depth 300 \
>>      post_depth 65495 \
>>      oversize_dir_length 500 \
>>      max_header_length 750 \
>>      max_headers 100 \
>>      ports { 80 81 311 591 593 901 1220 1414 1830 2301 2381 2809 3128
>>3702 5250 7001 7777 7779 8000 8008 8028 8080 8088 8118 8123 8180 8181
>>8243 8280 8888 9090 9091 9443 9999 11371 } \
>>      non_rfc_char { 0x00 0x01 0x02 0x03 0x04 0x05 0x06 0x07 } \
>>      enable_cookie \
>>      extended_response_inspection \
>>      inspect_gzip \
>>      normalize_utf \
>>      unlimited_decompress \
>>      apache_whitespace no \
>>      ascii no \
>>      bare_byte no \
>>      directory no \
>>      double_decode no \
>>      iis_backslash no \
>>      iis_delimiter no \
>>      iis_unicode no \
>>      multi_slash no \
>>      utf_8 no \
>>      u_encode yes \
>>      webroot no
>> preprocessor dnp3: ports { 20000 } \
>>     memcap 262144 \
>>     check_crc
>> include snort.rules
>>
>> And only one rule from /usr/local/etc/snort/snort.rules:
>>
>> alert  tcp $HOME_NET 21 -> $EXTERNAL_NET any (msg:"APP-DETECT failed
>>FTP login attempt"; flow:to_client,established; content:"530 ";
>>metadata:policy security-ips alert, service ftp;
>>reference:url,www.ietf.org/rfc/rfc0959.txt; classtype:misc-activity;
>>sid:13360; rev:6;)
>>
>>
>> Then I execute snort as follows:
>>
>> # snort -e --pid-path /var/run -i ethX -c
>>/usr/local/etc/snort/snort.conf \
>>    -l /var/log/snort --perfmon-file /var/log/snort/snort.stats
>>
>> If I inject traffic from, for example:
>>
>> http://download.netresec.com/pcap/maccdc-2012/maccdc2012_00000.pcap.gz
>>
>> Using tcpreplay to do it:
>>
>> # tcpreplay -l0 --topspeed --intf1=ethX maccdc2012_00000.pcap
>>
>>
>> then snort detect and launch event with sig id 13660:
>>
>> # u2spewfoo /var/log/snort/snort.log.1403536922 | tail -n 15
>> (Event)
>>      sensor id: 0    event id: 213    event second: 1403537205    event
>>microsecond: 356246
>>      sig id: 13360    gen id: 1    revision: 6     classification: 1
>>      priority: 3    ip source: 192.168.21.101    ip destination:
>>192.168.202.102
>>      src port: 21    dest port: 4004    protocol: 6    impact_flag: 0
>> blocked: 0
>>
>> Packet
>>      sensor id: 0    event id: 213    event second: 1403537205
>>      packet second: 1403537205    packet microsecond: 356246
>>      linktype: 1    packet_length: 80
>> [    0] F0 DE F1 2E 6A 5A 00 16 47 9D F2 C2 81 00 00 78
>>....jZ..G......x
>> [   16] 08 00 45 00 00 3E 39 3A 40 00 3F 06 A1 63 C0 A8
>>..E..>9:@.?..c..
>> [   32] 15 65 C0 A8 CA 66 00 15 0F A4 CE 3D D4 EB 56 E2
>>.e...f.....=..V.
>> [   48] CF 68 50 18 03 91 C4 A2 00 00 35 33 30 20 4C 6F  .hP.......530
>>Lo
>> [   64] 67 69 6E 20 69 6E 63 6F 72 72 65 63 74 2E 0D 0A  gin
>>incorrect...
>>
>>
>> Some Stream5 statisticss from stats file:
>>
>> # declare -A v; \
>>    keys=( $(head /var/log/snort/snort.stats -n2 | tail -n1 | sed
>>'s/^#//' | tr ',' ' ') ); \
>>    count=0; \
>>    for n in $(tail /var/log/snort/snort.stats -n1 | tr ',' ' '); do \
>>      v[${keys[$count]}]=$n; \
>>      count=$(($count+1)); \
>>    done; \
>>    echo "stream5_mem_in_use: ${v['stream5_mem_in_use']}"; \
>>    echo "curr_tcp_sessions_established:
>>${v['curr_tcp_sessions_established']}"
>> stream5_mem_in_use: 13950060
>> curr_tcp_sessions_established: 5195
>>
>>
>> If I send HUP signal to snort process, then the event 13660 never
>> appears again and Stream5 and Sessions statistics goes to zero:
>>
>> # declare -A v; \
>>    keys=( $(head /var/log/snort/snort.stats -n2 | tail -n1 | sed
>>'s/^#//' | tr ',' ' ') ); \
>>    count=0; \
>>    for n in $(tail /var/log/snort/snort.stats -n1 | tr ',' ' '); do \
>>      v[${keys[$count]}]=$n; \
>>      count=$(($count+1)); \
>>    done; \
>>    echo "stream5_mem_in_use: ${v['stream5_mem_in_use']}"; \
>>    echo "curr_tcp_sessions_established:
>>${v['curr_tcp_sessions_established']}"
>> stream5_mem_in_use: 0
>> curr_tcp_sessions_established: 0
>>
>>
>> The only way to recovery service is to restart it completly.
>>
>> Is it a bug in stream5 preprocessor, like I suspect?
>>
>> Thanks in advance and regards.
>>
>>
>>
>>
>> 
>>-------------------------------------------------------------------------
>>-----
>> HPCC Systems Open Source Big Data Platform from LexisNexis Risk
>>Solutions
>> Find What Matters Most in Your Big Data with HPCC Systems
>> Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
>> Leverages Graph Analysis for Fast Processing & Easy Data Exploration
>> http://p.sf.net/sfu/hpccsystems
>> _______________________________________________
>> Snort-devel mailing list
>> Snort-devel at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/snort-devel
>> Archive:
>> http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel
>>
>> Please visit http://blog.snort.org for the latest news about Snort!





More information about the Snort-devel mailing list