[Snort-devel] Bus errors and segmentation faults after upgrade to 2.9.7.3 and daq 2.0.5

Hui cao huica at ...3461...
Thu Jun 4 18:12:13 EDT 2015


Can you disable dce/rpc preprocessor in your configruation and restart 
snort?

Best,
Hui.

On 06/04/2015 06:00 PM, elof at ...969... wrote:
>
> So, my other sensor, on which I disabled chroot and uid/gid change in
> snort.conf to keep snort running as root, I now got a signal 10 and a 
> core
> dumped.
>
> pid 3744 (snort), uid 0: exited on signal 10 (core dumped)
>
>
> # gdb /usr/local/bin/snort snort.core
> GNU gdb 6.1.1 [FreeBSD]
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and 
> you are
> welcome to change it and/or distribute copies of it under certain 
> conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB.  Type "show warranty" for 
> details.
> This GDB was configured as "amd64-marcel-freebsd"...
> Core was generated by `snort'.
> Program terminated with signal 10, Bus error.
> Reading symbols from /usr/local/lib/libdnet.so.1...done.
> Loaded symbols for /usr/local/lib/libdnet.so.1
> Reading symbols from /usr/local/lib/libpcre.so.1...done.
> Loaded symbols for /usr/local/lib/libpcre.so.1
> Reading symbols from /lib/libm.so.5...done.
> Loaded symbols for /lib/libm.so.5
> Reading symbols from /lib/libcrypto.so.7...done.
> Loaded symbols for /lib/libcrypto.so.7
> Reading symbols from /lib/libpcap.so.8...done.
> Loaded symbols for /lib/libpcap.so.8
> Reading symbols from /usr/local/lib/libsfbpf.so.0...done.
> Loaded symbols for /usr/local/lib/libsfbpf.so.0
> Reading symbols from /lib/libz.so.6...done.
> Loaded symbols for /lib/libz.so.6
> Reading symbols from /usr/lib/liblzma.so.5...done.
> Loaded symbols for /usr/lib/liblzma.so.5
> Reading symbols from /lib/libthr.so.3...done.
> Loaded symbols for /lib/libthr.so.3
> Reading symbols from /lib/libc.so.7...done.
> Loaded symbols for /lib/libc.so.7
> Reading symbols from 
> /usr/local/lib/snort_dynamicengine/libsf_engine.so...done.
> Loaded symbols for /usr/local/lib/snort_dynamicengine/libsf_engine.so
> Reading symbols from 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so...done.
> Loaded symbols for 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so
> Reading symbols from 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_dce2_preproc.so...done.
> Loaded symbols for 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_dce2_preproc.so
> Reading symbols from 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_pop_preproc.so...done.
> Loaded symbols for 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_pop_preproc.so
> Reading symbols from 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_ssh_preproc.so...done.
> Loaded symbols for 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_ssh_preproc.so
> Reading symbols from 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_imap_preproc.so...done.
> Loaded symbols for 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_imap_preproc.so
> Reading symbols from 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_dns_preproc.so...done.
> Loaded symbols for 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_dns_preproc.so
> Reading symbols from 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_ssl_preproc.so...done.
> Loaded symbols for 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_ssl_preproc.so
> Reading symbols from 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_sip_preproc.so...done.
> Loaded symbols for 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_sip_preproc.so
> Reading symbols from 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_gtp_preproc.so...done.
> Loaded symbols for 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_gtp_preproc.so
> Reading symbols from 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_ftptelnet_preproc.so...done.
> Loaded symbols for 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_ftptelnet_preproc.so
> Reading symbols from 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_reputation_preproc.so...done.
> Loaded symbols for 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_reputation_preproc.so
> Reading symbols from 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_sdf_preproc.so...done.
> Loaded symbols for 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_sdf_preproc.so
> Reading symbols from 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_modbus_preproc.so...done.
> Loaded symbols for 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_modbus_preproc.so
> Reading symbols from 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_dnp3_preproc.so...done.
> Loaded symbols for 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_dnp3_preproc.so
> Reading symbols from /libexec/ld-elf.so.1...done.
> Loaded symbols for /libexec/ld-elf.so.1
> #0  0x0000000804c48193 in ?? () from 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_sip_preproc.so
> [New Thread 815b9f800 (LWP 100714/snort)]
> [New Thread 802806400 (LWP 100635/snort)]
> (gdb)
> (gdb) backtrace full
> #0  0x0000000804c48193 in ?? () from 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_sip_preproc.so
> No symbol table info available.
> #1  0x0000000804c47e8f in ?? () from 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_sip_preproc.so
> No symbol table info available.
> #2  0x000000000052c967 in s5_paf_callback (ps=0x80ffbd350, 
> ssn=0x8c2942bf0, data=0x8c7c36f48 "32142740|PONG\r\n", len=16, 
> flags=128) at stream_paf.c:185
>         bit = 128
>         paf = PAF_ABORT
>         mask = 128
>         update = false
>         i = 7
> #3  0x000000000052bf7f in s5_paf_eval (pc=0x80374d000, ps=0x80ffbd350, 
> ssn=0x8c2942bf0, port=5600, flags=128, fuzz=150, data=0x8c7c36f48 
> "32142740|PONG\r\n", len=16, ft=0x7fffffffe050) at stream_paf.c:243
> No locals.
> #4  0x000000000052bbdd in s5_paf_check (pv=0x80374d000, 
> ps=0x80ffbd350, ssn=0x8c2942bf0, data=0x8c7c36f48 "32142740|PONG\r\n", 
> len=16, total=32, seq=3414157013, port=5600, flags=0x7fffffffe2b0, 
> fuzz=150)
>     at stream_paf.c:437
>         ft = FT_NOP
>         idx = 16
>         shift = 28893
>         cont = false
>         pc = (PAF_Config *) 0x80374d000
> #5  0x0000000000520766 in flush_pdu_ackd (config=0x80372c000, 
> ssn=0x80ffbd1e0, trk=0x80ffbd338, pkt=0xf18ad0, flags=0x7fffffffe2b0) 
> at snort_stream_tcp.c:9571
>         flush_pt = 8
>         size = 16
>         end = 3414157029
>         pos = 3414157013
>         to_srv = true
>         srv_port = 5600
>         total = 32
>         seg = (StreamSegment *) 0x8c7c36ec0
>         snort_ticks_start = 34512853664
>         snort_ticks_end = 4237555046520209951
> #6  0x000000000051ff15 in CheckFlushPolicyOnAck (config=0x80372c000, 
> tcpssn=0x80ffbd1e0, talker=0x80ffbd338, listener=0x80ffbd1e0, 
> tdb=0x7fffffffe710, p=0xf18ad0) at snort_stream_tcp.c:9729
>         flags = 128
>         flush_amt = 8
>         flushed = 0
> #7  0x0000000000518e0f in ProcessTcp (scb=0x8c2942bf0, p=0xf18ad0, 
> tdb=0x7fffffffe710, s5TcpPolicy=0x812c06000) at snort_stream_tcp.c:9260
>         retcode = 0
>         eventcode = 0
>         ignore = 0 '\0'
>         got_ts = 0
>         new_ssn = 0
>         ts_action = 0
>         tcpssn = (TcpSession *) 0x80ffbd1e0
>         talker = (StreamTracker *) 0x80ffbd338
>         listener = (StreamTracker *) 0x80ffbd1e0
>         require3Way = 0
>         snort_ticks_start = 12884901889
>         snort_ticks_end = 0
> #8  0x0000000000514fd4 in StreamProcessTcp (p=0xf18ad0, 
> scb=0x8c2942bf0, s5TcpPolicy=0x812c06000, skey=0x7fffffffe7c0) at 
> snort_stream_tcp.c:5655
>         tdb = {seq = 3659739341, ack = 3414157029, win = 5159, end_seq 
> = 3659739341, ts = 0}
>         rc = 0
>         status = 15829712
>         snort_ticks_start = 140737488348992
>         snort_ticks_end = 5129361
> #9  0x00000000004dc96b in StreamProcess (p=0xf18ad0, context=0x0) at 
> spp_stream6.c:751
>         key = {ip_l = {4294961120, 32767, 5559925, 0}, ip_h = 
> {1045822549, 0, 364268896, 8}, port_l = 59392, port_h = 65535, 
> vlan_tag = 32767, protocol = 0 '\0', pad = 0 '\0', mplsLabel = 4526162,
>   addressSpaceId = 0, addressSpaceIdPad1 = 256}
>         scb = (SessionControlBlock *) 0x8c2942bf0
>         snort_ticks_start = 0
>         snort_ticks_end = 18446744065119617024
> #10 0x000000000044f542 in DispatchPreprocessors (p=0xf18ad0, 
> policy_id=0, policy=0x802fb2000) at detect.c:136
>         scb = (SessionControlBlock *) 0x8c2942bf0
>         ppn = (PreprocEvalFuncNode *) 0x815b7aee0
>         pps_enabled_foo = 3219496
>         alerts_processed = true
> #11 0x000000000044ef88 in Preprocess (p=0xf18ad0) at detect.c:234
>         retval = 0
>         policy_id = 0
>         policy = (SnortPolicy *) 0x802fb2000
>         pktcnt = 0
>         snort_ticks_start = 34413820928
>         snort_ticks_end = 2683929608
> #12 0x000000000043e9e8 in ProcessPacket (p=0xf18ad0, 
> pkthdr=0x7fffffffe9a0, pkt=0x821a3f77a "", ft=0x0) at snort.c:1873
>         verdict = DAQ_VERDICT_PASS
> #13 0x0000000000445608 in PacketCallback (user=0x0, 
> pkthdr=0x7fffffffe9a0, pkt=0x821a3f77a "") at snort.c:1718
>         inject = 0
>         verdict = DAQ_VERDICT_PASS
>         snort_ticks_start = 34896609280
>         snort_ticks_end = 34896609306
> #14 0x000000000056dc6a in pcap_process_loop ()
> No symbol table info available.
> #15 0x00000008014d0554 in pcap_platform_finddevs () from 
> /lib/libpcap.so.8
> No symbol table info available.
> #16 0x000000000056d7d8 in pcap_daq_acquire ()
> No symbol table info available.
> #17 0x000000000046b66b in DAQ_Acquire (max=0, callback=0x445420 
> <PacketCallback>, user=0x0) at sfdaq.c:541
>         err = 32767
> #18 0x000000000043e47c in PacketLoop () at snort.c:3268
>         error = 0
>         pkts_to_read = 0
> #19 0x000000000043d3d9 in SnortMain (argc=6, argv=0x7fffffffec90) at 
> snort.c:921
>         tmp_ptr = 0x0
>         intf = 0x8028527c8 "mon0"
>         daqInit = 1
> #20 0x000000000043d1f8 in main (argc=6, argv=0x7fffffffec90) at 
> snort.c:817
> No locals.
> (gdb)
> (gdb) info registers
> rax            0x7669643c00000000       8532461177890930688
> rbx            0x15e0   5600
> rcx            0x1      1
> rdx            0x804c47e80      34439724672
> rsi            0x8a46f3b20      37118491424
> rdi            0x33     51
> rbp            0x7fffffffded0   0x7fffffffded0
> rsp            0x7fffffffdea0   0x7fffffffdea0
> r8             0x80     128
> r9             0x80ffbd360      34627900256
> r10            0x7fffffffe050   140737488347216
> r11            0x8c7c36f48      37711212360
> r12            0x821a3f77a      34924132218
> r13            0x821a3f760      34924132192
> r14            0x96     150
> r15            0x3c     60
> rip            0x804c48193      0x804c48193 <strchr at ...3580...+40023>
> eflags         0x10206  66054
> cs             0x43     67
> ss             0x3b     59
> ds             0x0      0
> es             0x0      0
> fs             0x0      0
> gs             0x0      0
>
> (gdb) x/16i $pc
> 0x804c48193 <strchr at ...3580...+40023>: mov    (%rax),%cl
> 0x804c48195 <strchr at ...3580...+40025>: mov    %cl,-0x11(%rbp)
> 0x804c48198 <strchr at ...3580...+40028>: movsbl -0x11(%rbp),%edx
> 0x804c4819c <strchr at ...3580...+40032>: cmp    $0x0,%edx
> 0x804c481a2 <strchr at ...3580...+40038>: jne    0x804c48209 <strchr at ...3580...+40141>
> 0x804c481a8 <strchr at ...3580...+40044>: movzbl -0x1(%rbp),%eax
> 0x804c481ac <strchr at ...3580...+40048>: cmp    $0x3a,%eax
> 0x804c481b1 <strchr at ...3580...+40053>: jne    0x804c481c6 <strchr at ...3580...+40074>
> 0x804c481b7 <strchr at ...3580...+40059>: mov    -0x10(%rbp),%rax
> 0x804c481bb <strchr at ...3580...+40063>: movl   $0x2,(%rax)
> 0x804c481c1 <strchr at ...3580...+40069>: jmpq   0x804c48204 <strchr at ...3580...+40136>
> 0x804c481c6 <strchr at ...3580...+40074>: mov    $0x20000,%rsi
> 0x804c481d0 <strchr at ...3580...+40084>: movzbl -0x1(%rbp),%edi
> 0x804c481d4 <strchr at ...3580...+40088>: callq  0x804c485c0 <strchr at ...3580...+41092>
> 0x804c481d9 <strchr at ...3580...+40093>: cmp    $0x0,%eax
> 0x804c481de <strchr at ...3580...+40098>: jne    0x804c481ff <strchr at ...3580...+40131>
> (gdb)
> (gdb) thread apply all backtrace
>
> Thread 2 (Thread 802806400 (LWP 100635/snort)):
> #0  0x0000000804c48193 in ?? () from 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_sip_preproc.so
> #1  0x0000000804c47e8f in ?? () from 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_sip_preproc.so
> #2  0x000000000052c967 in s5_paf_callback (ps=0x80ffbd350, 
> ssn=0x8c2942bf0, data=0x8c7c36f48 "32142740|PONG\r\n", len=16, 
> flags=128) at stream_paf.c:185
> #3  0x000000000052bf7f in s5_paf_eval (pc=0x80374d000, ps=0x80ffbd350, 
> ssn=0x8c2942bf0, port=5600, flags=128, fuzz=150, data=0x8c7c36f48 
> "32142740|PONG\r\n", len=16, ft=0x7fffffffe050) at stream_paf.c:243
> #4  0x000000000052bbdd in s5_paf_check (pv=0x80374d000, 
> ps=0x80ffbd350, ssn=0x8c2942bf0, data=0x8c7c36f48 "32142740|PONG\r\n", 
> len=16, total=32, seq=3414157013, port=5600, flags=0x7fffffffe2b0, 
> fuzz=150)
>     at stream_paf.c:437
> #5  0x0000000000520766 in flush_pdu_ackd (config=0x80372c000, 
> ssn=0x80ffbd1e0, trk=0x80ffbd338, pkt=0xf18ad0, flags=0x7fffffffe2b0) 
> at snort_stream_tcp.c:9571
> #6  0x000000000051ff15 in CheckFlushPolicyOnAck (config=0x80372c000, 
> tcpssn=0x80ffbd1e0, talker=0x80ffbd338, listener=0x80ffbd1e0, 
> tdb=0x7fffffffe710, p=0xf18ad0) at snort_stream_tcp.c:9729
> #7  0x0000000000518e0f in ProcessTcp (scb=0x8c2942bf0, p=0xf18ad0, 
> tdb=0x7fffffffe710, s5TcpPolicy=0x812c06000) at snort_stream_tcp.c:9260
> #8  0x0000000000514fd4 in StreamProcessTcp (p=0xf18ad0, 
> scb=0x8c2942bf0, s5TcpPolicy=0x812c06000, skey=0x7fffffffe7c0) at 
> snort_stream_tcp.c:5655
> #9  0x00000000004dc96b in StreamProcess (p=0xf18ad0, context=0x0) at 
> spp_stream6.c:751
> #10 0x000000000044f542 in DispatchPreprocessors (p=0xf18ad0, 
> policy_id=0, policy=0x802fb2000) at detect.c:136
> #11 0x000000000044ef88 in Preprocess (p=0xf18ad0) at detect.c:234
> #12 0x000000000043e9e8 in ProcessPacket (p=0xf18ad0, 
> pkthdr=0x7fffffffe9a0, pkt=0x821a3f77a "", ft=0x0) at snort.c:1873
> #13 0x0000000000445608 in PacketCallback (user=0x0, 
> pkthdr=0x7fffffffe9a0, pkt=0x821a3f77a "") at snort.c:1718
> #14 0x000000000056dc6a in pcap_process_loop ()
> #15 0x00000008014d0554 in pcap_platform_finddevs () from 
> /lib/libpcap.so.8
> #16 0x000000000056d7d8 in pcap_daq_acquire ()
> #17 0x000000000046b66b in DAQ_Acquire (max=0, callback=0x445420 
> <PacketCallback>, user=0x0) at sfdaq.c:541
> #18 0x000000000043e47c in PacketLoop () at snort.c:3268
> #19 0x000000000043d3d9 in SnortMain (argc=6, argv=0x7fffffffec90) at 
> snort.c:921
> #20 0x000000000043d1f8 in main (argc=6, argv=0x7fffffffec90) at 
> snort.c:817
>
> Thread 1 (Thread 815b9f800 (LWP 100714/snort)):
> #0  0x000000080209a8ba in nanosleep () from /lib/libc.so.7
> #1  0x0000000801fd72ea in sleep () from /lib/libc.so.7
> #2  0x0000000801d5ec63 in sleep () from /lib/libthr.so.3
> #3  0x0000000000446448 in ReloadConfigThread (data=0x0) at snort.c:5695
> #4  0x0000000801d5c4f5 in pthread_create () from /lib/libthr.so.3
> #5  0x0000000000000000 in ?? ()
> #0  0x0000000804c48193 in ?? () from 
> /usr/local/lib/snort_dynamicpreprocessor//libsf_sip_preproc.so
> (gdb)
> (gdb) quit
>
> /Elof
>
>
> On Thu, 4 Jun 2015, Hui Cao (huica) wrote:
>
>> Thanks!
>>
>> The issue happens on smtp preprocessor, but the so is not compiled with
>> debug enabled. Can you recompile it with ―enable-debug ?
>>
>> Best,
>> Hui.
>>
>> On 6/4/15, 12:10 PM, "elof at ...969..." <elof at ...969...> wrote:
>>
>>>
>>> So I just had a signal 6...
>>>
>>> I assume I can't attach files to the mailing list, so here it is,
>>> directly
>>> in the mailbody. :-)
>>>
>>>
>>>
>>>
>>>
>>> gdb /usr/local/bin/snort 11057
>>>
>>> GNU gdb 6.1.1 [FreeBSD]
>>> Copyright 2004 Free Software Foundation, Inc.
>>> GDB is free software, covered by the GNU General Public License, and 
>>> you
>>> are
>>> welcome to change it and/or distribute copies of it under certain
>>> conditions.
>>> Type "show copying" to see the conditions.
>>> There is absolutely no warranty for GDB.  Type "show warranty" for
>>> details.
>>> This GDB was configured as "amd64-marcel-freebsd"...
>>> Attaching to program: /usr/local/bin/snort, process 11057
>>> Reading symbols from /usr/local/lib/libdnet.so.1...done.
>>> Loaded symbols for /usr/local/lib/libdnet.so.1
>>> Reading symbols from /usr/local/lib/libpcre.so.1...done.
>>> Loaded symbols for /usr/local/lib/libpcre.so.1
>>> Reading symbols from /lib/libm.so.5...done.
>>> Loaded symbols for /lib/libm.so.5
>>> Reading symbols from /lib/libcrypto.so.6...done.
>>> Loaded symbols for /lib/libcrypto.so.6
>>> Reading symbols from /lib/libpcap.so.8...done.
>>> Loaded symbols for /lib/libpcap.so.8
>>> Reading symbols from /usr/local/lib/libsfbpf.so.0...done.
>>> Loaded symbols for /usr/local/lib/libsfbpf.so.0
>>> Reading symbols from /lib/libz.so.6...done.
>>> Loaded symbols for /lib/libz.so.6
>>> Reading symbols from /usr/lib/liblzma.so.5...done.
>>> Loaded symbols for /usr/lib/liblzma.so.5
>>> Reading symbols from /lib/libthr.so.3...done.
>>> [New Thread 815a59400 (LWP 100459/snort)]
>>> [New Thread 802407400 (LWP 100375/snort)]
>>> Loaded symbols for /lib/libthr.so.3
>>> Reading symbols from /lib/libc.so.7...done.
>>> Loaded symbols for /lib/libc.so.7
>>> Reading symbols from
>>> /usr/local/lib/snort_dynamicengine/libsf_engine.so...done.
>>> Loaded symbols for /usr/local/lib/snort_dynamicengine/libsf_engine.so
>>> Reading symbols from
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_dce2_preproc.so...done.
>>> Loaded symbols for
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_dce2_preproc.so
>>> Reading symbols from
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_dnp3_preproc.so...done.
>>> Loaded symbols for
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_dnp3_preproc.so
>>> Reading symbols from
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_dns_preproc.so...done.
>>> Loaded symbols for
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_dns_preproc.so
>>> Reading symbols from
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_ftptelnet_preproc.so...don 
>>>
>>> e.
>>> Loaded symbols for
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_ftptelnet_preproc.so
>>> Reading symbols from
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_gtp_preproc.so...done.
>>> Loaded symbols for
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_gtp_preproc.so
>>> Reading symbols from
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_imap_preproc.so...done.
>>> Loaded symbols for
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_imap_preproc.so
>>> Reading symbols from
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_modbus_preproc.so...done. 
>>>
>>> Loaded symbols for
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_modbus_preproc.so
>>> Reading symbols from
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_pop_preproc.so...done.
>>> Loaded symbols for
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_pop_preproc.so
>>> Reading symbols from
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_reputation_preproc.so...do 
>>>
>>> ne.
>>> Loaded symbols for
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_reputation_preproc.so
>>> Reading symbols from
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_sdf_preproc.so...done.
>>> Loaded symbols for
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_sdf_preproc.so
>>> Reading symbols from
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_sip_preproc.so...done.
>>> Loaded symbols for
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_sip_preproc.so
>>> Reading symbols from
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so...done.
>>> Loaded symbols for
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so
>>> Reading symbols from
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_ssh_preproc.so...done.
>>> Loaded symbols for
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_ssh_preproc.so
>>> Reading symbols from
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_ssl_preproc.so...done.
>>> Loaded symbols for
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_ssl_preproc.so
>>> Reading symbols from /libexec/ld-elf.so.1...done.
>>> Loaded symbols for /libexec/ld-elf.so.1
>>> [Switching to Thread 815a59400 (LWP 100459/snort)]
>>> 0x0000000801faa40c in nanosleep () from /lib/libc.so.7
>>> (gdb) set logging file gdb-snort.txt
>>> (gdb) set logging on
>>> Copying output to gdb-snort.txt.
>>> (gdb) continue
>>> Continuing.
>>>
>>>
>>>
>>> <...it has just been a few minutes when I receive a SIGABRT>
>>>
>>>
>>>
>>> Program received signal SIGABRT, Aborted.
>>> [Switching to Thread 802407400 (LWP 100375/snort)]
>>> 0x0000000801f2364c in thr_kill () from /lib/libc.so.7
>>>
>>> (gdb) backtrace full
>>> #0  0x0000000801f2364c in thr_kill () from /lib/libc.so.7
>>> No symbol table info available.
>>> #1  0x0000000801fc7c4b in abort () from /lib/libc.so.7
>>> No symbol table info available.
>>> #2  0x0000000801fab315 in __assert () from /lib/libc.so.7
>>> No symbol table info available.
>>> #3  0x0000000805068395 in ?? () from
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so
>>> No symbol table info available.
>>> #4  0x0000000805068781 in ?? () from
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so
>>> No symbol table info available.
>>> #5  0x000000080506afd0 in ?? () from
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so
>>> No symbol table info available.
>>> #6  0x000000080506b85b in ?? () from
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so
>>> No symbol table info available.
>>> #7  0x000000080506c150 in ?? () from
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so
>>> No symbol table info available.
>>> #8  0x000000080506cb27 in ?? () from
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so
>>> No symbol table info available.
>>> #9  0x00000000004423f7 in DispatchPreprocessors (p=0x8033a3e00,
>>> policy_id=0, policy=0x802faa000) at detect.c:136
>>>     scb = (SessionControlBlock *) 0x8a1aad2f0
>>>     ppn = (PreprocEvalFuncNode *) 0x8033ff0a0
>>>     pps_enabled_foo = 1123336
>>>     alerts_processed = true
>>> #10 0x000000000044286d in Preprocess (p=0x8033a3e00) at detect.c:234
>>>     retval = 0
>>>     policy_id = 0
>>>     policy = (SnortPolicy *) 0x802faa000
>>>     pktcnt = 0
>>>     snort_ticks_start = 34413886976
>>>     snort_ticks_end = 34413888664
>>> #11 0x00000000004e44b8 in _flush_to_seq (tcpssn=0x80811ce50,
>>> st=0x80811cfa8, bytes=3644, p=0x8033a4900, sip=0x8033a51a4,
>>> dip=0x8033a51b8, sp=39946, dp=6400, dir=128) at snort_stream_tcp.c:4352
>>>     tmp_do_detect = 1
>>>     tmp_do_detect_content = 1
>>>     snort_ticks_start = 37073416192
>>>     snort_ticks_end = 37069258752
>>>     start_seq = 846966387
>>>     stop_seq = 1940818286
>>>     footprint = 3644
>>>     bytes_processed = 3644
>>>     flushed_bytes = 3644
>>>     pkth = {ts = {tv_sec = 100375, tv_usec = 0}, caplen = 0, pktlen 
>>> = 0,
>>> ingress_index = -1, egress_index = -1, ingress_group = -1, 
>>> egress_group =
>>> -1, flags = 0, opaque = 8, priv_ptr = 0x8a1800000, flow_id = 535241216,
>>> address_space_id = 0}
>>>     enc_flags = 2147483648
>>>     snort_ticks_start = 51544732022
>>>     snort_ticks_end = 113187
>>> #12 0x00000000004e3e06 in flush_to_seq (tcpssn=0x80811ce50,
>>> st=0x80811cfa8, bytes=3644, p=0x8033a4900, sip=0x8033a51a4,
>>> dip=0x8033a51b8, sp=39946, dp=6400, dir=128) at snort_stream_tcp.c:4493
>>> No locals.
>>> #13 0x00000000004e4f8e in flush_ackd (tcpssn=0x80811ce50, 
>>> st=0x80811cfa8,
>>> p=0x8033a4900, sip=0x8033a51a4, dip=0x8033a51b8, sp=39946, dp=6400,
>>> dir=128) at snort_stream_tcp.c:4559
>>>     bytes = 3644
>>> #14 0x00000000004e3bcb in flush_stream (tcpssn=0x80811ce50,
>>> st=0x80811cfa8, p=0x8033a4900, sip=0x8033a51a4, dip=0x8033a51b8,
>>> sp=39946, dp=6400, dir=128) at snort_stream_tcp.c:4588
>>>     fm = (FlushMgr *) 0x80811cfb4
>>> #15 0x00000000004e60b4 in FlushQueuedSegs (scb=0x8a1aad2f0,
>>> tcpssn=0x80811ce50) at snort_stream_tcp.c:5074
>>>     p = (Packet *) 0x8033a4900
>>>     flushed = 1926
>>>     tmp_pcap_hdr = {ts = {tv_sec = 1433431165, tv_usec = 321125}, 
>>> caplen =
>>> 94, pktlen = 94, ingress_index = 5004089, egress_index = 0, 
>>> ingress_group
>>> = 38246208, egress_group = 8, flags = 4294960320, opaque = 32767,
>>> priv_ptr = 0x4b4c81, flow_id = 0, address_space_id = 0}
>>> #16 0x00000000004e61bd in TcpSessionCleanup (scb=0x8a1aad2f0,
>>> freeApplicationData=1) at snort_stream_tcp.c:5115
>>>     tcpssn = (TcpSession *) 0x80811ce50
>>> #17 0x00000000004e61ea in TcpSessionCleanupWithFreeApplicationData
>>> (scb=0x8a1aad2f0) at snort_stream_tcp.c:5122
>>> No locals.
>>> #18 0x00000000004e69e1 in StreamProcessTcp (p=0xee2a40, 
>>> scb=0x8a1aad2f0,
>>> s5TcpPolicy=0x812807000, skey=0x7fffffffe6c0) at 
>>> snort_stream_tcp.c:5648
>>>     sscc = {old_mem_in_use = 15788887, client_ip = {family = 2, bits 
>>> = 32,
>>> ip = {u6_addr8 = "\nm\027L", '\0' <repeats 11 times>, u6_addr16 = 
>>> {27914,
>>> 19479, 0, 0, 0, 0, 0, 0}, u6_addr32 = {1276603658, 0, 0, 0}}}, 
>>> server_ip
>>> = {family = 2, bits = 32, ip = {u6_addr8 = "\nm\026\024", '\0' <repeats
>>> 11 times>,
>>>       u6_addr16 = {27914, 5142, 0, 0, 0, 0, 0, 0}, u6_addr32 =
>>> {337014026, 0, 0, 0}}}, client_port = 39946, server_port = 6400,
>>> lw_session_state = 200, lw_session_flags = 4284679, app_proto_id = 0}
>>>     tdb = {seq = 1940818286, ack = 2349672268, win = 64032, end_seq =
>>> 1940818325, ts = 0}
>>>     rc = 0
>>>     status = 4512282
>>>     snort_ticks_start = 34397587520
>>>     snort_ticks_end = 140737488348864
>>> #19 0x00000000004b5a14 in StreamProcess (p=0xee2a40, context=0x0) at
>>> spp_stream6.c:751
>>>     key = {ip_l = {0, 0, 4216431, 0}, ip_h = {0, 2, 362856224, 8}, 
>>> port_l =
>>> 59136, port_h = 65535, vlan_tag = 32767, protocol = 0 '\0', pad = 0 
>>> '\0',
>>> mplsLabel = 5328944, addressSpaceId = 0, addressSpaceIdPad1 = 0}
>>>     scb = (SessionControlBlock *) 0x8a1aad2f0
>>>     snort_ticks_start = 140737488348960
>>>     snort_ticks_end = 34722594592
>>> #20 0x00000000004423f7 in DispatchPreprocessors (p=0xee2a40, 
>>> policy_id=0,
>>> policy=0x802faa000) at detect.c:136
>>>     scb = (SessionControlBlock *) 0x8a1aad2f0
>>>     ppn = (PreprocEvalFuncNode *) 0x815b61340
>>>     pps_enabled_foo = 1123336
>>>     alerts_processed = true
>>> #21 0x000000000044286d in Preprocess (p=0xee2a40) at detect.c:234
>>>     retval = 0
>>>     policy_id = 0
>>>     policy = (SnortPolicy *) 0x802faa000
>>>     pktcnt = 0
>>>     snort_ticks_start = 0
>>>     snort_ticks_end = 6059431713369489410
>>> #22 0x00000000004351b3 in ProcessPacket (p=0xee2a40,
>>> pkthdr=0x7fffffffe8c0, pkt=0x82251deaa "\220?"\b?, ft=0x0) at 
>>> snort.c:1873
>>>     verdict = DAQ_VERDICT_PASS
>>>     __func__ = "ProcessPacket"
>>> #23 0x0000000000434ccd in PacketCallback (user=0x0,
>>> pkthdr=0x7fffffffe8c0, pkt=0x82251deaa "\220?"\b?) at snort.c:1718
>>>     inject = 0
>>>     verdict = DAQ_VERDICT_PASS
>>>     snort_ticks_start = 34894979584
>>>     snort_ticks_end = 34367935488
>>> #24 0x000000000052fe34 in pcap_process_loop ()
>>> No symbol table info available.
>>> #25 0x0000000801429dcd in pcap_create_interface () from 
>>> /lib/libpcap.so.8
>>> No symbol table info available.
>>> #26 0x000000000053025f in pcap_daq_acquire ()
>>> No symbol table info available.
>>> #27 0x000000000045a1b4 in DAQ_Acquire (max=0, callback=0x434b40
>>> <PacketCallback>, user=0x0) at sfdaq.c:541
>>>     err = 0
>>> #28 0x0000000000437616 in PacketLoop () at snort.c:3268
>>>     error = 0
>>>     pkts_to_read = 0
>>> #29 0x00000000004337c7 in SnortMain (argc=6, argv=0x7fffffffebc0) at
>>> snort.c:921
>>>     tmp_ptr = 0x0
>>>     intf = 0x8024c4540 "mon0"
>>>     daqInit = 1
>>> #30 0x000000000043364f in main (argc=6, argv=0x7fffffffebc0) at
>>> snort.c:817
>>> No locals.
>>> rax            0x0    0
>>> rbx            0x7fffffffddec    140737488346604
>>> rcx            0x801fc8fbc    34393067452
>>> rdx            0x0    0
>>> rsi            0x6    6
>>> rdi            0x18817    100375
>>> rbp            0x7fffffffde60    0x7fffffffde60
>>> rsp            0x7fffffffddd8    0x7fffffffddd8
>>> r8             0x0    0
>>> r9             0xfffffe0032ea54a8    -2198169037656
>>> r10            0x59    89
>>> r11            0x202    514
>>> r12            0x80811ce50    34495123024
>>> r13            0x8033a51b8    34413892024
>>> r14            0x82251deaa    34935529130
>>> r15            0x1ba24    113188
>>> rip            0x801f2364c    0x801f2364c <thr_kill+12>
>>> eflags         0x206    518
>>> cs             0x43    67
>>> ss             0x3b    59
>>> ds             0x0    0
>>> es             0x0    0
>>> fs             0x0    0
>>> gs             0x0    0
>>> 0x801f2364c <thr_kill+12>:    jb     0x801f2364f <thr_kill+15>
>>> 0x801f2364e <thr_kill+14>:    retq
>>> 0x801f2364f <thr_kill+15>:    mov 0x2d6bea(%rip),%rcx        #
>>> 0x8021fa240 <__nsdefaultsrc+5696>
>>> 0x801f23656 <thr_kill+22>:    jmpq   *%rcx
>>> 0x801f23658 <thr_kill+24>:    nop
>>> 0x801f23659 <thr_kill+25>:    nop
>>> 0x801f2365a <thr_kill+26>:    nop
>>> 0x801f2365b <thr_kill+27>:    nop
>>> 0x801f2365c <thr_kill+28>:    nop
>>> 0x801f2365d <thr_kill+29>:    nop
>>> 0x801f2365e <thr_kill+30>:    nop
>>> 0x801f2365f <thr_kill+31>:    nop
>>> 0x801f23660 <thr_self>:    mov    $0x1b0,%rax
>>> 0x801f23667 <thr_self+7>:    mov    %rcx,%r10
>>> 0x801f2366a <thr_self+10>:    syscall
>>> 0x801f2366c <thr_self+12>:    jb     0x801f2366f <thr_self+15>
>>>
>>> Thread 2 (Thread 802407400 (LWP 100375/snort)):
>>> #0  0x0000000801f2364c in thr_kill () from /lib/libc.so.7
>>> #1  0x0000000801fc7c4b in abort () from /lib/libc.so.7
>>> #2  0x0000000801fab315 in __assert () from /lib/libc.so.7
>>> #3  0x0000000805068395 in ?? () from
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so
>>> #4  0x0000000805068781 in ?? () from
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so
>>> #5  0x000000080506afd0 in ?? () from
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so
>>> #6  0x000000080506b85b in ?? () from
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so
>>> #7  0x000000080506c150 in ?? () from
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so
>>> #8  0x000000080506cb27 in ?? () from
>>> /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so
>>> #9  0x00000000004423f7 in DispatchPreprocessors (p=0x8033a3e00,
>>> policy_id=0, policy=0x802faa000) at detect.c:136
>>> #10 0x000000000044286d in Preprocess (p=0x8033a3e00) at detect.c:234
>>> #11 0x00000000004e44b8 in _flush_to_seq (tcpssn=0x80811ce50,
>>> st=0x80811cfa8, bytes=3644, p=0x8033a4900, sip=0x8033a51a4,
>>> dip=0x8033a51b8, sp=39946, dp=6400, dir=128) at snort_stream_tcp.c:4352
>>> #12 0x00000000004e3e06 in flush_to_seq (tcpssn=0x80811ce50,
>>> st=0x80811cfa8, bytes=3644, p=0x8033a4900, sip=0x8033a51a4,
>>> dip=0x8033a51b8, sp=39946, dp=6400, dir=128) at snort_stream_tcp.c:4493
>>> #13 0x00000000004e4f8e in flush_ackd (tcpssn=0x80811ce50, 
>>> st=0x80811cfa8,
>>> p=0x8033a4900, sip=0x8033a51a4, dip=0x8033a51b8, sp=39946, dp=6400,
>>> dir=128) at snort_stream_tcp.c:4559
>>> #14 0x00000000004e3bcb in flush_stream (tcpssn=0x80811ce50,
>>> st=0x80811cfa8, p=0x8033a4900, sip=0x8033a51a4, dip=0x8033a51b8,
>>> sp=39946, dp=6400, dir=128) at snort_stream_tcp.c:4588
>>> #15 0x00000000004e60b4 in FlushQueuedSegs (scb=0x8a1aad2f0,
>>> tcpssn=0x80811ce50) at snort_stream_tcp.c:5074
>>> #16 0x00000000004e61bd in TcpSessionCleanup (scb=0x8a1aad2f0,
>>> freeApplicationData=1) at snort_stream_tcp.c:5115
>>> #17 0x00000000004e61ea in TcpSessionCleanupWithFreeApplicationData
>>> (scb=0x8a1aad2f0) at snort_stream_tcp.c:5122
>>> #18 0x00000000004e69e1 in StreamProcessTcp (p=0xee2a40, 
>>> scb=0x8a1aad2f0,
>>> s5TcpPolicy=0x812807000, skey=0x7fffffffe6c0) at 
>>> snort_stream_tcp.c:5648
>>> #19 0x00000000004b5a14 in StreamProcess (p=0xee2a40, context=0x0) at
>>> spp_stream6.c:751
>>> #20 0x00000000004423f7 in DispatchPreprocessors (p=0xee2a40, 
>>> policy_id=0,
>>> policy=0x802faa000) at detect.c:136
>>> #21 0x000000000044286d in Preprocess (p=0xee2a40) at detect.c:234
>>> #22 0x00000000004351b3 in ProcessPacket (p=0xee2a40,
>>> pkthdr=0x7fffffffe8c0, pkt=0x82251deaa "\220?"\b?, ft=0x0) at 
>>> snort.c:1873
>>> #23 0x0000000000434ccd in PacketCallback (user=0x0,
>>> pkthdr=0x7fffffffe8c0, pkt=0x82251deaa "\220?"\b?) at snort.c:1718
>>> #24 0x000000000052fe34 in pcap_process_loop ()
>>> #25 0x0000000801429dcd in pcap_create_interface () from 
>>> /lib/libpcap.so.8
>>> #26 0x000000000053025f in pcap_daq_acquire ()
>>> #27 0x000000000045a1b4 in DAQ_Acquire (max=0, callback=0x434b40
>>> <PacketCallback>, user=0x0) at sfdaq.c:541
>>> #28 0x0000000000437616 in PacketLoop () at snort.c:3268
>>> #29 0x00000000004337c7 in SnortMain (argc=6, argv=0x7fffffffebc0) at
>>> snort.c:921
>>> #30 0x000000000043364f in main (argc=6, argv=0x7fffffffebc0) at
>>> snort.c:817
>>>
>>> Thread 1 (Thread 815a59400 (LWP 100459/snort)):
>>> #0  0x0000000801faa40c in nanosleep () from /lib/libc.so.7
>>> #1  0x0000000801f15a58 in sleep () from /lib/libc.so.7
>>> #2  0x0000000801ca8078 in sleep () from /lib/libthr.so.3
>>> #3  0x000000000043b215 in ReloadConfigThread (data=0x0) at snort.c:5695
>>> #4  0x0000000801ca5dc4 in pthread_getprio () from /lib/libthr.so.3
>>> #5  0x0000000000000000 in ?? ()
>>> #0  0x0000000801f2364c in thr_kill () from /lib/libc.so.7
>>> The program is running.  Quit anyway (and detach it)? (y or n) 
>>> Detaching
>>> from program: /usr/local/bin/snort, process 11057
>>>
>>>
>>>
>>>
>>> As gdb detached from snort, I got the signal 6 in my syslog:
>>> 2015-06-04 17:51:53 +02:00 foobar kernel: pid 11057 (snort), uid 100:
>>> exited on signal 6
>>>
>>>
>>>
>>>
>>>
>>> So, this time we got a signal 6 but during this sensor's 14 hour uptime
>>> we've seen:
>>> pid 1199 (snort), uid 100: exited on signal 10
>>> pid 4503 (snort), uid 100: exited on signal 10
>>> pid 5908 (snort), uid 100: exited on signal 10
>>> pid 11057 (snort), uid 100: exited on signal 6
>>>
>>>
>>>
>>>
>>>
>>> I hope this gdb was helpful.
>>> Let me know if it should be run again.
>>>
>>>
>>>
>>>
>>>
>>> This was all performed on a sensor running:
>>>
>>>    ,,_     -*> Snort! <*-
>>>   o"  )~   Version 2.9.7.3 (Build 217)
>>>    ''''    By Martin Roesch & The Snort Team:
>>> http://www.snort.org/contact#team
>>>            Copyright (C) 2014-2015 Cisco and/or its affiliates. All
>>> rights reserved.
>>>            Copyright (C) 1998-2013 Sourcefire, Inc., et al.
>>>            Using libpcap version 1.4.0
>>>            Using PCRE version: 8.37 2015-04-28
>>>            Using ZLIB version: 1.2.8
>>>
>>>
>>> daq-2.0.5
>>>
>>> FreeBSD 9.3-RELEASE-p13
>>>
>>>
>>> /Elof
>>>
>>>
>>>
>>>
>>>
>>> On Thu, 4 Jun 2015, Hui Cao (huica) wrote:
>>>
>>>> That¹s cool. All looks good to me. No need to do more things...
>>>>
>>>> Best,
>>>> Hui
>>>>
>>>> On 6/4/15, 11:35 AM, "elof at ...969..." <elof at ...969...> wrote:
>>>>
>>>>>
>>>>> Hi Hui.
>>>>>
>>>>> That much I know. It is the debugging steps I'm curious about.
>>>>>
>>>>> (I think you forgot one important first command: continue )
>>>>>
>>>>>
>>>>> Is this a good start:
>>>>>
>>>>> gdb /path/to/snort 1222
>>>>> (gdb) set logging file gdb-snort.txt
>>>>> (gdb) set logging on
>>>>> (gdb) continue
>>>>>
>>>>> <wait for it to crash>
>>>>>
>>>>> (gdb) backtrace full
>>>>> (gdb) info registers
>>>>> (gdb) x/16i $pc
>>>>> (gdb) thread apply all backtrace
>>>>> (gdb) quit
>>>>>
>>>>> Email the report.
>>>>>
>>>>>
>>>>> Should I prepare more stuff before the 'continue'?
>>>>> Like "handle SIG33 pass nostop noprint" or something?
>>>>>
>>>>> /Elof
>>>>>
>>>>>
>>>>> On Thu, 4 Jun 2015, Hui Cao (huica) wrote:
>>>>>
>>>>>> Try
>>>>>>
>>>>>> Assume snort pid is 1222
>>>>>>
>>>>>> gdb /path/to/snort 1222
>>>>>>
>>>>>> Best,
>>>>>> Hui.
>>>>>> On 6/4/15, 10:37 AM, "elof at ...969..." <elof at ...969...> wrote:
>>>>>>
>>>>>>>
>>>>>>> An update:
>>>>>>>
>>>>>>> On a sensor where snort crashed with signal 6 three times, I
>>>>>>> downgraded
>>>>>>> daq to 2.0.4_1 and rebooted the machine to rule out if the problem
>>>>>>> seem
>>>>>>> to
>>>>>>> be in 'snort' or 'daq'.
>>>>>>>
>>>>>>> With snort 2.9.7.3 and daq 2.0.4_1 I got signal 6 again.
>>>>>>>
>>>>>>>
>>>>>>> This make me believe that there's something wrong in snort 2.9.7.3
>>>>>>> and
>>>>>>> not in daq 2.0.5.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On this sensor I have now done the opposite, upgraded daq to 2.0.5
>>>>>>> and
>>>>>>> downgraded snort to 2.9.7.2 to see if I get any more signal 6.
>>>>>>>
>>>>>>> On another sensor, I'm running 2.9.7.3 (compiled with debug) and 
>>>>>>> daq
>>>>>>> 2.0.5
>>>>>>> without chroot and uid/gid change, i.e. running as root, in 
>>>>>>> order to
>>>>>>> create a core file, if the problem happen again.
>>>>>>> (if it doesn't happen on this sensor, I guess the problem lies
>>>>>>> somewhere
>>>>>>> in the chrooting code in snort. I know it has been updated between
>>>>>>> 2.9.7.2
>>>>>>> and 2.9.7.3)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Russ C also wrote:
>>>>>>>> Elof - since this is happening frequently, you could try attaching
>>>>>>>> the
>>>>>>>> debugger to one of your Snort processes and wait wait for 
>>>>>>>> segfault.
>>>>>>>
>>>>>>> I know too little about debugging. :-/ Can you give me instructions
>>>>>>> or
>>>>>>> point me to a guide that describes the steps I should take?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> /Elof
>>>>>>>
>>>>>>>
>>>>>>> On Thu, 4 Jun 2015, elof at ...969... wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> Five different sensors have now had bus errors (signal 10),
>>>>>>>> segmentation
>>>>>>>> faults (signal 11) and even signal 6 (SIGABRT).
>>>>>>>>
>>>>>>>> My snort config uses both chroot and dropping user privileges, so
>>>>>>>> even
>>>>>>>> if
>>>>>>>> I start out as root with ulimit unlimited, this doesn't seem to be
>>>>>>>> in
>>>>>>>> effect
>>>>>>>> after the chroot/uid-change.
>>>>>>>>
>>>>>>>> So currently I have no core-file to debug. :-/
>>>>>>>>
>>>>>>>> Anyone know how to set the ulimits for a chrooted and
>>>>>>>> uid/gid-changed
>>>>>>>> process in FreeBSD?
>>>>>>>>
>>>>>>>> /Elof
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, 4 Jun 2015, elof at ...969... wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Hi Hui!
>>>>>>>>>
>>>>>>>>> Yes, the dynamic engine/preproc files are updated as well.
>>>>>>>>>
>>>>>>>>> Last night the problem reocurred, so this seem to be 
>>>>>>>>> reproduceable.
>>>>>>>>> Good.
>>>>>>>>> Then there's a good chance this problem can be sorted out.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> A few minutes ago a signal 10 happened on another sensor (running
>>>>>>>>> FreeBSD 10.1 amd64), so the problem must be in DAQ 2.0.5 or in
>>>>>>>>> Snort
>>>>>>>>> 2.9.7.3 and not in the hardware nor in FreeBSD.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I will compile a debug-snort and try to generate core files.
>>>>>>>>> I'll let you know the outcome next week.
>>>>>>>>>
>>>>>>>>> /Elof
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, 3 Jun 2015, Hui cao wrote:
>>>>>>>>>
>>>>>>>>>> Hi Elof,
>>>>>>>>>>
>>>>>>>>>> Are snort and snort dynamic preprocessors are in sync?
>>>>>>>>>>
>>>>>>>>>> If so, can you help us get a backtrace from the crush? You need
>>>>>>>>>> 1)  build snort with ./configure --enable-debug
>>>>>>>>>> 2)  allowing core dump (ulimit -c unlimited)
>>>>>>>>>> 3) run the snort
>>>>>>>>>> 4) use "gdb snort core_file " and them type "bt" in the gdb
>>>>>>>>>> command
>>>>>>>>>> line
>>>>>>>>>>
>>>>>>>>>> Best,
>>>>>>>>>> Hui.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 06/03/2015 05:51 AM, elof at ...969... wrote:
>>>>>>>>>>> Hi all!
>>>>>>>>>>>
>>>>>>>>>>> This is just a report to inform that after I updated snort and
>>>>>>>>>>> DAQ
>>>>>>>>>>> to the
>>>>>>>>>>> latest versions, one of my sensors started throwing signal 10
>>>>>>>>>>> (bus
>>>>>>>>>>> error)
>>>>>>>>>>> and signal 11 (segmentation fault).
>>>>>>>>>>>
>>>>>>>>>>> # uptime
>>>>>>>>>>> 11:32AM  up 1 day,  9:48, 1 user, load averages: 0.36, 0.37, 
>>>>>>>>>>> 0.38
>>>>>>>>>>> # dmesg | grep snort
>>>>>>>>>>> pid 1183 (snort), uid 100: exited on signal 11
>>>>>>>>>>> pid 16920 (snort), uid 100: exited on signal 11
>>>>>>>>>>> pid 17502 (snort), uid 100: exited on signal 11
>>>>>>>>>>> pid 18862 (snort), uid 100: exited on signal 11
>>>>>>>>>>> pid 20223 (snort), uid 100: exited on signal 11
>>>>>>>>>>> pid 20927 (snort), uid 100: exited on signal 11
>>>>>>>>>>> pid 1193 (snort), uid 100: exited on signal 11
>>>>>>>>>>> pid 2447 (snort), uid 100: exited on signal 11
>>>>>>>>>>> pid 3811 (snort), uid 100: exited on signal 10
>>>>>>>>>>> pid 7881 (snort), uid 100: exited on signal 11
>>>>>>>>>>> pid 9252 (snort), uid 100: exited on signal 10
>>>>>>>>>>> pid 25593 (snort), uid 100: exited on signal 11
>>>>>>>>>>> pid 26627 (snort), uid 100: exited on signal 11
>>>>>>>>>>> pid 56658 (snort), uid 100: exited on signal 11
>>>>>>>>>>> pid 57237 (snort), uid 100: exited on signal 10
>>>>>>>>>>> pid 58595 (snort), uid 100: exited on signal 11
>>>>>>>>>>> pid 68639 (snort), uid 100: exited on signal 11
>>>>>>>>>>> pid 70008 (snort), uid 100: exited on signal 11
>>>>>>>>>>> pid 71361 (snort), uid 100: exited on signal 10
>>>>>>>>>>> pid 72725 (snort), uid 100: exited on signal 11
>>>>>>>>>>>
>>>>>>>>>>> 20 crashes in a day...
>>>>>>>>>>> A reboot didn't help.
>>>>>>>>>>>
>>>>>>>>>>> This sensor has never behaved like this during its lifetime (1
>>>>>>>>>>> year).
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> FreeBSD 9.3 amd64
>>>>>>>>>>>
>>>>>>>>>>>      ,,_     -*> Snort! <*-
>>>>>>>>>>>     o"  )~   Version 2.9.7.3 (Build 217)
>>>>>>>>>>>      ''''    By Martin Roesch & The Snort Team:
>>>>>>>>>>> http://www.snort.org/contact#team
>>>>>>>>>>>              Copyright (C) 2014-2015 Cisco and/or its 
>>>>>>>>>>> affiliates.
>>>>>>>>>>> All rights
>>>>>>>>>>> reserved.
>>>>>>>>>>>              Copyright (C) 1998-2013 Sourcefire, Inc., et al.
>>>>>>>>>>>              Using libpcap version 1.4.0
>>>>>>>>>>>              Using PCRE version: 8.37 2015-04-28
>>>>>>>>>>>              Using ZLIB version: 1.2.8
>>>>>>>>>>>
>>>>>>>>>>> daq-2.0.5
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Bus errors are quite unusual in general, so I'll keep 
>>>>>>>>>>> looking at
>>>>>>>>>>> this,
>>>>>>>>>>> trying to see if it is e.g. paging errors.
>>>>>>>>>>> It doesn't look like it though:
>>>>>>>>>>> # swapinfo
>>>>>>>>>>> Device          1K-blocks     Used    Avail Capacity
>>>>>>>>>>> /dev/mirror/swap   4194300        0 4194300     0%
>>>>>>>>>>>
>>>>>>>>>>> The machine doesn't seem to be overheated either:
>>>>>>>>>>> System Temp:    30 degrees C
>>>>>>>>>>> Peripheral Temp: 40 degrees C
>>>>>>>>>>> CPU Temp: Low
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> If you need me to do something special to debug this 
>>>>>>>>>>> further, let
>>>>>>>>>>> me
>>>>>>>>>>> know.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> PS. It is only one sensor, out of 20, that behaves like 
>>>>>>>>>>> this. So
>>>>>>>>>>> perhaps
>>>>>>>>>>> it is something in the mirrored traffic that make DAQ or snort
>>>>>>>>>>> point
>>>>>>>>>>> at
>>>>>>>>>>> illegal memory addresses and crash.
>>>>>>>>>>> Or this particular machine is having hardware issues. 
>>>>>>>>>>> However, it
>>>>>>>>>>> is
>>>>>>>>>>> strange that those hw-issues should suddenly start right 
>>>>>>>>>>> after I
>>>>>>>>>>> updated
>>>>>>>>>>> the software on the machine...
>>>>>>>>>>>
>>>>>>>>>>> When I write this, the current snort process has been alive 
>>>>>>>>>>> for 5
>>>>>>>>>>> hours.
>>>>>>>>>>> It's going to be interesting to see if the traffic tonight will
>>>>>>>>>>> cause it
>>>>>>>>>>> to crash many times again.
>>>>>>>>>>>
>>>>>>>>>>> /Elof
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> ------------------------------------------------------------------ 
>>>>>>>>>>>
>>>>>>>>>>> -- 
>>>>>>>>>>> -- 
>>>>>>>>>>> --------
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Snort-devel mailing list
>>>>>>>>>>> Snort-devel at lists.sourceforge.net
>>>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/snort-devel
>>>>>>>>>>> Archive:
>>>>>>>>>>>
>>>>>>>>>>> http://sourceforge.net/mailarchive/forum.php?forum_name=snort-deve 
>>>>>>>>>>>
>>>>>>>>>>> l
>>>>>>>>>>>
>>>>>>>>>>> Please visit http://blog.snort.org for the latest news about
>>>>>>>>>>> Snort!
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ------------------------------------------------------------------- 
>>>>>>>>>>
>>>>>>>>>> -- 
>>>>>>>>>> -- 
>>>>>>>>>> -------
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Snort-devel mailing list
>>>>>>>>>> Snort-devel at lists.sourceforge.net
>>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/snort-devel
>>>>>>>>>> Archive:
>>>>>>>>>>
>>>>>>>>>> http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel 
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Please visit http://blog.snort.org for the latest news about
>>>>>>>>>> Snort!
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> -------------------------------------------------------------------- 
>>>>>>>>>
>>>>>>>>> -- 
>>>>>>>>> -- 
>>>>>>>>> ------
>>>>>>>>> _______________________________________________
>>>>>>>>> Snort-devel mailing list
>>>>>>>>> Snort-devel at lists.sourceforge.net
>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/snort-devel
>>>>>>>>> Archive:
>>>>>>>>> http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel 
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Please visit http://blog.snort.org for the latest news about 
>>>>>>>>> Snort!
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --------------------------------------------------------------------- 
>>>>>>>>
>>>>>>>> -- 
>>>>>>>> -- 
>>>>>>>> -----
>>>>>>>> _______________________________________________
>>>>>>>> Snort-devel mailing list
>>>>>>>> Snort-devel at lists.sourceforge.net
>>>>>>>> https://lists.sourceforge.net/lists/listinfo/snort-devel
>>>>>>>> Archive:
>>>>>>>> http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel 
>>>>>>>>
>>>>>>>>
>>>>>>>> Please visit http://blog.snort.org for the latest news about 
>>>>>>>> Snort!
>>>>>>>>
>>>>>>
>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------------- 
>>>>
>>>> -----
>>>> _______________________________________________
>>>> Snort-devel mailing list
>>>> Snort-devel at lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/snort-devel
>>>> Archive:
>>>> http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel
>>>>
>>>> Please visit http://blog.snort.org for the latest news about Snort!
>>
>>





More information about the Snort-devel mailing list