[Snort-devel] Segfault, massive memory usage in 1.8.1beta3 build 47

Martin Roesch roesch at ...402...
Fri Jul 20 14:38:59 EDT 2001


It's probably the XML code that's leaking, stream4 and frag2 have hard
memory caps that are enforced aggressively.  Try it without and let's
see what happens.

     -Marty

Seth Leger wrote:
> 
> I got this segfault after running for about 2 hours under constant
> Nessus strain:
> 
> =======================
> 
> #0  0x80744fb in Frag2CompareFunc (ItemPtr=0xbfffebf0,
> NodePtr=0x13873950) at spp_frag2.c:154
> #1  0x80700c6 in qFind (cmp=0x80744f0 <Frag2CompareFunc>,
> FindMe=0xbfffebf0, p=0x13577b30)
>    at ubi_BinTree.c:231
> #2  0x80705ba in ubi_btFind (RootPtr=0x80998e4, FindMe=0xbfffebf0) at
> ubi_BinTree.c:861
> #3  0x807099c in ubi_sptFind (RootPtr=0x80998e4, FindMe=0xbfffebf0) at
> ubi_SplayTree.c:458
> #4  0x8074bcd in GetFragTracker (p=0xbfffecb0) at spp_frag2.c:474
> #5  0x8074a78 in Frag2Defrag (p=0xbfffecb0) at spp_frag2.c:410
> #6  0x8054f32 in Preprocess (p=0xbfffecb0) at rules.c:3427
> #7  0x804a80b in ProcessPacket (user=0x0, pkthdr=0x80d6378,
> pkt=0x80d6478 "") at snort.c:512
> #8  0x8074ee3 in RebuildFrag (ft=0x13577b30, p=0xbffff1d0) at
> spp_frag2.c:633
> #9  0x8074b08 in Frag2Defrag (p=0xbffff1d0) at spp_frag2.c:433
> #10 0x8054f32 in Preprocess (p=0xbffff1d0) at rules.c:3427
> #11 0x804a80b in ProcessPacket (user=0x0, pkthdr=0xbffff690,
> pkt=0x8312c62 "") at snort.c:512
> #12 0x807762c in pcap_read ()
> #13 0x8077c1b in pcap_loop ()
> #14 0x804bbb0 in InterfaceThread (arg=0x0) at snort.c:1441
> #15 0x804a6db in main (argc=1, argv=0xbffff834) at snort.c:445
> #16 0x40077f31 in __libc_start_main (main=0x804a07c <main>, argc=1,
> ubp_av=0xbffff834,
>    init=0x80497b8 <_init>, fini=0x807ea1c <_fini>, rtld_fini=0x4000e274
> <_dl_fini>, stack_end=0xbffff82c)
>    at ../sysdeps/generic/libc-start.c:129
> (gdb) list spp_frag2.c:154
> 149         FragTracker *iFt;
> 150
> 151         nFt = (FragTracker *) NodePtr;
> 152         iFt = (FragTracker *) ItemPtr;
> 153
> 154         DebugMessage(DEBUG_FLOW,"NodePtr: sip: 0x%X  dip: 0x%X  ip:
> 0x%X  "
> 155                      "proto: 0x%X\n", nFt->sip, nFt->dip, nFt->id,
> nFt->protocol);
> 156         DebugMessage(DEBUG_FLOW,"ItemPtr: sip: 0x%X  dip: 0x%X  ip:
> 0x%X  "
> 157                      "proto: 0x%X\n", iFt->sip, iFt->dip, iFt->id,
> iFt->protocol);
> 158
> (gdb) print nFt
> $1 = (FragTracker *) 0x13873950
> (gdb) print nFt->sip
> Cannot access memory at address 0x13873960
> (gdb) print nFt->dip
> Cannot access memory at address 0x13873964
> (gdb) print nFt->id
> Cannot access memory at address 0x13873968
> (gdb) print nFt->protocol
> Cannot access memory at address 0x1387396a
> 
> =======================
> 
> Another note: I've noticed incredible memory usage for Snort over the
> last couple of runs. During the run that eventually ended in the seg
> fault above, after approximately 2 hours Snort was using upwards of 145
> MB of RAM.
> 
> Historically, I've noticed it leveling off at about 45 MB with my
> configuration (all current preprocessors on and XML logging enabled). In
> a run that I ran over night last night with the beta2 code, Snort grew
> to 445 MB of RAM, exhausting the machine's resources. Here's the info
> from that scan:
> 
> Snort analyzed 27638371 out of 27638371 packets, dropping 0(0.000%) packets
> 
> Breakdown by protocol:                Action Stats:
>    TCP: 26398288   (95.513%)         ALERTS: 82490
>    UDP: 753654     (2.727%)          LOGGED: 80793
>   ICMP: 56867      (0.206%)          PASSED: 0
>    ARP: 49962      (0.181%)
>   IPv6: 0          (0.000%)
>    IPX: 0          (0.000%)
>   OTHER: 10333      (0.037%)
> DISCARD: 0          (0.000%)
> ===============================================================================
> Fragmentation Stats:
> Fragmented IP Packets: 1007496    (3.645%)
>   Rebuilt IP Packets: 134481
>   Frag elements used: 0
> Discarded(incomplete): 0
>   Discarded(timeout): 119106
> ===============================================================================
> TCP Stream Reassembly Stats:
>   TCP Packets Used:      26398247   (95.513%)
>   Reconstructed Packets: 18073      (0.065%)
>   Streams Reconstructed: 13416940
> ===============================================================================
> Snort received signal 2, exiting
> 
> These alerts were achieved by intermittent Nessus scans (while I was at
> work) and then a constant stream of scripted nmap scans of the sensor
> overnight, along with pretty much constant output produced by the
> portscan plugin (since I had an NFS server outside of the exclude list).
> Incidentally, the portmap.log produced by the portscan plugin reached
> 800 MB overnight.
> 
> I have tested the system before with similar tests and have never seen
> the RAM climb as high as it did yesterday. I'm not sure which part of
> the process is introducing this load (stream4, frag2, or the XML
> plugin), but if anyone has some advice on how to reduce it some by
> changing the plugin settings, I'd really appreciate it.
> 
> Another note: the RAM usage seems to be proportional to the output of
> the XML plugin, not the size of the portmap log. Therefore, I assume
> that the problem lies in a method that is executed during each alert.
> Here's a copy of my config for interested onlookers:
> 
> var HOME_NET any
> var INTERNAL $HOME_NET
> var EXTERNAL_NET any
> var EXTERNAL $EXTERNAL_NET
> var SMTP $HOME_NET
> var HTTP_SERVERS $HOME_NET
> var SQL_SERVERS $HOME_NET
> var DNS_SERVERS $HOME_NET
> 
> #preprocessor defrag
> preprocessor frag2
> #preprocessor stream2: timeout 10, ports 21 23 80 110 143, maxbytes 16384
> preprocessor stream4
> preprocessor stream4_reassemble
> #preprocessor http_decode: 80 -unicode -cginull
> preprocessor unidecode: 80
> preprocessor rpc_decode: 111
> preprocessor bo: -nobrute
> preprocessor telnet_decode
> preprocessor portscan: $HOME_NET 4 3 portscan.log
> #preprocessor portscan-ignorehosts: $DNS_SERVERS
> 
> # Flat file logging:
> output xml: alert, file=/var/log/snort-xml
> 
> include classification.config
> 
> include master.rules
> 
> Thanks for the updates, it's definitely looking more stable than 1.8
> was! Great work,
> 
> Seth
> soleger at ...511...
> 
> _______________________________________________
> Snort-devel mailing list
> Snort-devel at lists.sourceforge.net
> http://lists.sourceforge.net/lists/listinfo/snort-devel

--
Martin Roesch
roesch at ...402...
http://www.sourcefire.com - http://www.snort.org




More information about the Snort-devel mailing list