[Snort-users] Performance again

Lawrence Reed Lawrence.Reed at ...1444...
Tue Dec 23 10:26:04 EST 2003


>
> Certainly much more of a factor than the number of sessions or the 
> size of packets is going to be what RTNs (rule tree nodes) the packets 
> need to be processed against.
>
> Snort does a "first pass" elimination of rules to run against a packet 
> based on the sourceIP, dest IP, protocol, source port and dest port 
> specified in the rules. This "first pass" is very much faster than the 
> content searching.
>
>  A packet that winds up matching a combination of RTNs that has a long 
> list of rules with lots of content searches is going to be a heavy hit 
> in processing time. On the other hand a packet which winds up matching 
> none of the combinations is going to have very little processing time.
>
> Thus really the type of packets matters quite a lot, and which packet 
> types are 'bad' depends a lot on how your ruleset is constructed.. if 
> every packet to a http server has to be evaluated against 800 content 
> rules, that hurts. If HTTP_SERVERS is set to "any" and EXTERNAL_NET is 
> set to "any" that hurts even more.
>
>
>
Has anyone done any work to try and quantify this?  Or developed 
hooks/patches to try and measure this?  I have a scenario that I cannot 
explain.  I have the following perf output:

20031223 
17:28:24,0.000,59.3,0.2,10.8,855,70.66,71.7,63.6,75.1,57.2,77994,100185,786.4,0,0,0.5,1.0,0.0,0.5,0,0,68.3,5.8,26.0
20031223 
17:28:34,0.000,57.5,0.1,10.6,824,72.70,74.3,68.3,81.3,62.3,78179,100185,744.7,0,0,0.4,0.8,0.0,0.4,0,0,67.3,6.8,25.9
20031223 
17:28:44,0.000,60.8,0.3,10.5,848,75.36,71.5,64.3,78.5,274.0,76246,100185,627.2,0,1,0.0,0.0,0.0,0.0,0,0,70.1,6.3,23.6
20031223 
17:28:54,0.000,65.3,0.1,12.4,750,77.32,74.1,63.7,78.6,47.7,76557,100185,553.4,0,0,0.0,0.0,0.0,0.0,0,0,62.2,7.3,30.5
20031223 
17:29:04,0.000,61.6,0.5,12.3,735,73.67,85.3,74.6,84.8,61.0,76815,100185,638.1,0,0,0.0,0.0,0.0,0.0,0,0,64.0,7.1,28.9
20031223 
17:29:14,0.000,51.9,0.2,9.6,810,74.58,71.4,63.4,73.4,58.0,76967,100185,613.6,0,0,0.0,0.0,0.0,0.0,0,0,51.8,9.0,39.2
20031223 
17:29:24,0.000,67.4,0.1,13.4,716,75.99,69.7,61.8,71.9,53.0,77144,100185,543.9,0,0,0.0,0.0,0.0,0.0,0,0,63.4,8.2,28.4
20031223 
17:29:34,0.000,59.5,0.1,11.9,735,73.08,79.8,72.9,80.9,59.9,77362,100185,611.6,0,0,0.0,0.0,0.0,0.0,0,0,56.6,7.1,36.3
20031223 
17:29:44,0.000,81.0,0.3,17.7,643,76.91,79.8,72.7,81.7,55.8,77617,100185,585.4,0,0,0.0,0.0,0.0,0.0,0,0,64.5,7.8,27.7
20031223 
17:29:54,0.000,72.4,0.5,14.1,740,75.62,78.7,72.4,83.1,56.4,77890,100185,622.2,0,0,0.0,0.0,0.0,0.0,0,0,64.0,6.9,29.1
20031223 
17:30:04,0.000,71.9,0.3,13.0,843,71.17,86.3,76.3,89.5,57.4,78196,100185,764.0,0,0,0.0,0.0,0.0,0.0,0,0,66.0,7.9,26.1
20031223 
17:30:14,0.000,80.9,0.1,15.4,805,68.35,112.8,89.3,104.9,65.5,78583,100185,950.0,0,0,0.0,0.0,0.0,0.0,0,0,75.5,9.0,15.6
20031223 
17:30:24,0.000,78.1,0.2,16.6,679,72.52,80.5,73.7,83.7,58.0,78851,100185,691.6,0,0,0.0,0.0,0.0,0.0,0,0,70.6,7.9,21.5
20031223 
17:30:34,0.000,73.4,0.5,14.6,745,71.33,65.6,59.9,67.2,46.5,79054,100185,724.4,0,0,0.2,0.4,1.7,0.2,18,0,77.4,8.3,14.3
20031223 
17:30:44,13.205,65.9,0.4,12.7,750,77.93,56.6,49.7,65.5,266.0,76928,100185,498.0,0,1,0.2,0.4,0.0,0.2,0,0,90.8,9.1,0.0
20031223 
17:30:54,6.039,70.9,0.2,13.2,771,77.08,73.7,68.4,86.0,52.2,77249,100185,579.7,0,0,0.0,0.0,0.0,0.0,0,0,91.5,8.4,0.1
20031223 
17:31:04,0.000,86.0,0.4,15.9,773,76.25,76.9,71.1,84.6,57.6,77526,100185,724.8,0,0,0.0,0.2,0.0,0.0,0,0,91.1,9.0,-0.1
20031223 
17:31:14,0.000,85.2,0.1,15.1,808,75.98,73.7,63.5,79.1,51.4,77790,100185,663.5,0,0,0.0,0.1,0.0,0.0,0,0,78.7,10.8,10.5
20031223 
17:31:24,0.000,74.9,0.1,14.1,774,74.38,81.4,71.5,84.8,59.0,78053,100185,695.2,0,0,0.0,0.2,0.0,0.0,0,0,64.1,7.0,28.9
20031223 
17:31:34,0.000,59.6,0.1,10.9,806,72.36,79.2,71.8,82.5,63.1,78254,100185,662.3,0,0,0.0,0.1,0.0,0.0,0,0,59.5,5.9,34.7
20031223 
17:31:44,0.000,58.2,0.4,11.3,769,72.73,72.7,64.9,73.4,52.0,78470,100185,656.3,0,0,0.0,0.0,0.0,0.0,0,0,63.8,7.0,29.2
20031223 
17:31:54,0.000,56.8,0.1,11.2,746,73.51,69.6,61.9,84.2,52.9,78764,100185,591.7,0,0,0.0,0.0,0.0,0.0,0,0,56.6,7.3,36.0


Notice the packet drop rate is 0.000 most of the time with occasional 
spikes to a very high numbers.   I have not been able to track this down 
to a particular packet/session/time/mem limit/traffic rate.  

You might also notice the my max-sessions is very high ( 100185 ).  I 
have increased the timeout and memcap parameters as follows:

preprocessor stream4: disable_evasion_alerts, keepstats binary, timeout 
3600, memcap 178000000

Initially this caused packet loss problems when the memcap was reached 
and a stream4 fault occurred.  The fault logic only deleted 5 streams, 
obviously that is not enough when I have 100,000 streams in memory.  I 
changed the code to delete 10% of max-streams and this packet loss has 
stopped.

I understand that is problem is easily solved with a faster CPU, but 
thought the research would be interesting and fun.  I am currently 
running on a 1.2Ghz P III with 512Mb memory, snort 2.0.5  ( some stream4 
patches ), Phil Wood's pcap with max ring buffer size, and a 100mb fast 
ethernet sniffing interface.

The ruleset I am using is based on the current snort.org rules with many 
modifications.  Most of the mods are to increase the packets logged by 
adding tag options.


>
>
>
>> I am simply trying to understand how is everything working together 
>> as one complex system.
>
>
> Understood, and I'm just trying to explain the parts I understand. 
> Your post had the conception that when a new packet arrived, the one 
> that was being processed got dropped in the middle of some part of 
> snort processing it.. it doesn't. The actual mechanism of drop winds 
> up dropping the oldest packet in the queue that snort hasn't started 
> processing yet.
>
>
-- 

Larry Reed  Lawrence.Reed at ...1444...
NOAA IT Security Office
PGP Public Key:  http://search.keyserver.net:11371/pks/lookup?op=get&search=0x7A998772






More information about the Snort-users mailing list