[Snort-devel] performance: CheckDstIp

Mark Vevers mark at ...1209...
Wed Mar 20 08:32:13 EST 2002


Chris Green / Christian Mock wrote:
>Hi,
>>
>> while profiling snort, I found an interesting fact today: CheckDstIp is 
>> number one in the profiling chart.


>Yuppers and it's called a lot!  One of the things we're working on is
>optimizing out the detection enigne so that we don't have to call it
>nearly as often.

Chris(tian),

I've been looking at the source code, and this explains why
having $HOME_NET set to anything other than 'any' absolutely destroys
snorts performonce (I think I'm right in saying that the CheckDstIP 
function doesn't get added to the list of check funcs for an RTN
if the dst ip is any).  I'd just done the profiling and come up 
with an even worse result (62% of cpu in DST IP)

Anyway, being an ISP we have 9 CIDR blocks allocated to us by RIPE 
totalling about a /12, which means that the network section snort 
is monitoring  could potentially be anywhere within those nine blocks, 
so we can't aggregate the blocks.  There are approx 1000 ip + tcp 
rules which evaluate to $HOME_NET given a default configuation.
(snort runs the proto list first and if no match is found runs the
ip list)

So,  1000 rules * 9 checks => Snort checks the IP address 9000
times for each tcp packet that fails to generate an alert instead 
of the minimum 9 times.  => CPU hits roof and stays there. ;*(

Ok, so what's the fix?  Assuming I'm not teaching Grandma to suck
eggs (please excuse me if I am) the basic premise should be to ignore
a packet that doesn't match ASAP.   So would it be better to compile
a list of rules that can be rejected based on a given criteria for 
the key criteria:

$EXTERNAL_NET = any;
$HOME_NET = [abcd/x]
All other $ default to $HOME_NET

Total default rules : 1259

Criteria.    #exclude checks # No. Rules excluded  Rules Excluded/check    
---------    ------------------------------------------------------- 
Proto        3 checks        1228                  409
Src IP       4 checks        77                    19
Dst IP       1               1095                  1095 
Src port     4               179                   44
Dst port     102             929                   9

Total Checks 114

The above figures for a default ruleset mean we should check dst_ip 
followed by proto followed by src port *IF* we work on an exclusion 
basis.

Now in practice how many checks need to be performed depends on the 
traffic mix, however it can be seen that the current mechanism is
not v.efficient since we perform 1000 identical checks alone just 
to determine if the DST_IP matches if we set $HOME_NET.  
(NB. If we set $EXTERNAL_NET the src_ip rules excluded / check in the
 table above rises to 233).

Currently we effectively have a one dimensional array of proto to
optimize which checks need to be performed.

So instead of 
  if (proto!='ip') match = check_proto_list(proto);

  if (!match) check_proto_list(ip) /*  ip proto is proto equiv of any */

We have 
   dst_ip_array[0-n]->ip_addr
                    ->mask
                    ->match
                    ->list of rules

   proto_array[0-n]->proto
                   ->match
                   ->list of rules

 So for each packet, we run through proto_array and say which match.
 (tcp and ip for a tcp packet). 

 If we then run thorugh each dst_ip block and say which rules match.
 (any is just ip_addr 0.0.0.0 and mask 0.0.0.0).

 The intersection (back to school and venn diagrams) of those two sets 
 is now the set of rules we need to check.

 Abstracting this to multiple criteria, if we take criteria 1 
 ( lets assume dst_ip for example) take an array rules[n], for
 each rule in 'list of rules' for an ip dst that matches set rules[n]
 to 1.  If we want to be clever and we are using bit fields for the
 list of rules we could 'pre or' the list of rules for each match 
 with the 'any' list of rules so that if we only match on one element
 of a criteria no further processing is required for that criteria.
 
 Do this for each criteria, so if we use the 5 criteria above that
 gives us five arrays which we logically 'and' together to give us the
 resulting set of rules to check.  
 
 So if we just checked dst_ip this way for the example where we have 
 $HOME_NET net set we have excluded all but 90 rules for a packet
 which doesn't match our $HOME_NET.

 Now whilst I appreciate this is highly config dependent, it should
 be possible to have a mechanism to say which checks are performed
 this way and which the convetional way, allowing people to tune the
 checks to the type of ruleset they use and traffic they see.

 I thought about doing this with dst_ip in the current code, but 
 linked lists are not conducive to this approach so I'd have to indirect
 through another array and it all got horribly messy and did my head in.
 
 Anyway if you've read this far then you're probably bored, so other 
 than to feel free to tell me I'm talking out of my bottom I'll stop 
 here.
 
Cheers
Mark

-- 
Mark Vevers.    mark at ...1121... / mvevers at ...1186...
Internet Backbone Engineering Team
Internet for Learning, Research Machines Plc 






More information about the Snort-devel mailing list