[Snort-devel] performance: CheckDstIp
mark at ...1209...
Wed Mar 20 08:32:13 EST 2002
Chris Green / Christian Mock wrote:
>> while profiling snort, I found an interesting fact today: CheckDstIp is
>> number one in the profiling chart.
>Yuppers and it's called a lot! One of the things we're working on is
>optimizing out the detection enigne so that we don't have to call it
>nearly as often.
I've been looking at the source code, and this explains why
having $HOME_NET set to anything other than 'any' absolutely destroys
snorts performonce (I think I'm right in saying that the CheckDstIP
function doesn't get added to the list of check funcs for an RTN
if the dst ip is any). I'd just done the profiling and come up
with an even worse result (62% of cpu in DST IP)
Anyway, being an ISP we have 9 CIDR blocks allocated to us by RIPE
totalling about a /12, which means that the network section snort
is monitoring could potentially be anywhere within those nine blocks,
so we can't aggregate the blocks. There are approx 1000 ip + tcp
rules which evaluate to $HOME_NET given a default configuation.
(snort runs the proto list first and if no match is found runs the
So, 1000 rules * 9 checks => Snort checks the IP address 9000
times for each tcp packet that fails to generate an alert instead
of the minimum 9 times. => CPU hits roof and stays there. ;*(
Ok, so what's the fix? Assuming I'm not teaching Grandma to suck
eggs (please excuse me if I am) the basic premise should be to ignore
a packet that doesn't match ASAP. So would it be better to compile
a list of rules that can be rejected based on a given criteria for
the key criteria:
$EXTERNAL_NET = any;
$HOME_NET = [abcd/x]
All other $ default to $HOME_NET
Total default rules : 1259
Criteria. #exclude checks # No. Rules excluded Rules Excluded/check
Proto 3 checks 1228 409
Src IP 4 checks 77 19
Dst IP 1 1095 1095
Src port 4 179 44
Dst port 102 929 9
Total Checks 114
The above figures for a default ruleset mean we should check dst_ip
followed by proto followed by src port *IF* we work on an exclusion
Now in practice how many checks need to be performed depends on the
traffic mix, however it can be seen that the current mechanism is
not v.efficient since we perform 1000 identical checks alone just
to determine if the DST_IP matches if we set $HOME_NET.
(NB. If we set $EXTERNAL_NET the src_ip rules excluded / check in the
table above rises to 233).
Currently we effectively have a one dimensional array of proto to
optimize which checks need to be performed.
So instead of
if (proto!='ip') match = check_proto_list(proto);
if (!match) check_proto_list(ip) /* ip proto is proto equiv of any */
->list of rules
->list of rules
So for each packet, we run through proto_array and say which match.
(tcp and ip for a tcp packet).
If we then run thorugh each dst_ip block and say which rules match.
(any is just ip_addr 0.0.0.0 and mask 0.0.0.0).
The intersection (back to school and venn diagrams) of those two sets
is now the set of rules we need to check.
Abstracting this to multiple criteria, if we take criteria 1
( lets assume dst_ip for example) take an array rules[n], for
each rule in 'list of rules' for an ip dst that matches set rules[n]
to 1. If we want to be clever and we are using bit fields for the
list of rules we could 'pre or' the list of rules for each match
with the 'any' list of rules so that if we only match on one element
of a criteria no further processing is required for that criteria.
Do this for each criteria, so if we use the 5 criteria above that
gives us five arrays which we logically 'and' together to give us the
resulting set of rules to check.
So if we just checked dst_ip this way for the example where we have
$HOME_NET net set we have excluded all but 90 rules for a packet
which doesn't match our $HOME_NET.
Now whilst I appreciate this is highly config dependent, it should
be possible to have a mechanism to say which checks are performed
this way and which the convetional way, allowing people to tune the
checks to the type of ruleset they use and traffic they see.
I thought about doing this with dst_ip in the current code, but
linked lists are not conducive to this approach so I'd have to indirect
through another array and it all got horribly messy and did my head in.
Anyway if you've read this far then you're probably bored, so other
than to feel free to tell me I'm talking out of my bottom I'll stop
Mark Vevers. mark at ...1121... / mvevers at ...1186...
Internet Backbone Engineering Team
Internet for Learning, Research Machines Plc
More information about the Snort-devel