[Snort-users] SnortSnarf performance

Siddhartha Jain s_i_d_j at ...131...
Wed Mar 28 22:23:28 EST 2001


Isn't DNS lookup turned off by default? And you have to throw a switch to
enable it?
Also, to fill up alerts in Mysql and then use ACID, i used something like :-

output database: alert, mysql, user=root password=xxxx dbname=snort
host=localhost

Now i have ACID working fine, but Snortsnarf is down because the 'alert'
file is no longer getting alerts. So how can i send alerts to a database as
well as flat file in the log directory?

Siddhartha

----- Original Message -----
From: "Tony Lill" <ajlill at ...1676...>
To: "Siddhartha Jain" <s_i_d_j at ...131...>
Cc: <snort-users at lists.sourceforge.net>
Sent: Wednesday, March 28, 2001 12:55 AM
Subject: Re: [Snort-users] SnortSnarf performance


> There are three big problems that make that version of snortsnarf
> slow:
> 1) DNS lookups - a lot of alerts are generated by sites without
> addr to name mappings. Each of these failures causes a 90s stall,
> although I presume you have these turned off.
> 2) Time comparisons - the timestamps on alerts are compared field by
> field instead of converted to a time() value. This cause a
> surprisingly large slowdown in sorting lists (and is just broken when
> the year rolls over).
> 3) Memory - if the footprint of snortsnarf forces you to swap, you're
> doomed.
>
> The following patch addresses the first two issues. I dropped a 1 hour
> run to 2 minutes. The savings were 45 min from the dns and 15 minutes
> from the timestamps. For the third, 128Mb of memory was $89 CDN for my pc,
> so it wasn't worth the effort to look at. Sorry about your sun. All I
> can suggest is to look for data that can be discarded after it is
> written out.
>
> Linux-i686-2.4.1 $ cvs patch -r SNORTSNARF
network/monitoring/snort/SnortSnarf
> cvs server: Diffing network/monitoring/snort/SnortSnarf
> Index: network/monitoring/snort/SnortSnarf/snortsnarf.pl
> diff -c network/monitoring/snort/SnortSnarf/snortsnarf.pl:1.1.1.2
network/monitoring/snort/SnortSnarf/snortsnarf.pl:1.5
> *** network/monitoring/snort/SnortSnarf/snortsnarf.pl:1.1.1.2 Fri Jan  5
20:39:59 2001
> --- network/monitoring/snort/SnortSnarf/snortsnarf.pl Sun Jan  7 01:40:13
2001
> ***************
> *** 142,147 ****
> --- 142,152 ----
>   # -refresh=<secs> add a refresh tag to every HTML page generated,
causing
>   # browsers to refresh the page ever <secs> seconds.
>
> + # -dnscache file file is a file to cache ip to hostname
> + # translations. The file contains an Ip address, a space, and text to
> + # return. If gethostbyname fails, store the IP address as the text. A
> + # clever person could write a post-processor to replace this by
> + # something clever from whois
>
>
>   # Snortsnarf is believed to work with Perl 5 on the following OS's
> ***************
> *** 189,195 ****
>   # Kudos to Steve Northcutt who has taught us all what an intrusion
>   # detection analyst should be.
>
> ! # Version control info: $Id: snortsnarf.pl,v 1.1.1.2 2001/01/06 01:39:59
ajlill Exp $
>
>   $script = "<a href=\"http://www.silicondefense.com/snortsnarf/\">".
>   "SnortSnarf</a>";
> --- 194,200 ----
>   # Kudos to Steve Northcutt who has taught us all what an intrusion
>   # detection analyst should be.
>
> ! # Version control info: $Id: snortsnarf.pl,v 1.5 2001/01/07 06:40:13
ajlill Exp $
>
>   $script = "<a href=\"http://www.silicondefense.com/snortsnarf/\">".
>   "SnortSnarf</a>";
> ***************
> *** 328,333 ****
> --- 333,345 ----
>   $logo_url= "../$logo_filename";
>   &output_anom_sect();
>
> + if( $dnscache ) {
> +   open( DNSCACHE, ">$dnscache" ) || die "Can't open $dnscache for
writing: $!\n";
> +   foreach $ip ( keys %dnscacheout ) {
> +     print DNSCACHE "$ip $dnscache{$ip}\n";
> +   }
> +   close DNSCACHE;
> + }
>
>
############################################################################
##
>
> ***************
> *** 454,459 ****
> --- 466,483 ----
>

>   $refreshsecs = $arg;
>   }
> + elsif($arg =~ /-dnscache/)
> + {
> +   $dnscache = shift(@ARGV);
> +   if( open(DNSCACHE,$dnscache) ) {
> +     while(<DNSCACHE>) {
> +       chop;
> +       /^([0-9.]+) (.*)$/;
> +       $dnscache{$1} = $2;
> +     }
> +     close DNSCACHE;
> +   }
> + }
>   else
>   {
>     print "Unknown option $arg\n";
> ***************
> *** 1407,1412 ****
> --- 1431,1459 ----
>
>   }
>
> + # cache negative dns responses. If given a filefor persistent storage,
> + # this gives a massive speedup.
> + sub mygethostbyaddr
> + {
> +   my($ip) = @_;
> +
> +   $dnscacheout{$ip}++; # Mark this IP as used to prune old entries
> +   if( defined $dnscache{$ip} ) {
> +     if( $dnscache{$ip} eq $ip ) {
> +       return undef;
> +     } else {
> +       return $dnscache{$ip};
> +     }
> +   } else {
> +     my $host = gethostbyaddr(inet_aton($ip), AF_INET);
> +     if( ! defined $host ) {
> +       $dnscache{$ip} = $ip;
> +     } else {
> +       $dnscache{$ip} = $host;
> +     }
> +     return( $host );
> +   }
> + }
>
############################################################################
##
>
>   # print the line of a table to look up an IP address in whois databases;
> ***************
> *** 1415,1421 ****
>   {
>     my($ip) = @_;
>
> !   my $host = gethostbyaddr(inet_aton($ip), AF_INET) if defined
$dns_option;
>
>     my $target= &target('lookup');
>     print "<tr><td rowspan=2>$ip</td>\n";
> --- 1462,1468 ----
>   {
>     my($ip) = @_;
>
> !   my $host = mygethostbyaddr($ip) if defined $dns_option;
>
>     my $target= &target('lookup');
>     print "<tr><td rowspan=2>$ip</td>\n";
> ***************
> *** 1483,1497 ****
>   # sort $a and $b (parsed alerts) by time, returning a <=>/cmp type
result
>   sub sort_by_time
>

> !   my(@pieces1) = ($a->{'month'},$a->{'date'},split(':',$a->{'time'}));
> !   my(@pieces2) = ($b->{'month'},$b->{'date'},split(':',$b->{'time'}));
> !
> !   foreach (0..$#pieces1)
> !    {
> ! return -1 if $pieces1[$_] < $pieces2[$_];
> ! return 1 if $pieces1[$_] > $pieces2[$_];
> !    }
> !  return 0;
>   }
>
>
############################################################################
##
> --- 1530,1536 ----
>   # sort $a and $b (parsed alerts) by time, returning a <=>/cmp type
result
>   sub sort_by_time
>

> !   return($a->{'ltime'} <=> $b->{'ltime'});
>   }
>
>
############################################################################
##
> ***************
> *** 1504,1518 ****
>    return $alert1 unless defined $alert2;
>    return $alert2 unless defined $alert1;
>
> !  my(@pieces1) =
($alert1->{'month'},$alert1->{'date'},split(':',$alert1->{'time'}));
> !  my(@pieces2) =
($alert2->{'month'},$alert2->{'date'},split(':',$alert2->{'time'}));
> !
> !  foreach (0..$#pieces1)
> !   {
> !    return $alert1 if $pieces1[$_] < $pieces2[$_];
> !    return $alert2 if $pieces1[$_] > $pieces2[$_];
> !   }
> !  return $alert1;
>   }
>
>
############################################################################
##
> --- 1543,1550 ----
>    return $alert1 unless defined $alert2;
>    return $alert2 unless defined $alert1;
>
> !  return $alert1 if ($alert1->{'ltime'} < $alert2->{'ltime'});
> !  return $alert2;
>   }
>
>
############################################################################
##
> ***************
> *** 1525,1538 ****
>    return $alert1 unless defined $alert2;
>    return $alert2 unless defined $alert1;
>
> !  my(@pieces1) =
($alert1->{'month'},$alert1->{'date'},split(':',$alert1->{'time'}));
> !  my(@pieces2) =
($alert2->{'month'},$alert2->{'date'},split(':',$alert2->{'time'}));
> !
> !  foreach (0..$#pieces1)
> !   {
> !    return $alert1 if $pieces1[$_] > $pieces2[$_];
> !    return $alert2 if $pieces1[$_] < $pieces2[$_];
> !   }
>    return $alert2;
>   }
>
> --- 1557,1563 ----
>    return $alert1 unless defined $alert2;
>    return $alert2 unless defined $alert1;
>
> !  return $alert1 if ($alert1->{'ltime'} > $alert2->{'ltime'});
>    return $alert2;
>   }
>
> cvs server: Diffing network/monitoring/snort/SnortSnarf/cgi
> cvs server: Diffing network/monitoring/snort/SnortSnarf/include
> Index: network/monitoring/snort/SnortSnarf/include/snort_alert_parse.pl
> diff -c
network/monitoring/snort/SnortSnarf/include/snort_alert_parse.pl:1.1.1.2
network/monitoring/snort/SnortSnarf/include/snort_alert_parse.pl:1.3
> ***
network/monitoring/snort/SnortSnarf/include/snort_alert_parse.pl:1.1.1.2 Fri
Jan  5 20:39:59 2001
> --- network/monitoring/snort/SnortSnarf/include/snort_alert_parse.pl Sat
Jan  6 01:00:48 2001
> ***************
> *** 22,27 ****
> --- 22,30 ----
>
>   $::sap_version = "v111500.1";
>
> + use Time::Local;
> + @ltime = localtime;
> +
>   my %monthnum=('Jan' => 1,
>   'Feb' => 2,
>   'Mar' => 3,
> ***************
> *** 266,271 ****
> --- 269,282 ----
>   }
>
>   # unlike in the old snortsnarf code, we do not record 'file' and we
record format, not file type
> +
> + # Calculate the seconds since the epoch. Speeds up sorting 1500%
> + # and the entire run by 30%
> + my(@hms) = split(':',$alert{'time'});
> + # If the alert month appears to be in the future, it's really
> + # from last year
> + my($year) = $alert{'month'}-1>$ltime[4]?$ltime[5]-1:$ltime[5];
> + $alert{'ltime'} =
timelocal($hms[2],$hms[1],$hms[0],$alert{'date'},$alert{'month'}-1,$year);
>
>   return \%alert;
>   }
> ***************
> *** 280,285 ****
> --- 291,298 ----
>   $file_type = 'scan';
>   } elsif($log =~ /syslog/) {
>   $file_type = 'syslog';
> +   } elsif($log =~ /secure/) {
> + $file_type = 'syslog';
>     } elsif($log =~ /messages/) {
>   $file_type = 'syslog';
>   } else {
> ***************
> *** 358,361 ****
>
>   1;
>
> ! # $Id: snort_alert_parse.pl,v 1.1.1.2 2001/01/06 01:39:59 ajlill Exp $
> --- 371,374 ----
>
>   1;
>
> ! # $Id: snort_alert_parse.pl,v 1.3 2001/01/06 06:00:48 ajlill Exp $
> cvs server: Diffing network/monitoring/snort/SnortSnarf/nmap2html
> cvs server: Diffing network/monitoring/snort/SnortSnarf/sisr
> cvs server: Diffing network/monitoring/snort/SnortSnarf/sisr/cgi
> cvs server: Diffing network/monitoring/snort/SnortSnarf/sisr/include
> cvs server: Diffing network/monitoring/snort/SnortSnarf/sisr/modules
> cvs server: Diffing network/monitoring/snort/SnortSnarf/utilities
> Linux-i686-2.4.1 $


_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com





More information about the Snort-users mailing list