[Snort-users] sizing a snort ids system?

Jed Pickel jed at ...153...
Fri Aug 11 15:55:16 EDT 2000


Hey John,

Comments are included inline...

> I've been testing (just for fun) snort on a Dual P-III 800Mhz, 1GB RAM
> and 2x17GB SCSI disks (AHA-2940U2 controller).  The machine has a
> 100Mb/s network card, but in peek hours there are up to 140Mb/s passing
> through the switch.
> 
> >From top:
> 
>   9:40am  up 21 days, 20:24,  2 users,  load average: 1.64, 1.05, 0.50
> 30 processes: 28 sleeping, 2 running, 0 zombie, 0 stopped
> CPU states:  3.5% user, 29.2% system, 27.3% nice, 39.7% idle
> Mem:  1036256K av,  536808K used,  499448K free, 11896K shrd, 475916K buff
> Swap:  128512K av,       0K used,  128512K free               18800K cached
> 
>   PID USER     PRI  NI  SIZE  RSS SHARE STAT  LIB %CPU %MEM   TIME COMMAND
> 24738 mysql     12   5  9540 9540  1044 R N     0 88.4  0.9   2:13 mysqld
> 24737 root       1   0  1088 1088   588 S       0 20.0  0.1   0:32 snort
>  1199 mysql      6   5  9540 9540  1044 S N     0  9.9  0.9   1:16 mysqld
> 24769 root       1   0   864  864   684 R       0  0.7  0.0   0:02 top

Wow.. Your processor appears to be at operating at more than %118
CPU. ;) I take it you are using so much CPU as a result of logging
_ALL_ packets to the mysql database.

> And the size of the MySQL database:
> 
>         09:52:06: 100 k (an empty database)
>         09:52:30: 3796 k
>         09:52:38: 5012 k
>         09:53:40: 14544 k
>         09:54:00: 17620 k
>         09:54:30: 22096 k
>         09:54:43: 24124 k
>         09:55:55: 34932 k

Yup.. If you want to log all packets on a busy link you will need a
lot of disk space.

> [root at ...311...:/data/mysql/snort]# ls -l *ISD
> -rw-rw----    1 mysql    mysql     6391045 Aug 11 09:56 event.ISD
> -rw-rw----    1 mysql    mysql        3872 Aug 11 09:56 icmphdr.ISD
> -rw-rw----    1 mysql    mysql     5229009 Aug 11 09:56 iphdr.ISD
> -rw-rw----    1 mysql    mysql          24 Aug 11 09:52 sensor.ISD
> -rw-rw----    1 mysql    mysql     3865060 Aug 11 09:56 tcphdr.ISD
> -rw-rw----    1 mysql    mysql         992 Aug 11 09:56 udphdr.ISD
> 
> 
> 
> And a litle example of why I don't like that all packages are creating
> an event:

For the benefit of those that have not kept up with this thread, the
database plugin does not log all packets unless it is explicitly
configured to do so.

> mysql> select count(*) from event;        
> +----------+
> | count(*) |
> +----------+
> |   193668 |
> +----------+
> 1 row in set (0.00 sec)
> 
> mysql> select * from event where signature != "NULL MESSAGE";
> +-----+-------+---------------+---------------------+
> | sid | cid   | signature     | timestamp           |
> +-----+-------+---------------+---------------------+
> |   1 | 14040 | FTP-bad-login | 2000-08-11 09:52:24 |
> +-----+-------+---------------+---------------------+
> 1 row in set (0.65 sec)

It does seem that the problem you are trying to point out her is with
all of the space consumed by the string "NULL MESSAGE".

   "NULL MESSAGE" requires 13 bytes storage in MySQL.
   193667 packets you logged had this in the message column.

   13 * 193667 = 2517671

   So you have about 2.5 Megs worth of "NULL MESSAGE" strings.

The rest of the info in the event table is the cid (4 bytes) sid (4
bytes) and the timestamp (8 bytes). Then there is also some space
required for the indexes. All total your event table takes up about
6.4 Megs...

> -rw-rw----    1 mysql    mysql     6391045 Aug 11 09:56 event.ISD

This can be reduced by enabling a one character message of '\0' aka ""
(requiring 1 byte) instead of writing out the string "NULL MESSAGE". I
am going to add a patch to the latest development version of the
database plugin to fix this. The only reason it can not be a real NULL
is because there is an INDEX on the signature column. It looks like 
(from a very quick test) the event table now requires about 1Mbyte for
every 50,000 events assuming you have a NULL message. 

> And a litle example of why I don't like that all packages are creating
> an event:

The "event" table keeps track of additional meta-data that is not a
part of a [ip|tcp|udp|icmp] header. It is necessary to keep track of
information such as a timestamps, names of signatures, and event id
numbers. If you have found a situation where you do not want to log
this information let me know and I can make a configuration option to
disable logging any info to that table.

* Jed




More information about the Snort-users mailing list