[Snort-users] A complication with an unconventional use of Snort
bahdko at ...71...
bahdko at ...71...
Tue Sep 19 11:49:52 EDT 2006
I have snort sensors deployed in an uncommon scenario, and I'm having a complication I'm hoping someone could give me input on.
The first part of the process does not experience problems.
The data from the LAN is first gathered by a snort process that is run that just creates a binary file. The snort program that does this is started like so:
/usr/local/bin/snort -l /var/log/snort -bD -i eth1
This process results in a binary log file named something similar to snort.log.1126876613.
Then, when the Snort program has been restarted and the binary log file is closed, the second part of the process is as follows.
I use another instance of Snort to read this binary log file and re-log it into a fully-expanded ASCII format. An example command I use to do this is as follows:
/usr/local/bin/snort -dvCeq -K ascii -r /var/log/snort/snort.log.1126876613 net 192.168.10.0/24 -D -l /var/log/asciilogs/
The result of this operation is that there are directories within /var/log/asciilogs that contain all of the packet information, in ASCII, from the binary file.
My problem is this:
During this second instance of Snort, Snort appears to take the entire binary log file, no matter how big it is, and steadily read the whole thing into memory as it processes it. I can watch in "top" as the memory and swap available on the machine are steadily sucked away until the amount of memory displacing the binary file size are represented, and then the process ends and frees up the memory.
Does Snort really have to do it this way and take all of that memory? I know that Snort is processing this file sequentially, more or less packet-by-packet, it would seem that it shouldn't be necessary to hold the entire file in memory the whole time.
I periodically have some binary log files that exceed the swap+memory of my sensor, and I can't process those because of the way Snort is doing this.
More information about the Snort-users