[Snort-devel] Snort full log files need heirarchy

James Hoagland hoagland at ...60...
Fri Mar 22 09:25:07 EST 2002

Hello Eric,

At 8:52 AM -0500 3/22/02, Bit Stream wrote:
>In working with Snort, I've found that when running in full log 
>mode, that having one directory per IP address doesn't scale well. 
>While I recognize the argument that a well tuned IDS shouldn't go 
>off very often, there are a few cases where attack patterns will 
>create a large number of victim IP addresses in a short period of 
>time.  When this occurs, the logdir suddenly contains scores and 
>scores of directories.  This is a practical problem for things that 
>want to stat the directory (like ls), it also creates unsolvable 
>problems if the number of directories exceeds the 32k limit.
>I'd propose that for the next version of snort, that you create a
>directory heirarchy.  Where IP becomes $LOGDIR/1/2/3/4/ when 
>snort is running in full log mode.  This would solve a lot of 
>problems by limiting the total number of directories in any 
>directory to 255.  Perhaps it is problematic performance wise, but 
>real performance oriented people shouldn't be running in full log 
>mode in real-time anyway.

This might be a good idea.  It is not a particularly good idea to let 
the attacker control your performance.  Especially if they can push 
it over a file system limit.  (Although, of course, they can control 
how many alerts you get.)  I would suggest this as an option though, 
to preserve backwards compatibility.

SnortSnarf used to place all its files in one directory too.  But 
now, it stores it in heirarchical directories for reasons like you 
describe.  The pages corresponding to IP files are in nested 
directories named after the first 3 bytes of the IP address.  I could 
see why you might use 4 bytes here though.

>This has some significant impacts on support programs such as 
>snortsnarf, but that should be solvable.  I was thinking it would 
>also be neat if snort provided a library for doing the filename 
>lookup so that every script didn't have to reinvent the path to the 
>details log file, but that's much less important to me.

It shouldn't be hard to adjust SnortSnarf's log linking.  Just give 
an argument to tell SnortSnarf how the logs are organized.  I'm not 
sure how much benefit a library like you describe would provide; 
maybe if there we more than two possibilities.  Curiously enough, 
SnortSnarf has an internal library to tell it where a particular page 
should be located, although no external program cares.


|*      Jim Hoagland, Associate Researcher, Silicon Defense      *|
|*            --- Silicon Defense: IDS Solutions ---             *|
|*  hoagland at ...60..., http://www.silicondefense.com/  *|
|*   Voice: (530) 756-7317                 Fax: (530) 756-7297   *|

More information about the Snort-devel mailing list