[Snort-devel] last_cid in new database scheme v106

Ian Macdonald secsnortdev at ...1490...
Sat Sep 14 19:29:02 EDT 2002


I agree that some of the things I suggested are database dependant, however
we don't just support oracle so we have to think about what a good overall
solution would be that works well with most databases.

(see comments inline)

Ian

----- Original Message -----
From: <ian.willis at ...1523...>
To: "Ian Macdonald" <secsnortdev at ...1490...>
Cc: <snort-devel at lists.sourceforge.net>
Sent: Friday, September 13, 2002 10:07 PM
Subject: Re: [Snort-devel] last_cid in new database scheme v106


> Some possible problems that relate to making optimisation such as you
> mention
>
> 1 Some of the optimisations that you are making are not universal
> solutions, they are database dependant. I believe that Oracle properly
> configured doesn't have any real gains from some of the changes that you
> believe will speed up queries.

Perhaps you would be kind enough to list the things that you don't think
will make a difference?

> 2 Proper indexing should reduce query time by log(n) unless a complete
> table scan is needed.

Agreed, but what would you rather do, insert a value into a character column
that has been indexed or into a int column that is indexed, or a int column
that isn't indexed at all? For speed I would choose the last option, if I am
doing any joins against the column I would choose an indexed int as a last
resort I would choose a varchar. I would only suggest using non indexed
tables for dataload part done by snort. All tables that are used by the
front end reporting system should be indexed.

> 3 I have worked in oraganisation where we have denormalized data
> structures to gain performance only to have the very changes that we made
> with the best intentions restricting future innovations, our crystal ball
> tended to be very clouded.

Noted. What would you recommend instead? and why

> 4 When doing bulk loads from files the integrity checking that the
> database normally does may not apply, again this is implementation
> dependant.

Yes, agreed, I was just putting an idea out there that may make databases
that support this feature the prefered databases for high volume systems.

>
>
> ----------------------
> I would love to see a different database structure, as long as we keep the
> existing spo for compatibility. For databases that support views it would
> also be possible to build one that maps back to the original db design.
>
> Based on my exprience build large dataload systems we should think think
> about a couple things.
>
> What would the goals for a new database structure be?
> Improve logging speed,
> Improve ability to pull out records quicker from db the front end,
> Move rule sets and snort configs to the database rather than text files.?
> ......
>






More information about the Snort-devel mailing list