[Snort-users] Questions/Suggestion: Which data to put in the DB?

Jed Pickel jed at ...153...
Tue Aug 1 22:09:33 EDT 2000

> [Mike Andersen]
> | 
> | Why split up the ip_src and ip_dst into four different elements?

You have made some convincing arguments supporting the idea of having
only one four byte column to represent an IP address. I would like to
ask snortdb application developers (Yen-Ming, and anyone else that has
developed scripts or done any sort of manual interaction with the
snort database) your opinion on changing to one four byte column
instead of four one byte columns to represent IPs.

My only hesitation is that makes it significantly more difficult for a
human to interact directly with the database (I actually do most of my
analysis by interacting manually with the db). That being said, should
we make this switch, I want to be certain that someone is working on
some some apps to present data and do analysis based on data in the
database. Anyone? :-)

If we do make this switch I think it would be a good idea to include
both the four one byte columns and the one four byte column until the
1.7 release so we can transition smoothly.

> But doesn't it put a higher workload on snort?  

The difference is minor; however, if you were to log every packet on a
busy network it might be possible to measure a small performance

> And as long as we are talking about clean B or C class addresses, it
> might be easier (for the human), but what in the situations where
> you want to make a query for all addresses in a range with for
> example 23 or 25 net bits?

This is a good point.

> | I wanted to avoid having to rely on a lot of processing of
> | netmasks and such for applications, and I wanted to make SELECTs
> | and sorting as quick as possible. 
> Well, I have to disagree with you. :-) In most languages there are
> functions available to do this kind of processing, and I don't believe
> the overhead for such functions are any higher than the overhead of
> functions that have to handle a .128 subnet mask.

Well, there could be a small performance difference with some queries
based on how the columns are indexed but I think it is so minor that
it would be even hard to measure. 

> | The other reason I decided to split the IP into four fields is that is
> | generally how humans think about IP addresses, as four separate
> | numbers.
> I understand, but personally I like to have a clear distinguish between
> representation and presentation of a dataset. :-)

Ok.. You got me on this one. :)

> | You can get the timestamp by joining the event table with the iphdr
> | where both the "cid" (count id) and "sid" (sensor id) fields are
> | equal.
> But for IP packages that are not generating an event (if you are using
> snort only to collect packages)?

The way things are right now, you can only log events that are
triggered by matching a rule. The way I log all packets 
is to have these rules:

alert tcp any any -> any any (msg:"tcp";)
alert udp any any -> any any (msg:"udp";)
alert icmp any any -> any any (msg:"icmp";)

> | Note that I am also working on storing the data portion of the
> | packet. I am planning to do this using either base64 or just ASCII
> | with the binary filtered out (depending on what command line options
> | are used to invoke snort - notably whether or not the -C is used with
> | -d or not). I can use the same base64 routine to store the 256 bytes
> | in the ICMP header.
> What you are telling me here, makes me surer that snort might be the
> right tool to use as the fundament for our own IDS tools. :-)

Excellent. Snort is the swiss army knife of network based IDSs. Snort
and some duct tape and you are set for pretty much anything. ;)

> [Mike Andersen]
> |
> | UDP:
> | Seems that everything is logged... :-)
> [Jed Pickel]
> | 
> | Unless you want the checksum.. ;)
> Hehe...  Actually I was spending some time thinking about that. :-)
> Is there any reason that we would want to throw away that piece of
> information?

I think it is a good idea to include the more I think about it. Then
you could (for example) write a cool app that queries the database
and writes out tcpdump files. 

> [Mike Andersen]
> |
> | And a question: 
> | 
> | What about develop a standard database layout for anomaly based
> | IDS?  Or does this already exist?
> [Jed Pickel]
> | 
> | There is an IETF working group working on this. 
> I've browsed through it, and got the understanding that it only cared
> about the exchange format between IDS's.  I probably have to take some
> time and read it more carefully. :-)

Yeah. It is not currently within the mission of the IDWG to define
database formats; however, the IDWG work is very closely related. I
should have been more explicit about that.

* Jed

More information about the Snort-users mailing list