[Snort-devel] Last CID and Duplicate Records

Ron Shuck rshuck at ...1949...
Mon Jan 12 11:17:04 EST 2004

Hi All,

I have been experiencing a problem MySQL problem lately. When archiving
records in ACID, I would receive duplicate record errors. The
snort-users archive provided an answer to why. To summarize, when a
sensor starts it uses the max(cid) in the snort.event database table,
but due to archiving this can often be 0 or less that the max(cid) in
the snort_archive.event DB table. This causes duplicates on subsequent
archives if the sensor is restarted. 

The snort.sensor table stores the last_cid and this is updated on proper
start up and shut down. But here is the problem (IMHO). In case of a
crash, if the max(cid) is greater than the last_cid, the field is
updated. This is great and necessary. However, regardless of the value
of last_cid, the max(cid) is always used to determine the next cid to
use by the sensor. So, the last_cid field is just info, it is never
really used.

So, to correct this for my installation, I modified the database code to
use the greater of the max(cid) or last_cid. The code that updates the
last_cid if max(cid) is greater is still valid. What this does is almost
eliminate duplicate records caused by archiving. Of course, if there is
a crash duplicates could still be possible. The exception is that if
there is a crash, and you manually update the last_cid. It also means
that you can set the next cid to be used by stopping the sensor,
updating the sensor.last_cid field with the desired value. As long as
the last_cid is greater than the max(cid) all is well.
Modified 'output-plugin/spo_database.c' (476)
OLD: data->shared->cid = event_cid
NEW: data->shared->cid = event_cid > sensor_cid ? event_cid : sensor_cid

This was valid for 2.0.0, the line numbers may be off a bit for later

Best Regards and happy Snorting,

Ron Shuck, CISSP, GCIA, CCSE - Managing Consultant 
Buchanan Associates - A Technology Company in the People Business 

More information about the Snort-devel mailing list