[Snort-devel] Distributed Snort
peter at ...799...
Thu Nov 14 04:37:01 EST 2002
i like your suggestion in that you are still collecting the data and once it
detects that the database has come back online it would try and then insert
the data. The only issue i have is how you would be storing the queued data ?
in a log file or something similar ?
if you are storing it in a file then why not just store it on the current
sensor instead of the remote one?
you could still queue the data if the database connection is lost.
you would probably want to then think about how many retries you want in case
the remote database server has had to be rebooted or the database daemon
restarted or perhaps if the database is temporarily unavailable for backup
purposes and the like.
currently i log data from my sensors to two different database servers
(PostgreSQL) and am looking at a way of syncing them should one go down. i
believe it can be done via PostgreSQL but i havent delved into it yet.
i'd also be keen to have some code which checks to see that the data has been
inserted correctly and if the error returned is that of a lost connection or
termination etc etc then it should try to reconnect "n" times to the database
based on the original startup parameters. if it then fails again after the
"nth" time then it invokes your queuing method :)
i'd be interested in this so "go for it" ;-)
peter at ...799...
>While planning a large Snort roll-out with centralized logging, I'm
>starting to see some problem withs having the sensors log directly to a
>database. My main gripe is performance and reliability. Barnyard can
>help in the performance area, but one seems to be limited to to MySQL on
>the back end. Also, the database plugins doesn't seem to check for and
>handle error conditions very well. What does a sensor do when the link
>to the database is temporarily down?
>I'm considering writing an output and input plugin for Snort that could
>alleviate these problems. I'm posting here a) for a sanity check, and b)
>if there is an interest in this setup (in which case I will program so
>that the code can be easily shared and integrated in other setups).
>My plans is this: A custom output plugin in Snort (decoupled in a
>thread, or a socket), and also in Barnyard, will send event and packet
>data via TCP to a remote Snort sensor (instead of a database). The
>decoupled output plugin would realize a lost connection and reconnect
>after a short waiting period. At the same time, Snort is still running,
>catching alerts, and queueing alert and logging data to this plugin.
>[This Snort box sniffs packets from the network, runs it through the
>preprocessors and detection engine, and then queues and transmits the
>alert and log output].
>On the receiving end I envision a normal Snort installation, except that
>it is started with a new command line parameter which will cause Snort
>not to read packets from the network as it usually does, but instead
>accept the TCP connection from the remote Snort sensor. It will then use
>the received information to call its output plugins as usual. [This
>Snort box reads the event and otn data from the TCP session, and only
>calls the log and alert plugins, bypassing preprocessors and detection
>With this setup it should be possible to have several sensors do the
>detection, pass the data to a central Snort instance which will make use
>of the already existing plugins, including Postgres, tcpdump, ascii,
>In addition, the receiving Snort box can in turn forward the data to yet
>another Snort instance, mainly for redundancy. The remote sensor can
>connect to the receiving pair in a round robin fashion, the the
>receiving box can log to it's database, forward it to its Snort mirror
>which will log it into another database. Or forward to a final tcpdump
>Unless you can convince me that this is a bad idea, I will start coding.
>If there is an interest in such a distributed Snort setup, please let me
>know and I plan accordingly, although the addins shouldn't be very
>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG v1.0.7 (GNU/Linux)
>-----END PGP SIGNATURE-----
>This sf.net email is sponsored by: To learn the basics of securing
>your web site with SSL, click here to get a FREE TRIAL of a Thawte
>Server Certificate: http://www.gothawte.com/rd524.html
>Snort-devel mailing list
>Snort-devel at lists.sourceforge.net
More information about the Snort-devel