[Snort-devel] Plugin API Feature Request

Milani Paolo Paolo.Milani at ...866...
Wed Feb 15 01:02:03 EST 2006

Hi everyone,

I think the requests Thomas is making are very reasonable. Whether his plugin idea is good or not, a richer plugin API would be to everyone's benefit. I have ran into all three of the issues mentioned by him while developing my own plugins.

1- CleanExit being called in signal handler context: I would be so bold as to call this a bug, it means you can't do anything useful in exit function. (like reporting or logging final statistics, alerting that you were shut down, saving state for any plugins which need to keep state between snort sessions). Our local branch of Snort uses a flag exactly as Thomas suggested so the CleanExit can happen outside signal handler, and the main reason was that we have plugins that need to save state between sessions. Anyhow we've been told this will be fixed, so we're cool.

2- Periodic callbacks: these also are useful for a number of things. Heartbeat functionality is the first, but not the only one (and a plugin API function is the best way, so that anyone can implement their own heartbeat function the way they like it). Another issue (that we've encountered) is the need to do some periodic cleanup in preprocessors that keep state.. The work around is always to use a preprocessor which tests if a certain time has elapsed or not, but I think it would be cleaner (and more efficient) to have a periodic callback API. It could be implemented as a priority queue of timers, or more likely in some simpler way (example: decide a minimum time resolution of 5 seconds, and call all registered callbacks every 5 seconds) It is more efficient because you don't have N different preprocessors each testing if a certain time has elapsed. It is cleaner because there is no current packet (which really has nothing to do with periodic cleanup). 

3- callback at end of initialization: not much to say here. Just that we also came up against this issue, needed to do some postprocessing of our option nodes after ALL of them were created during rule parsing. Needless to say, the workaround is similar to the one that Thomas suggested: postprocess each option the first time it is reached in rule evaluation. This adds computational overhead ("am i initialized?" each time we reach the option) and bug-prone complexity.

There are other extensions to the plugin API which I think would be useful and would allow the snort code to grow in a more structured way. A richer API allows outside developers to extend snort to their needs much more easily, and it also allows the main Snort code to be more modular.

A "RegisterConfig" plugin API func to allow any snort plugin to add configuration keywords to snort (and get a callback to their parameter parsing func). This way when we want to add a "config: something" functionality to snort we don't have to tamper with parser.c anymore. So if config: stateful is really a Stream4 option, all the code concerning it should be in stream4 source files. 

Paolo Milani

Subject: RE: [Snort-devel] Plugin API Feature Request
Date: Mon, 13 Feb 2006 16:55:54 +0100
From: <Thomas.Seiler at ...2736...>
To: <jeff at ...835...>, <snort-devel at lists.sourceforge.net>
Cc: <Patrick.Bizeau at ...2736...>

Hi Jeff,
Hi list,=20

> I don't completely understand your issue. I still suspect that a
> callback within the main pcap loop would be sub-optimal.

Yes, I was not very clear in my last mail, sorry for that.
I will try to be clearer in this mail even if this means that the mail
will get long.

I agree that it is not very optimal to break the pcap loop for
performance reasons, but I am not sure how one could address the issues
in another way. That is why I wanted to start a discussion. I think the
best way currently is to have two version of the main loop:
- one with pcap_dispatch if there is a plug-in loaded who benefits from
- one with pcap_loop for those who only need the speed.

Maybe I should first tell you something about the context. We are trying
to build a new and highly experimental output plug-in for snort. The
idea is to fork a child process or second thread depending on platform
which handles the output-plugin so that the snort decoder is not blocked
by the output. The goal is to reduce sensor deployment complexity.

While working on this plug-in, I found these issues with the current
snort plug-in API:

(issue 1 : callbacks in signal handler context)

CleanExit() callback is currently called from a signal handler context.
On many platforms this means that it is not safe to call any libc
function that is not known to be re-entrant (thread-safe). This can lead
worst case to a segfault killing snort. The problem is, that CleanExit()
is also called when snort receives a SIGHUP - to reload its

We wanted to log the reload / shutdown of snort in the database as a
kind of meta-alarm, but I cannot execute an SQL statement in my
CleanExit() callback, because it is called asynchronously to the packet
logging. It is therefore quite probable that the database connection is
midway in the execution of another statement from the packet processing
callbacks, and logging the reload/shutdown event will result in
desynchronizing the db connection instead of logging anything.

Also, if I start threads or child processes from my output plug-in in
order to decouple logging from the detection engine of snort (i.e. see
prelude plug-in), then I cannot cleanly shutdown these threads / childs
because any call to libc is dangerous, and could kill snort when
reloading the config.

Currently the prelude plug-in is suffering from this as well. If you
have prelude logging enabled and signal SIGHUP to snort, you will end up
with zombie processes.

This is fixable by a pcap_dispatch instead of pcap_loop in the main
loop. The signal handler would just set a global variable which is
queried to exit the main loop. After this the CleanExit() handlers can
be called.

This might however have an impact on performance.

Steven Sturges [steve.sturges at ...402...] wrote on 09.02.06 14:52
> We are already working on rearranging the main loop and using pcap=20
> dispatch.  We are also addressing the non-reentrant nature of the
> signal handlers along with this set of changes.

I was very glad to hear that. So hopefully, the next snort release will
already have solved the CleanExit() issues.

(issue 2 : no regular callback for heartbeat functionality available)=20

It was also one of the goals to have a kind of heartbeat from the
sensors. Every N seconds, it should send a heartbeat to the database,
saying that it is up and running, and also giving some status about the
system (i.e. total number of pakets, number of dropped packets and CPU
usage) It is important to get this heartbeat from the detection thread /
process, because then we can supervise the whole logging path, from the
detector, via logging thread to the database.

I can currently do this in the packet processing callbacks and check
each time if I should send the next heartbeat. But what if no packet
arrives for more than N seconds? At the moment I can't tell a dead
sensor from a sensor without traffic, which somehow makes the whole
heartbeat idea somehow useless.

It would also be interesting to use heartbeat callback to feed commands
to a sensor via the database, i.e. the heartbeat callback could poll for
a pending command. (i.e reload,stop,...)

Jeff Nathan [jeff at ...835...] wrote on 13.02.06 13:03
> If you traverse a list of plugin callbacks and one of them blocks, =20
> then all of snort is blocked.  Perhaps this is a circular argument, =20
> it's not intended to be so, however...

Well the same is true for the alert logging callbacks, nevertheless they
are very useful. The overhead for this should be in the order of
magnitude for one additional alert logged every N seconds, otherwise the
idea is not practical. I think nobody will have to low-level-format a
hard drive in this callback ;)

The problem is that currently pcap_loop is blocking when there is no
packet to sniff. To execute a callback every N seconds, one would have
to exit the pcap_loop from time to time, check if its time to heartbeat
and then call the callbacks. It is important that such a heartbeat
callback happens in sync with the packet processing, because only then
it can use whatever resources (db handles, sockets, etc) the normal
alert logging uses.

Jeff Nathan [jeff at ...835...] wrote on 13.02.06 13:03
> Perhaps your database provides you a handle that is selectable or
> similar?  In that case you could register an event for the database
> handle and its callback could call your intended callback, provided
> the database was in a suitable state.

I am not sure I understand your solution correctly. I think you are
calling libevent's event_loop() whenever there is an alert to be logged.
But when there is no alert for a long period of time, and a heartbeat
should be sent, how (from where ?) is the callback then called ? This is
the part that I don't understand... I think it has to be called form an
alarm handler, and then I have the same issues as in (1).
If snort will change in the near future to a pcap_dispatch main loop
exit the packet handling from time to time (i.e. once a second) to
handle signals, there is another way to add heartbeats to snort.

time_t last_heartbeat;
time_t between_heartbeats;

while(running) {
   // ... snort main loop ...
   if (last_hearbeat + between_hearbeats < time()) {

   // ... snort main loop ...


   // ... snort main loop ...


(issue 3 : callbacks at end of initialisation )

This is a just a minor issue. We wanted to be able to save the whole
ruleset of snort at startup to the database, instead of checking for
each alert if the corresponding signature was already saved in the
database as the current spo_database.c does it. But the problem is that
at initialization time, the plugin might not see the whole
configuration. The order at which plugins are initialized / rules are
parsed depends on the order in which they are in the config file. If the
plugin gets loaded before any rules, it will see nothing.

So we had to wait for the first packet before we could synchronize the
signatures with the database. The question was, if there is a way to be
called after all rules are parsed, so that the plug-in can have a look
at the rules.

I hope I was able to clarify my issues. Feel free to ask if there are
any questions left open. My intention was to start a discussion about
the mentioned issues and possible solutions...

Best Regards,
Thomas Seiler

Thomas Seiler
Ing. sys. com. dipl. EPFL
Security and Service Management
Ostermundigenstrasse 93
CH - 3050 Bern

Phone:  +41 (0)31 342 42 69
Mobile: +41 (0)79 427 97 26
Fax:    +41 (0)31 892 62 27

thomas.seiler at ...2736...

Gruppo Telecom Italia - Direzione e coordinamento di Telecom Italia S.p.A.

This message and its attachments are addressed solely to the persons
above and may contain confidential information. If you have received
the message in error, be informed that any use of the content hereof
is prohibited. Please return it immediately to the sender and delete
the message. Should you have any questions, please send an e_mail to 
MailAdmin at ...2137... Thank you

More information about the Snort-devel mailing list