[Snort-devel] more fun questions

Martin Roesch roesch at ...48...
Tue Dec 19 18:14:31 EST 2000

Todd Lewis wrote:
> On Fri, 8 Dec 2000, Todd Lewis wrote:
> > It would probably look something like this in each paengine:
> >
> >       static void take_action(Packet *p);
> >
> >       void paengine_main_loop(void (*munge_packet_callback)())
> >       {
> >               u_char buf[BUFSIZE];
> >               Packet p;
> >
> >               while(get_packet(&buf, sizeof(buf))){
> >                       (*munge_packet_callback)(&buf, &p); /* dependent on packet type */
> >                       process_packet(&p);                 /* global, general for snort */
> >                       take_action(&p);
> >               }
> >       }
> >
> > That way, the paengine has visibility into the Packet struct, which
> > can be extended to pass around whatever info is needed.
> >
> > What do people think of these two approaches?
> Ok, so I am looking into doing this, and it seems easy enough.
> The first one, munge_packet_callback, is the grinder, and the second
> one is rules.c:Preprocess().  This entire loop is reminiscent of
> snort.c:ProcessPacket(), and this fact has got me thinking, is the
> paengine really the place to do this?  After all, all of this
> code is going to be duplicated in each paengine, and that's a very
> good warning sign that your layering is wrong.

Yes, it's wrong.  :)  The PaEngine code should merely provide an interface to
the packet acquisition mechanism, not the actual main packet processing loop. 
It initializes the interface, then hands packets up to the program when
requested.  The get_packet() call should actually be something like
(*acquire_packet)(&raw_pkt) which calls your packet acquisition function and
returns a filled in raw packet struct.  I think that this raw packet struct
should look something like this:

struct _RawPkt
    snort_timebuf ts;  /* timestamp */
    u_int32_t caplen;  /* captured buffer length */
    u_int8_t *raw;
} RawPkt;

Sharp eyed individuals will note that this is pretty much the same as what
libpcap hands you, but I'm pretty sure that this is the bare minimum you need
to have enough data to do the rest of the job (well, ok, you probably don't
need the timestamp).

Decoders are then called by passing the Packet struct and the RawPacket

(*grinder)(&RawPkt, &p);

"grinder" should probably be renamed to "decoder" or some such, since that's
the actual function it performs now.  So, the decoder stage gets handed the
raw data and a Packet struct to populate, does its job and when it returns the
Packet struct gets handed to the traffic analysis stage.  Traffic analysis
(detection) makes its decisions about the code and initiates any
responses/output based on detection events that are made.

> So, here's another idea.  What if we took ProcessPacket() and turned
> that into the main loop.  It would look basically like it does now,
> only it'd be a loop and would call the paengine to get and dispose of
> packets, like this:
>         void
>         ProcessPackets(int (*grinder))
>         {
>                 struct pastructure pas;
>                 u_char buf[10240];
>                 Packet p;
>                 pas.buf=&buf;
>                 pas.buf_size=sizeof(buf);
>                 pas.Packet = &p;
>                 while(1){
>                         pa->get_packet(&pas);
>                         (*grinder)(&pas);
>                         Preprocess(&p);
>                         switch(p.disposition){
>                                 case DISCARD:
>                                         if(!pa->discard(&pas))
>                                                 log_error("DISCARD called on packet where paengine %s does not support it!\n", \
>                                                         pa->name);
>                                         break;
>                                 case ALLOW:
>                                         pa->pass(&pas);
>                                         break;
>                                 /* Add in other actions here */
>                                 default:
>                                         pa->pass(&pas);
>                                         break;
>                         }
>                 }
> By hoisting this logic into the snort core, out of the paengine, I think that
> we would get several wins:
>         - it makes the paengines generic and therefore portable among
>           projects other than snort;
>         - it reduces code duplicated among paengines;
>         - it is easier to expand the range of supported actions on packets;
>         - it makes the whole thing multi-threadable, which can be
>           important for performance reasons, especially once snort is
>           in the critical path on firewalls.

I'm not sure if I necessarily agree with embedding the decision logic into the
main processing loop.  Not all applications of Snort are going to be
interested in dispositioning the packets after the decision code runs, so it's
probably best to treat it in a more modular fashion.  I'd actually recommend
an output plugin as the place to perform this disposition function, it's
logically executed in the same place as what you've done here and allows for
modular configuration at run-time (i.e. we don't execute the code if we're not
in "Gateway mode").

> The last one is important to me.  With Linux 2.4 having a completely
> multi-threaded networking stack in the kernel, it is conceivable that on
> an oct-way intel machine, you could user-space firewall gigabit ethernet
> at wire speed, which would be such a coup that I get a woody just thinking
> about it.  Ditto for the mainstream packet examination code.  (We would
> have to clean up some of the globals, etc., but it shouldn't be too bad.)

How's the SMP code in the 2.4 kernels?  On uniprocessor machines (i.e. the
vast majority of machines Snort runs on) multi-threading the engine has yet to
be proven as a Good Thing since all the overhead to perform the context
switching may have an impact that overrides the benefits of multi-threading in
the first place!

Additionally, on *BSD kernels SMP is poorly supported, while the BSD kernels
ostensibly have the best packet acquisition interfaces they may not be the
best platforms for multi-threading, not to mention the additional support
costs of going multi-threaded (people have a hard enough time installing
libpcap and libnet right now, requiring a threading library is going to add
another variable to the mix).  Additionally, we need to agree on a threading
library that works on all the platforms that Snort works on.  Libpth? 
Pthreads?  If we're going to multi-thread, we need to make sure that we can
answer these questions, or at the very least provide more than one compilation
path (#ifdefs for multithreaded code with a build time switch to activate it).

> So, this is my thinking.  Agree or disagree?  I would love to hear
> people's thoughts on this stuff, but if I don't then I'll just go ahead
> and code it along these lines.

I like the general direction, but we need to think about how it's going to fit
with the non-Linux, non-SMP crowd. :)  Remember, we're supporting over 21
platforms on a variety of architectures, so the base engine has to remain
compatible across all of these architectures!


> --
> Todd Lewis                                       tlewis at ...120...
>   God grant me the courage not to give up what I think is right, even
>   though I think it is hopeless.          - Admiral Chester W. Nimitz
> _______________________________________________
> Snort-devel mailing list
> Snort-devel at lists.sourceforge.net
> http://lists.sourceforge.net/mailman/listinfo/snort-devel

Martin Roesch
roesch at ...48...

More information about the Snort-devel mailing list