[Snort-devel] [Snort-users] How snort handles several copies of the same packet?

elof at ...969... elof at ...969...
Fri Oct 26 09:48:56 EDT 2012


(I'm adding snort-devel to the cc)


1)
Perfect! That was the short and simple explaination I was looking for. :-)
Thanks.


2)
Do you know if it is an easy task to add a counter to count the number of 
"redundant" packets that are received by snort, but skipped due to the 
fact that they are considered duplicates and therefore redundant?
(I.e. the packets we've been discussing in this thread.)

Like adding something simillar to this:

Packet I/O Totals:
   Received:    3716022
   Analyzed:    3716022 (100.000%)
   Dropped:     0 ( 0.000%)
   Filtered:    0 ( 0.000%)
   Outstanding: 0 ( 0.000%)
   Injected:    0
   Redundant:   1855011 ( 49.919%)

It would be very interesting to have a counter that states how many 
percent of the incoming packets is actually redundant to snort and 
is therefore discarded.
If one has a really high Redundant count, this indicates that the SPAN 
should probably be reviewed and reconfigured to get rid of those 
duplicates.
(...or if this is not possible place a smart tap in front of the sensor, 
and have the tap remove the duplicates)
The above example value, almost 50% should indicate that half the mirrored 
traffic is mirrored twice to the destination port.


3)
Under "Stream5 statistics:", would it be hard to also add a counter for 
the amount of detected TCP Retransmissions?

If one take the total amount of Redundant packets (see section 2 above) 
and subtract the amount of detected TCP Retransmissions, one should 
have an estimate of how much unneccessary traffic snort is actually 
burdened with.


4)
Under "Stream5 statistics:" I see
   TCP Discards: 12345
   TCP Gaps:     2222

I guess that "TCP Gaps" is a counter of missing packets in the SPAN, in an 
otherwise working tcp flow.

What are "TCP Discards"?


5)
Given the following example counters...

Breakdown by protocol (includes rebuilt packets):
   Eth:       100000 (100%)
   TCP:       80000  (80%)
Packet I/O Totals:
   Dropped:   1000   (1%)
Stream5 statistics:
   TCP Gaps:  2222

Could I estimate the amount of EXTERNAL drops in the mirrored traffic 
by doing something like this? :

There are 2222 detected TCP gaps.
1% of all traffic is randomly dropped internally on the sensor itself.
1% of 80 000 tcp packets = 800.
So roughly 800 of the 2222 tcp gaps are due to the sensor itself.
The rest, roughly 1400 gaps, should therefore be EXTERNAL drops.

Am I thinking correctly?

/Elof


On Wed, 24 Oct 2012, Russ Combs wrote:

> On Wed, Oct 24, 2012 at 11:35 AM, <elof at ...969...> wrote:
>
>>
>> Open Source Snort.
>> No global threshold.
>>
>> I'm logging directly to ascii from snort and to unified2 where barnyard2
>> then take over.
>> In both my ascii alert log and in my postgres I only get one alert.
>>
>> I think this is great! ...but I'm curious and would like to know how and
>> where in the process this filtering/aggregation is done.
>>
>
> There are a couple things going on here.  Stream5 basically processes data
> the same way a receiving host would.  For example, retransmitted or
> duplicated data that falls to the left of the window (ie it was already
> acknowledged) will just be discarded.
>
> And Snort will not queue events that have already fired on the same session
> (or fragment).  These non-events are counted and output at shutdown as
> Limits::Alert:
>
> Limits:
>      Match:            0
>      Queue:            0
>        Log:            0
>      Event:            0
>      Alert:            0
>
> Hope that helps.
>
>>
>> /Elof
>>
>>
>> On Wed, 24 Oct 2012, Joel Esler wrote:
>>
>>> Are you talking about Open Source Snort?  Or Sourcefire product?
>>>
>>> Do you have a global threshold in place?
>>>
>>>
>>> On Oct 24, 2012, at 10:42 AM, elof at ...969... wrote:
>>>
>>>>
>>>> Yes, there's an performance impact. That is expected.
>>>>
>>>> But what about the alerting? Somewhere snort must be
>> filtering/aggregating the packets, understanding that the "duplicates" are
>> actually the same packet, and only generate ONE alert for its bad payload
>> data.
>>>> I'm asking for a description of this part.
>>>>
>>>> How does snort detect and filter out these "duplicates"?
>>>> Which packets are disregarded and which are kept?
>>>>
>>>> Like if the packet in my example contain malicious code, will the
>> logged packet be
>>>>
>>>> routing)
>>>> The first packet with TTL 60?
>>>>
>>>> retransmission)
>>>> The first packet with ipid 3333?
>>>>
>>>> duplicate SPAN)
>>>> Simply the first packet?
>>>> Another question: Are true duplicates seen as retransmissions and
>> processed as such?
>>>>
>>>>
>>>>
>>>> Perhaps the answer is that the logging system simply detects that the
>> next received, analyzed and logged packet is the same as the one just
>> logged, and silently supresses it.
>>>> I don't think this filtering/aggregation happen this late in the
>> process though.
>>>> Some clarification of how this works would be appreciated.
>>>>
>>>> /Elof
>>>>
>>>>
>>>> On Wed, 24 Oct 2012, Joel Esler wrote:
>>>>
>>>>> On Oct 24, 2012, at 4:48 AM, elof at ...969... wrote:
>>>>>> I know that snort only generates ONE alert even if the mirrored
>> traffic
>>>>>> see the same packet twice or more:
>>>>>>
>>>>>> ...like before and after a router:
>>>>>> x:x:x:x:x:x y:y:y:y:y:y 1.1.1.1:1234 -> 2.2.2.2:80 ipid 3333, TTL 60
>>>>>> y:y:y:y:y:y z:z:z:z:z:z 1.1.1.1:1234 -> 2.2.2.2:80 ipid 3333, TTL 59
>>>>>> ^^^^^^^^^^^^^^^^^^^^^^^                                           ^^
>>>>>>
>>>>>> ...or tcp retransmissions:
>>>>>> x:x:x:x:x:x y:y:y:y:y:y 1.1.1.1:1234 -> 2.2.2.2:80 ipid 3333, TTL 60
>>>>>> x:x:x:x:x:x y:y:y:y:y:y 1.1.1.1:1234 -> 2.2.2.2:80 ipid 3334, TTL 60
>>>>>> x:x:x:x:x:x y:y:y:y:y:y 1.1.1.1:1234 -> 2.2.2.2:80 ipid 3335, TTL 60
>>>>>>                                                        ^^^^
>>>>>>
>>>>>> ...or two *exact* duplicates of every packet due to faulty SPAN:
>>>>>> x:x:x:x:x:x y:y:y:y:y:y 1.1.1.1:1234 -> 2.2.2.2:80 ipid 3333, TTL 60
>>>>>> x:x:x:x:x:x y:y:y:y:y:y 1.1.1.1:1234 -> 2.2.2.2:80 ipid 3333, TTL 60
>>>>>>
>>>>>>
>>>>>> Only having one alert in the above cases is really nice, but I wonder:
>>>>>>
>>>>>> Can someone describe how this is done and what is happening in snort,
>> both
>>>>>> on the individual packet level, and in stream5?
>>>>>>
>>>>>> How does snort detect and filter out these "duplicates"?
>>>>>> Which packets are disregarded and which are kept?
>>>>>
>>>>> Everything is analyzed independently.  I've seen the problem commonly
>> at many sites.  Filtering out the duplicate traffic on a span is important
>> for optimum performance.
>>>>>
>>>>> --
>>>>> Joel Esler
>>>>> Senior Research Engineer, VRT
>>>>> OpenSource Community Manager
>>>>> Sourcefire
>>>
>>
>>
>> ------------------------------------------------------------------------------
>> Everyone hates slow websites. So do we.
>> Make your web apps faster with AppDynamics
>> Download AppDynamics Lite for free today:
>> http://p.sf.net/sfu/appdyn_sfd2d_oct
>> _______________________________________________
>> Snort-users mailing list
>> Snort-users at lists.sourceforge.net
>> Go to this URL to change user options or unsubscribe:
>> https://lists.sourceforge.net/lists/listinfo/snort-users
>> Snort-users list archive:
>> http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users
>>
>> Please visit http://blog.snort.org to stay current on all the latest
>> Snort news!
>>
>




More information about the Snort-devel mailing list