[Snort-devel] [Snort-users] How snort handles several copies of the same packet?

Russ Combs rcombs at ...402...
Fri Oct 26 15:23:02 EDT 2012


On Fri, Oct 26, 2012 at 9:48 AM, <elof at ...969...> wrote:

> (I'm adding snort-devel to the cc)
>
>
> 1)
> Perfect! That was the short and simple explaination I was looking for. :-)
> Thanks.
>
>
> 2)
> Do you know if it is an easy task to add a counter to count the number of
> "redundant" packets that are received by snort, but skipped due to the fact
> that they are considered duplicates and therefore redundant?
> (I.e. the packets we've been discussing in this thread.)
>
> Like adding something simillar to this:
>
> Packet I/O Totals:
>   Received:    3716022
>   Analyzed:    3716022 (100.000%)
>   Dropped:     0 ( 0.000%)
>   Filtered:    0 ( 0.000%)
>   Outstanding: 0 ( 0.000%)
>   Injected:    0
>   Redundant:   1855011 ( 49.919%)
>
> It would be very interesting to have a counter that states how many
> percent of the incoming packets is actually redundant to snort and is
> therefore discarded.
> If one has a really high Redundant count, this indicates that the SPAN
> should probably be reviewed and reconfigured to get rid of those duplicates.
> (...or if this is not possible place a smart tap in front of the sensor,
> and have the tap remove the duplicates)
> The above example value, almost 50% should indicate that half the mirrored
> traffic is mirrored twice to the destination port.
>
> In general, that is not a trivial task and would probably be performance
prohibitive.

>
> 3)
> Under "Stream5 statistics:", would it be hard to also add a counter for
> the amount of detected TCP Retransmissions?
>

That can be done.

>
> If one take the total amount of Redundant packets (see section 2 above)
> and subtract the amount of detected TCP Retransmissions, one should have an
> estimate of how much unneccessary traffic snort is actually burdened with.
>
>
> 4)
> Under "Stream5 statistics:" I see
>   TCP Discards: 12345
>   TCP Gaps:     2222
>
> I guess that "TCP Gaps" is a counter of missing packets in the SPAN, in an
> otherwise working tcp flow.
>

Yes, where missing means data not seen was acked.  When Snort is seeing
both directions (as it should) and nothing is being dropped, gaps should
not happen.

>
> What are "TCP Discards"?
>
> Stream5 can discard packets for a variety of reasons, including repeated
SYNs, bad sequence numbers or timestamps, reassembly issues.  Many of these
can be discovered by enabling Stream5 preprocessor rules.

>
> 5)
> Given the following example counters...
>
> Breakdown by protocol (includes rebuilt packets):
>   Eth:       100000 (100%)
>   TCP:       80000  (80%)
> Packet I/O Totals:
>   Dropped:   1000   (1%)
> Stream5 statistics:
>   TCP Gaps:  2222
>
> Could I estimate the amount of EXTERNAL drops in the mirrored traffic by
> doing something like this? :
>
> There are 2222 detected TCP gaps.
> 1% of all traffic is randomly dropped internally on the sensor itself.
> 1% of 80 000 tcp packets = 800.
> So roughly 800 of the 2222 tcp gaps are due to the sensor itself.
> The rest, roughly 1400 gaps, should therefore be EXTERNAL drops.
>
> That seems reasonable.  However, note that we have an open bug on gap
counting, and I suspect many of those gap counts are bogus.

Am I thinking correctly?
>
> /Elof
>
>
>
> On Wed, 24 Oct 2012, Russ Combs wrote:
>
>  On Wed, Oct 24, 2012 at 11:35 AM, <elof at ...969...> wrote:
>>
>>
>>> Open Source Snort.
>>> No global threshold.
>>>
>>> I'm logging directly to ascii from snort and to unified2 where barnyard2
>>> then take over.
>>> In both my ascii alert log and in my postgres I only get one alert.
>>>
>>> I think this is great! ...but I'm curious and would like to know how and
>>> where in the process this filtering/aggregation is done.
>>>
>>>
>> There are a couple things going on here.  Stream5 basically processes data
>> the same way a receiving host would.  For example, retransmitted or
>> duplicated data that falls to the left of the window (ie it was already
>> acknowledged) will just be discarded.
>>
>> And Snort will not queue events that have already fired on the same
>> session
>> (or fragment).  These non-events are counted and output at shutdown as
>> Limits::Alert:
>>
>> Limits:
>>      Match:            0
>>      Queue:            0
>>        Log:            0
>>      Event:            0
>>      Alert:            0
>>
>> Hope that helps.
>>
>>
>>> /Elof
>>>
>>>
>>> On Wed, 24 Oct 2012, Joel Esler wrote:
>>>
>>>  Are you talking about Open Source Snort?  Or Sourcefire product?
>>>>
>>>> Do you have a global threshold in place?
>>>>
>>>>
>>>> On Oct 24, 2012, at 10:42 AM, elof at ...969... wrote:
>>>>
>>>>
>>>>> Yes, there's an performance impact. That is expected.
>>>>>
>>>>> But what about the alerting? Somewhere snort must be
>>>>>
>>>> filtering/aggregating the packets, understanding that the "duplicates"
>>> are
>>> actually the same packet, and only generate ONE alert for its bad payload
>>> data.
>>>
>>>> I'm asking for a description of this part.
>>>>>
>>>>> How does snort detect and filter out these "duplicates"?
>>>>> Which packets are disregarded and which are kept?
>>>>>
>>>>> Like if the packet in my example contain malicious code, will the
>>>>>
>>>> logged packet be
>>>
>>>>
>>>>> routing)
>>>>> The first packet with TTL 60?
>>>>>
>>>>> retransmission)
>>>>> The first packet with ipid 3333?
>>>>>
>>>>> duplicate SPAN)
>>>>> Simply the first packet?
>>>>> Another question: Are true duplicates seen as retransmissions and
>>>>>
>>>> processed as such?
>>>
>>>>
>>>>>
>>>>>
>>>>> Perhaps the answer is that the logging system simply detects that the
>>>>>
>>>> next received, analyzed and logged packet is the same as the one just
>>> logged, and silently supresses it.
>>>
>>>> I don't think this filtering/aggregation happen this late in the
>>>>>
>>>> process though.
>>>
>>>> Some clarification of how this works would be appreciated.
>>>>>
>>>>> /Elof
>>>>>
>>>>>
>>>>> On Wed, 24 Oct 2012, Joel Esler wrote:
>>>>>
>>>>>  On Oct 24, 2012, at 4:48 AM, elof at ...969... wrote:
>>>>>>
>>>>>>> I know that snort only generates ONE alert even if the mirrored
>>>>>>>
>>>>>> traffic
>>>
>>>> see the same packet twice or more:
>>>>>>>
>>>>>>> ...like before and after a router:
>>>>>>> x:x:x:x:x:x y:y:y:y:y:y 1.1.1.1:1234 -> 2.2.2.2:80 ipid 3333, TTL 60
>>>>>>> y:y:y:y:y:y z:z:z:z:z:z 1.1.1.1:1234 -> 2.2.2.2:80 ipid 3333, TTL 59
>>>>>>> ^^^^^^^^^^^^^^^^^^^^^^^                                           ^^
>>>>>>>
>>>>>>> ...or tcp retransmissions:
>>>>>>> x:x:x:x:x:x y:y:y:y:y:y 1.1.1.1:1234 -> 2.2.2.2:80 ipid 3333, TTL 60
>>>>>>> x:x:x:x:x:x y:y:y:y:y:y 1.1.1.1:1234 -> 2.2.2.2:80 ipid 3334, TTL 60
>>>>>>> x:x:x:x:x:x y:y:y:y:y:y 1.1.1.1:1234 -> 2.2.2.2:80 ipid 3335, TTL 60
>>>>>>>                                                        ^^^^
>>>>>>>
>>>>>>> ...or two *exact* duplicates of every packet due to faulty SPAN:
>>>>>>> x:x:x:x:x:x y:y:y:y:y:y 1.1.1.1:1234 -> 2.2.2.2:80 ipid 3333, TTL 60
>>>>>>> x:x:x:x:x:x y:y:y:y:y:y 1.1.1.1:1234 -> 2.2.2.2:80 ipid 3333, TTL 60
>>>>>>>
>>>>>>>
>>>>>>> Only having one alert in the above cases is really nice, but I
>>>>>>> wonder:
>>>>>>>
>>>>>>> Can someone describe how this is done and what is happening in snort,
>>>>>>>
>>>>>> both
>>>
>>>> on the individual packet level, and in stream5?
>>>>>>>
>>>>>>> How does snort detect and filter out these "duplicates"?
>>>>>>> Which packets are disregarded and which are kept?
>>>>>>>
>>>>>>
>>>>>> Everything is analyzed independently.  I've seen the problem commonly
>>>>>>
>>>>> at many sites.  Filtering out the duplicate traffic on a span is
>>> important
>>> for optimum performance.
>>>
>>>>
>>>>>> --
>>>>>> Joel Esler
>>>>>> Senior Research Engineer, VRT
>>>>>> OpenSource Community Manager
>>>>>> Sourcefire
>>>>>>
>>>>>
>>>>
>>>
>>> ------------------------------**------------------------------**
>>> ------------------
>>> Everyone hates slow websites. So do we.
>>> Make your web apps faster with AppDynamics
>>> Download AppDynamics Lite for free today:
>>> http://p.sf.net/sfu/appdyn_**sfd2d_oct<http://p.sf.net/sfu/appdyn_sfd2d_oct>
>>> ______________________________**_________________
>>> Snort-users mailing list
>>> Snort-users at ...2402...**net <Snort-users at lists.sourceforge.net>
>>> Go to this URL to change user options or unsubscribe:
>>> https://lists.sourceforge.net/**lists/listinfo/snort-users<https://lists.sourceforge.net/lists/listinfo/snort-users>
>>> Snort-users list archive:
>>> http://sourceforge.net/**mailarchive/forum.php?forum_**name=snort-users<http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users>
>>>
>>> Please visit http://blog.snort.org to stay current on all the latest
>>> Snort news!
>>>
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.snort.org/pipermail/snort-devel/attachments/20121026/51fd372e/attachment.html>


More information about the Snort-devel mailing list