[Snort-users] [Emerging-Sigs] Reliability of signatures

Martin Roesch roesch at ...1935...
Fri Feb 11 13:49:09 EST 2011


There was a lot of discussion around false positive definitions and
solutions back in 2003.  I had a couple long posts on the focus-ids
mailing lists that discussed the topic from my viewpoint:

http://marc.info/?l=focus-ids&m=106061353102964&w=4

http://marc.info/?l=focus-ids&m=106688133229359&w=4

There was a fair amount of conversation on the lists around both of
those posts but I think they're still relevant.  We just have ~7+
years more experience with the systems and we know that automated or
manual contextualization is really necessary to get best use out of
IDS/IPS these days but it's interesting to go back and have a look at
what we were talking about back then (and how little things change
sometimes). :)

Marty

On Thu, Feb 10, 2011 at 10:33 AM, Jacob Kitchel <jacob.kitchel at ...11827...> wrote:
> On Thu, Feb 10, 2011 at 9:17 AM, Matthew Jonkman
> <jonkman at ...15020...> wrote:
>> I agree with Matt here completely, (see previous thread about the greatness
>> of Matts).
>> But we both have the point of view of signature writers, whereas an
>> analyst/incident responder in the field has a very different idea of an FP.
>
> That may be true, that every person in the field has a different idea
> of what an FP is. However, it doesn't mean they're all right. There is
> one definition of a False Positive. As complicated as our environments
> are we can't afford to use multiple definitions of the same concept
> because it leads to confusion and lack of specificity when describing
> a problem (among other things).
>
> False Positive = when a sig fires on network traffic that is not what
> it was intended to fire upon
> "Does not apply to this host" or lowered severity = when a valid sig
> fires on network traffic as the sig writer intended it to but the
> network traffic is going toward a destination which is not running the
> affected software/service/whatever.
>
> If the viewing/management mechanism used to review the alert record
> has the ability to evaluate and mark severity, that's a good thing.
> But if it doesn't, then it has to exist in the analyst's brain and the
> analyst has to make the right decision.
>
> -Jacob
>
>> So, just like there are 2 philosophies in how to tune a ruleset (i.e. only
>> fire on compromises vs I want to know who's making attempts and block), I
>> think we have to just accept that there are many definitions of a false
>> positive, and they're all valid.
>> And thus a "Report this as an FP" button is wildly complicated....
>> Matt
>> On Feb 10, 2011, at 10:04 AM, Matt Olney wrote:
>>
>> And I would argue that "no iis" here isn't a valid FP.  The signature
>> performed correctly and notified you that a scan attempt was under way.  It
>> is up to the system admin to correctly suppress/disable/modify rules that do
>> not target his network.  In our view, a FP only occurs when network traffic
>> triggers an alert that is specifically NOT traffic that the rule was
>> intended to fire on.  The rules are application/server agnostic (some wiggle
>> room in this comment both currently and in the future) they are solely based
>> on the traffic on the wire.
>>
>> On Thu, Feb 10, 2011 at 10:00 AM, Michael Scheidell
>> <michael.scheidell at ...8144...> wrote:
>>>
>>>
>>> On 2/10/11 9:55 AM, Matt Olney wrote:
>>>
>>>  Also, SPAM isn't an IDS issue, at
>>>
>>> ah, maybe I should have explained.  you missed the point.
>>>
>>> there needs to be a definition of FP vs MP for signatures for any of this
>>> to mean anything.
>>> one mans SPAM is another mans HAM,  one mans FP is another mans legit hit.
>>> would you BLOCK on FP #2? maybe not inline, but I sure would blacklist the
>>> ip.
>>>
>>> maybe in the 'human verified checkbox' you give them the ability to mark:
>>>
>>> [ ] FP:  signature too broad. matched legit traffic
>>> [ ] FP:  no IIS servers here,
>>>
>>> (just so we have something for them to check)
>>>
>>>
>>> --
>>> Michael Scheidell, CTO
>>> o: 561-999-5000
>>> d: 561-948-2259
>>> ISN: 1259*1300
>>> > | SECNAP Network Security Corporation
>>>
>>> Certified SNORT Integrator
>>> 2008-9 Hot Company Award Winner, World Executive Alliance
>>> Five-Star Partner Program 2009, VARBusiness
>>> Best in Email Security,2010: Network Products Guide
>>> King of Spam Filters, SC Magazine 2008
>>>
>>> ________________________________
>>>
>>> This email has been scanned and certified safe by SpammerTrap®.
>>> For Information please see http://www.secnap.com/products/spammertrap/
>>>
>>> ________________________________
>>
>> ------------------------------------------------------------------------------
>> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
>> Pinpoint memory and threading errors before they happen.
>> Find and fix more than 250 security defects in the development cycle.
>> Locate bottlenecks in serial and parallel code that limit performance.
>> http://p.sf.net/sfu/intel-dev2devfeb_______________________________________________
>> Snort-users mailing list
>> Snort-users at lists.sourceforge.net
>> Go to this URL to change user options or unsubscribe:
>> https://lists.sourceforge.net/lists/listinfo/snort-users
>> Snort-users list archive:
>> http://www.geocrawler.com/redir-sf.php3?list=snort-users
>>
>> ----------------------------------------------------
>> Matthew Jonkman
>> Emergingthreats.net
>> Emerging Threats Pro
>> Open Information Security Foundation (OISF)
>> Phone 765-807-8630
>> Fax 312-264-0205
>> http://www.emergingthreatspro.com
>> http://www.openinfosecfoundation.org
>> ----------------------------------------------------
>>
>> PGP: http://www.jonkmans.com/mattjonkman.asc
>>
>>
>>
>>
>> _______________________________________________
>> Emerging-sigs mailing list
>> Emerging-sigs at ...14333...
>> http://lists.emergingthreats.net/mailman/listinfo/emerging-sigs
>>
>> Support Emerging Threats! Subscribe to Emerging Threats Pro
>> http://www.emergingthreatspro.com
>> The ONLY place to get complete premium rulesets for Snort 2.4.0 through
>> Current!
>>
>
> ------------------------------------------------------------------------------
> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
> Pinpoint memory and threading errors before they happen.
> Find and fix more than 250 security defects in the development cycle.
> Locate bottlenecks in serial and parallel code that limit performance.
> http://p.sf.net/sfu/intel-dev2devfeb
> _______________________________________________
> Snort-users mailing list
> Snort-users at lists.sourceforge.net
> Go to this URL to change user options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://www.geocrawler.com/redir-sf.php3?list=snort-users
>



-- 
Martin Roesch - Founder/CTO, Sourcefire Inc. - +1-410-290-1616
Sourcefire - Security for the Real World - http://www.sourcefire.com
Snort: Open Source IDP - http://www.snort.org




More information about the Snort-users mailing list