[Snort-devel] BUG: corner case involving http_cookie

Will Metcalf william.metcalf at ...2499...
Wed Mar 17 12:05:50 EDT 2010


This is just my 2 cents, but I don't think the following behavior
makes sense.  I think that even though you are providing http_cookie
as a separate buffer to match on it should still be included in the
http_header buffer, well because it is part of the headers.

You can still match using the raw buffer but then you have to add
additional checks to try and differentiate between the headers and the
body which is why I'm guessing these modifiers were created in the
first place.  I realize that in most cases header order doesn't matter
but there may be instances where you can fingerprint a piece of
automated code (read malware) using a rule similar to sid 69 below.

Regards,

Will

#this matches but I loose the performance/accuracy benefit of only
matching within the buffer containing http_headers.
alert tcp any any -> any any (msg:"http_cookie + ";
content:"Cookie|3A|
e6504ae48c99f09df7f58996aacbb6b0=120e494ce857d6ceeef89f9678d4d703|0d
0a|Content-Type|3A| application"; classtype:bad-unknown; sid:68;
rev:1;)

#this fails to match as the cookie is not part of the http_header
buffer but is part of the real http headers.
alert tcp any any -> any any (msg:"http_cookie + ";
content:"Cookie|3A|
e6504ae48c99f09df7f58996aacbb6b0=120e494ce857d6ceeef89f9678d4d703|0d
0a|Content-Type|3A| application"; http_header; classtype:bad-unknown;
sid:69; rev:1;)


On Mon, Mar 15, 2010 at 10:08 AM, Will Metcalf
<william.metcalf at ...2499...> wrote:
> Thanks Steve!
>
> Regards,
>
> Will
>
> On Mon, Mar 15, 2010 at 10:06 AM, Steven Sturges
> <steve.sturges at ...402...> wrote:
>> Will et al--
>>
>> We're looking into these issues... Will follow up in the next
>> day or so.
>>
>> Cheers
>> -steve
>>
>> Will Metcalf wrote:
>>> Seems using http_(cookie|header) with isdataat acts the same way
>>> src/snort -k none -q -A console -c etc/snort.conf -l ./ -r
>>> /home/coz/oisfsearchnums.pcap
>>>
>>> #these rules fail to fire even though the isdataat check isn't
>>> relative client/server flow depth both set to 0
>>> #alert tcp any any -> any any (msg:"e6504ae48c99f09df7f58996aacbb6b0 +
>>> http_cookie + isdataat"; content:"e6504ae48c99f09df7f58996aacbb6b0";
>>> http_cookie; isdataat:596; classtype:bad-unknown; sid:104; rev:1;)
>>> #alert tcp any any -> any any (msg:"our website host header +
>>> http_header + isdataat"; content:"Host|3A|
>>> www.openinfosecfoundation.org"; http_header; isdataat:596;
>>> classtype:bad-unknown; sid:105; rev:1;)
>>>
>>>
>>> On Thu, Mar 11, 2010 at 10:02 AM, Will Metcalf
>>> <william.metcalf at ...2499...> wrote:
>>>> Fails on byte_jump as well, additionally http_header appears to act
>>>> the same way.
>>>>
>>>> Regards,
>>>>
>>>> Will
>>>>
>>>> #test 73 http_cookie + byte_jump.
>>>> #hmm interesting using http_cookie in combination with byte_jump seems
>>>> to fail always. Removing either the byte_jump check or the http_cookie
>>>> modifier will cause this sig to fire.  Notice that the byte_jump
>>>> #check isn't even relative to the content match.
>>>> #
>>>> #file oisfsearchnums.pcap
>>>> alert tcp any any -> any any (msg:"e6504ae48c99f09df7f58996aacbb6b0 +
>>>> http_cookie + byte_jump"; content:"e6504ae48c99f09df7f58996aacbb6b0";
>>>> http_cookie; byte_jump:1,596,string,dec; content:"0"; distance:0;
>>>> within:1; classtype:bad-unknown; sid:73; rev:1;)
>>>>
>>>>
>>>>
>>>> On Wed, Mar 10, 2010 at 4:56 PM, Will Metcalf <william.metcalf at ...3051......> wrote:
>>>>> How about this one, using the same pcap. I have both client and server
>>>>> flow_depth set to 0.
>>>>>
>>>>> #test 72 http_cookie + byte_test.
>>>>> #hmm interesting using http_cookie in combination with byte_test seems
>>>>> to fail always. Removing either the byte_test check or the http_cookie
>>>>> modifier will cause this sig to fire.  Notice that the byte_test
>>>>> #check isn't even relative to the content match.
>>>>> #
>>>>> #file oisfsearchnums.pcap
>>>>> alert tcp any any -> any any (msg:"e6504ae48c99f09df7f58996aacbb6b0 +
>>>>> http_cookie + byte_test"; content:"e6504ae48c99f09df7f58996aacbb6b0";
>>>>> http_cookie; byte_test:2,=,12,596,string,dec; classtype:bad-unknown;
>>>>> sid:72; rev:1;)
>>>>>
>>>>>
>>>>> Regards,
>>>>>
>>>>> Will
>>>>>
>>>>> On Wed, Mar 10, 2010 at 1:54 PM, Will Metcalf <william.metcalf at ...2840...9...> wrote:
>>>>>> I can see the need for the cut-off.  You have to make compromises
>>>>>> between accuracy and performance this tends to be the nature of the
>>>>>> beast with IDS's,  and sure there will always be evasions abut the
>>>>>> dynamic flow_depth thing still isn't making much sense to me and/or at
>>>>>> least this behavior should be documented somewhere.  Did I just miss
>>>>>> something in the readme?  Btw there is a typo both in the manual and
>>>>>> the README regarding client_flow_depth.  "It primarily eliminates
>>>>>> Snort fro inspecting larger HTTP Cookies that appear at the end of
>>>>>> many client request Headers."
>>>>>>
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Will
>>>>>>
>>>>>> On Wed, Mar 10, 2010 at 1:18 PM, Steven Sturges
>>>>>> <steve.sturges at ...402...> wrote:
>>>>>>> There are different settings for the requests & responses.
>>>>>>> See client_flow_depth and server_flow_depth in the readme/manual.
>>>>>>>
>>>>>>> Setting either of the flow depth values to 0 would inspect
>>>>>>> all of the respective request or response, but incurs a
>>>>>>> pretty big performance hit.
>>>>>>>
>>>>>>> And yes, the default (in code) is 300 for both.
>>>>>>>
>>>>>>> Will Metcalf wrote:
>>>>>>>> Ahhh ok so I want to make sure I understand correctly in 2.8.5 the
>>>>>>>> default 300 byte cutoff is not only applied to http_responses but also
>>>>>>>> http_requests (README.http_inspect) if there are http normalizations.
>>>>>>>> Is there not still a potential evasion here based on the tests below
>>>>>>>> as packet being normalized or potentially matching on one or more
>>>>>>>> rules can be fairly unpredictable?  Would your recommended
>>>>>>>> configuration then be to set client_flow_depth to 0 in the
>>>>>>>> http_inspect preproc for 2.8.5 users if they wanted to avoid the
>>>>>>>> potential evasion?
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>
>>>>>>>> Will
>>>>>>>>
>>>>>>>> On Wed, Mar 10, 2010 at 12:03 PM, Steven Sturges
>>>>>>>> <steve.sturges at ...402...> wrote:
>>>>>>>>> Seems that what is really coming into play is the flow depth.
>>>>>>>>> That is going to limit how much of the raw data is searched
>>>>>>>>> with the pattern matcher.
>>>>>>>>>
>>>>>>>>> To summarize how it works with 2.8.6:
>>>>>>>>>
>>>>>>>>> 1) If HTTP normalizes headers and cookies, the amount of raw data
>>>>>>>>> searched by fast pattern matcher is determined by client (or server)
>>>>>>>>> flow depth settings.
>>>>>>>>>
>>>>>>>>> 2) If there are no HTTP normalizations, flow depth is not applied, so
>>>>>>>>> all of the raw data is searched.
>>>>>>>>>
>>>>>>>>> With 2.8.5, in 1) above, the flow depth was applied only if there were
>>>>>>>>> rules searched for one of those HTTP normalized buffers.
>>>>>>>>>
>>>>>>>>> By default, client flow depth is 0 -- which is what Will is probably
>>>>>>>>> hitting in his tests.
>>>>>>>>>
>>>>>>>>> Alerts w/ combinations of rules & different flow depths in 2.8.6:
>>>>>>>>>
>>>>>>>>> Default flow depth (0)
>>>>>>>>> SID 59, SID 68 enabled
>>>>>>>>> 1 alert (sid 59)
>>>>>>>>>
>>>>>>>>> Default flow depth (0)
>>>>>>>>> SID 68 enabled
>>>>>>>>> 0 alerts
>>>>>>>>>
>>>>>>>>> [Same as above for client_flow_depth 300]
>>>>>>>>>
>>>>>>>>> Client flow depth (1460)
>>>>>>>>> SID 59, SID 68 enabled
>>>>>>>>> 3 alerts (sid 59, sid 68x2)
>>>>>>>>>
>>>>>>>>> Client flow depth (1460)
>>>>>>>>> SID 68 enabled
>>>>>>>>> 2 alerts (sid 68x2)
>>>>>>>>>
>>>>>>>>> -steve
>>>>>>>>>
>>>>>>>>> Matt Jonkman wrote:
>>>>>>>>>> Appreciate the clarification Steve. But I'm concerned, this will make
>>>>>>>>>> hundreds if not a few thousand rules not work correctly in our set and
>>>>>>>>>> in vrt/snort gpl. If someone has http_inspect on in a recent snort, but
>>>>>>>>>> does NOT have ALL of their http related rules converted to the new form
>>>>>>>>>> using the http_* modifiers (which we have none converted) then they're
>>>>>>>>>> going to have massive problems, no? I think a lot of people are missing
>>>>>>>>>> a lot of things right now.
>>>>>>>>>>
>>>>>>>>>> Is there a way to make http_inspect not do this? We have a lot of sigs
>>>>>>>>>> that won't work on normalized data, for instance the recent trojan sigs
>>>>>>>>>> that look at the order of the parameters in an http post for uniqieness.
>>>>>>>>>> With the normalized only we can't see that can we?
>>>>>>>>>>
>>>>>>>>>> Thanks!
>>>>>>>>>>
>>>>>>>>>> Matt
>>>>>>>>>>
>>>>>>>>>> On 3/10/10 10:26 AM, Will Metcalf wrote:
>>>>>>>>>>>> 1) Use http_cookie in the rule as you note in the case that works.
>>>>>>>>>>> Right so the reason that I cc'd the emerging list is that they are not
>>>>>>>>>>> using the http_* modifiers to maintain compatibility with older
>>>>>>>>>>> versions of snort.  Shouldn't this buffer at least be available to
>>>>>>>>>>> match on via rawbytes  which is what would be consistent with what you
>>>>>>>>>>> have done with telnet and dcerpc.  Does this also not add a potential
>>>>>>>>>>> evasion method if this is the intended behavior, one that perhaps
>>>>>>>>>>> VRT/ET should be made aware of.  I haven't gone through very many of
>>>>>>>>>>> the sigs but since the normalized buffer begins with the "Cookie:"
>>>>>>>>>>> instead of the value isn't there a now a potential evasion if I can
>>>>>>>>>>> get another sig to trip on the same packet as say sid 2136.
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>>
>>>>>>>>>>> Will
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Mar 10, 2010 at 8:15 AM, Steven Sturges
>>>>>>>>>>> <steve.sturges at ...402...> wrote:
>>>>>>>>>>>> Will--
>>>>>>>>>>>>
>>>>>>>>>>>> In the 2nd rule the only content is an HTTP cookie.  Without using
>>>>>>>>>>>> http_cookie, it would try to match the raw data.
>>>>>>>>>>>>
>>>>>>>>>>>> With HTTP Inspect enabled, it is separating the headers, cookie,
>>>>>>>>>>>> method, etc from the raw data, hence all of the modifier keywords
>>>>>>>>>>>> that you can use with content -- can use more than one together.
>>>>>>>>>>>>
>>>>>>>>>>>> Without specifying http_cookie in the rule and when HTTP Inspect
>>>>>>>>>>>> enabled (and cookie inspection enabled in 2.8.6), if the pattern
>>>>>>>>>>>> matcher searches any of the HTTP buffers, it doesn't search the raw
>>>>>>>>>>>> data.  If no rules use the specific HTTP buffers or there are no HTTP
>>>>>>>>>>>> buffers, the pattern matcher will search the raw data, which is why
>>>>>>>>>>>> the rule byte itself (without http_cookie) works.
>>>>>>>>>>>>
>>>>>>>>>>>> This was done to avoid going over the same data twice in the pattern
>>>>>>>>>>>> matcher, and it is working as it is designed.
>>>>>>>>>>>>
>>>>>>>>>>>> Two options:
>>>>>>>>>>>>
>>>>>>>>>>>> 1) Use http_cookie in the rule as you note in the case that works.
>>>>>>>>>>>>
>>>>>>>>>>>> 2) Turn off HTTP Inspect (not realistic)
>>>>>>>>>>>>
>>>>>>>>>>>> Cheers
>>>>>>>>>>>> -steve
>>>>>>>>>>>>
>>>>>>>>>>>> Will Metcalf wrote:
>>>>>>>>>>>>> hmmm I don't think so.  Look at first test.  both rules fire.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Will
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Mar 9, 2010 at 10:31 PM, beenph <beenph at ...2499...> wrote:
>>>>>>>>>>>>>> I will try a wild guess, what is your event_queue size like?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Its probably a bug or something  that need clarification regarding
>>>>>>>>>>>>>> http_cookie and http_inspect, but mabey http_cookie enable a modifier
>>>>>>>>>>>>>> in http_inspect that alter alerting behavior when event_queue is at 1
>>>>>>>>>>>>>> (since i guess both "alerts" are part of the same normalized http
>>>>>>>>>>>>>> stream)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> -elz
>>>>>>>>>>>>>> ps: didin't run the pcap and rules test.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Mar 9, 2010 at 11:15 PM, Will Metcalf <william.metcalf at ...2499...> wrote:
>>>>>>>>>>>>>>> failing to use the http_cookie modifier on a rule where there is
>>>>>>>>>>>>>>> another rule that matches the same packet makes a rule that should
>>>>>>>>>>>>>>> fire fail.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> src/snort -V
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>   ,,_     -*> Snort! <*-
>>>>>>>>>>>>>>>  o"  )~   Version 2.8.5.3 (Build 124)
>>>>>>>>>>>>>>>   ''''    By Martin Roesch & The Snort Team:
>>>>>>>>>>>>>>> http://www.snort.org/snort/snort-team
>>>>>>>>>>>>>>>           Copyright (C) 1998-2009 Sourcefire, Inc., et al.
>>>>>>>>>>>>>>>           Using PCRE version: 7.8 2008-09-05
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> src/snort -k none -q -A console -c etc/snort.conf -l ./ -r oisfsearchnums.pcap
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> #this combo works
>>>>>>>>>>>>>>> #alert tcp any any -> any any (msg:"http_client_body";
>>>>>>>>>>>>>>> content:"searchword="; uricontent:"/index.php"; nocase;
>>>>>>>>>>>>>>> classtype:bad-unknown; sid:59; rev:1;)
>>>>>>>>>>>>>>> #alert tcp any any -> any any (msg:"http_cookie match ";
>>>>>>>>>>>>>>> content:"e6504ae48c99f09df7f58996aacbb6b0=120e494ce857d6ceeef89f9678d4d703";
>>>>>>>>>>>>>>> http_cookie; classtype:bad-unknown; sid:68; rev:1;)
>>>>>>>>>>>>>>> #
>>>>>>>>>>>>>>> #03/07-21:19:54.242506  [**] [1:59:1] http_client_body [**]
>>>>>>>>>>>>>>> [Classification: Potentially Bad Traffic] [Priority: 2] {TCP}
>>>>>>>>>>>>>>> 192.168.100.17:38111 -> 96.43.130.5:80
>>>>>>>>>>>>>>> #03/07-21:19:54.242506  [**] [1:68:1] http_cookie match  [**]
>>>>>>>>>>>>>>> [Classification: Potentially Bad Traffic] [Priority: 2] {TCP}
>>>>>>>>>>>>>>> 192.168.100.17:38111 -> 96.43.130.5:80
>>>>>>>>>>>>>>> #03/07-21:19:54.364173  [**] [1:68:1] http_cookie match  [**]
>>>>>>>>>>>>>>> [Classification: Potentially Bad Traffic] [Priority: 2] {TCP}
>>>>>>>>>>>>>>> 192.168.100.17:38111 -> 96.43.130.5:80
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> #the second rule does not fire
>>>>>>>>>>>>>>> #alert tcp any any -> any any (msg:"http_client_body + depth";
>>>>>>>>>>>>>>> content:"searchword="; uricontent:"/index.php"; nocase;
>>>>>>>>>>>>>>> classtype:bad-unknown; sid:59; rev:1;)
>>>>>>>>>>>>>>> #alert tcp any any -> any any (msg:"http_cookie match";
>>>>>>>>>>>>>>> content:"e6504ae48c99f09df7f58996aacbb6b0=120e494ce857d6ceeef89f9678d4d703";
>>>>>>>>>>>>>>> classtype:bad-unknown; sid:68; rev:1;)
>>>>>>>>>>>>>>> #
>>>>>>>>>>>>>>> #03/07-21:19:54.242506  [**] [1:59:1] http_client_body + depth [**]
>>>>>>>>>>>>>>> [Classification: Potentially Bad Traffic] [Priority: 2] {TCP}
>>>>>>>>>>>>>>> 192.168.100.17:38111 -> 96.43.130.5:80
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> #this rule fires when used on it's own.
>>>>>>>>>>>>>>> #alert tcp any any -> any any (msg:"http_cookie match";
>>>>>>>>>>>>>>> content:"e6504ae48c99f09df7f58996aacbb6b0=120e494ce857d6ceeef89f9678d4d703";
>>>>>>>>>>>>>>> classtype:bad-unknown; sid:68; rev:1;)
>>>>>>>>>>>>>>> #
>>>>>>>>>>>>>>> #03/07-21:19:54.242506  [**] [1:68:1] http_cookie match [**]
>>>>>>>>>>>>>>> [Classification: Potentially Bad Traffic] [Priority: 2] {TCP}
>>>>>>>>>>>>>>> 192.168.100.17:38111 -> 96.43.130.5:80
>>>>>>>>>>>>>>> #03/07-21:19:54.364173  [**] [1:68:1] http_cookie match [**]
>>>>>>>>>>>>>>> [Classification: Potentially Bad Traffic] [Priority: 2] {TCP}
>>>>>>>>>>>>>>> 192.168.100.17:38111 -> 96.43.130.5:80
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>>>>>>>>> Download Intel® Parallel Studio Eval
>>>>>>>>>>>>>>> Try the new software tools for yourself. Speed compiling, find bugs
>>>>>>>>>>>>>>> proactively, and fine-tune applications for parallel performance.
>>>>>>>>>>>>>>> See why Intel Parallel Studio got high marks during beta.
>>>>>>>>>>>>>>> http://p.sf.net/sfu/intel-sw-dev
>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>> Snort-devel mailing list
>>>>>>>>>>>>>>> Snort-devel at lists.sourceforge.net
>>>>>>>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/snort-devel
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>>>>>>> Download Intel® Parallel Studio Eval
>>>>>>>>>>>>> Try the new software tools for yourself. Speed compiling, find bugs
>>>>>>>>>>>>> proactively, and fine-tune applications for parallel performance.
>>>>>>>>>>>>> See why Intel Parallel Studio got high marks during beta.
>>>>>>>>>>>>> http://p.sf.net/sfu/intel-sw-dev
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> Snort-devel mailing list
>>>>>>>>>>>>> Snort-devel at lists.sourceforge.net
>>>>>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/snort-devel
>>>>>>>>>>>>>
>>>>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>>>>> Download Intel® Parallel Studio Eval
>>>>>>>>>>> Try the new software tools for yourself. Speed compiling, find bugs
>>>>>>>>>>> proactively, and fine-tune applications for parallel performance.
>>>>>>>>>>> See why Intel Parallel Studio got high marks during beta.
>>>>>>>>>>> http://p.sf.net/sfu/intel-sw-dev
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Snort-devel mailing list
>>>>>>>>>>> Snort-devel at lists.sourceforge.net
>>>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/snort-devel
>>>
>>
>




More information about the Snort-devel mailing list