[Snort-sigs] Question about Content-Disposition, Content-Type, etc. and http_header buffer

Joel Esler jesler at ...435...
Thu Oct 25 16:07:40 EDT 2012


Mike,

So, exactly as I described below is what is going on here. 

Looking at the pcap:  (Forgive me for the screen shot, but this is probably the best way to explain it)



According to the RFC, as I understand it, the HTTP header ends when a |0d 0a 0d 0a| is present, indicating the end of the header.  So this is where http_header would end.  The rest would be in http_client_body.  (I underlined the end of header with yellow, and then each subsequent |0d 0a 0d 0a| that would indicate end of header.)  The Blue lines are just end of field.



I took your rules and I put them in my test suite and i got an alert here off of your post-multipart.pcap  (after I fixed the pcap up and wrapped it in a session):

Alerts (outfile.pcap)
1:22222:0       Test for multipart POST -- http_client_body                      Alerts: 1

Am I still missing the point?  Am I insane?  

--
Joel Esler
Senior Research Engineer, VRT
OpenSource Community Manager
Sourcefire



On Oct 25, 2012, at 2:22 PM, Mike Cox <mike.cox52 at ...2420...> wrote:

> Hi Joel,
> 
> I've attached some pcaps to assist you and Sourcefire in understanding this.  Here are some rules for testing as well:
> 
> alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"Test for multipart POST -- http_header"; flow:established,to_server; content:"Content-Disposition"; http_header; content:"form-data"; sid:11111;)
> 
> alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"Test for multipart POST -- http_client_body"; flow:established,to_server; content:"Content-Disposition"; http_client_body; content:"form-data"; sid:22222;)
> 
> I believe that the rules, pcaps, and previous description of the problem should be sufficient for you guys to get a good handle on the issue I am talking about.
> 
> At this point, I do not believe that Snort handles this as you suggest.  Please let me know if I am incorrect about this and what specific version added this support.
> 
> When can we expect http_inspect to be able to handle multi-part headers and make the appropriate data available in the http_* content buffers?  Until then, what is a good solution for this situation?  Should we simply not use the http_* buffers and just do global content matches?
> 
> Thanks for all your help!
> 
> -Mike Cox 
> 
> On Wed, Oct 17, 2012 at 1:18 PM, Joel Esler <jesler at ...435...> wrote:
> Mike,
> 
> I understand what you mean, however the http inspect preprocessor should handle multipart headers.  There's a format that they must adhere to in the RFC as well for that, and I know we handle that as I had to do some testing on that awhile back to ensure that we not only detect but handle that.    So I'd ask you to verify that this is, indeed the case.  (You say "I believe", so I'm just being sure)
> 
> That being said, yes, we are rewriting http_inspect right now and multipart headers are one of the feature requirements. (as the present http_inspect handles it).  I'll ask devel to take a look at this, but my testing and understanding is that we handle it.  
> 
> I know you said you can't get it, but that's why we always ask for a pcap.  We can't recreate every network in the world, and not all devices adhere to RFCs correctly.  Heck, some vendors don't adhere to their own RFC's.  Development will ask me for a pcap from you, as our pcaps that are testing with show we handle this multipart (if that's what it really is) scenario correctly.
> 
> I'm not trying to be difficult, and I'm not arrogant enough to think that there's some scenario out that there that we might not handle.  We're all humans, we know it probably exists, however, given the scenario that you describe, and from my experience, that shouldn't be a problem.
> 
> --
> Joel Esler
> Senior Research Engineer, VRT
> OpenSource Community Manager
> Sourcefire
> 
> 
> On Oct 17, 2012, at 1:04 PM, Mike Cox <mike.cox52 at ...2420...> wrote:
> 
>> Hi Joel,
>> 
>> I appreciate your timely response and feedback.
>> 
>> I believe the issue here is that HTTP headers normally seen in the initial HTTP data are being sent in subsequent packets (as a result of multipart/form-data POSTS) and formatted differently from the standard HTTP header.  I hope I made this clear in previous emails but some simple Google searches would also help illuminate what I'm talking about.  I'd rather not get in to a RFC argument with you (if you really want to we can but, apparently this is a standard format and accepted by both modern browsers and servers).
>> 
>> My concern here is Snort rules that are looking for data normally in the HTTP headers (using the http_header buffer) do not work in the case of some multipart/form data POSTS.  This means that snort rules are not alerting as intended (and introduces an IDS bypass avenue, albeit somewhat paltry in terms of eloquence but nonetheless functional).
>> 
>> Does Sourcefire/VRT,etc. intend to account for this in future http-inspect capabilities or should we modify our current rules to do inefficient, global content matches?  Or is there another solution that I am missing here?  What can we do *now*?
>> 
>> Thanks.
>> 
>> -Mike Cox
>> 
>> 
>> On Wed, Oct 17, 2012 at 10:51 AM, Joel Esler <jesler at ...435...> wrote:
>> Well, from the looks of the "dump" (or whatever you sent below) it looks like there are multiple carriage returns in there.
>> 
>> If the "header" of an http session has |0d 0a 0d 0a|, according to the RFC which states that character sequence indicates the end of the header, and the start of the packet data.  In which case the http_inspect preprocessor will move onto the http_client_data buffer.
>> 
>> Hence why I asked for the pcap, to see if this is true.  We can create a pcap that looks like yours below, but I know the exact behavior of the preprocessor in this case.
>> 
>> If you wanted to test it, you could do a 
>> content:"Content-Type"; http_header; content:"Content-Disposition"; http_header; distance:0;
>> 
>> which you indicated doesn't work.
>> 
>> then test by doing:
>> 
>> content:"Content-Type"; http_header; content:"Content-Disposition"; http_client_body;
>> 
>> and if that fires, then there's your problem.  Broken RFC adherence by whatever program is generating that traffic.
>> 
>> --
>> Joel Esler
>> Senior Research Engineer, VRT
>> OpenSource Community Manager
>> Sourcefire
>> 
>> 
>> On Oct 17, 2012, at 11:45 AM, Mike Cox <mike.cox52 at ...2420...> wrote:
>> 
>>> Hey Joel, thanks for the quick response.  Right now I don't have a pcap ... the info I got was reproduced from what I saw on a screen while helping our guys/gals track down an issue/RCA in the "Temptest" (SOC) room and I can't get any data out of there and if I could, it would be breaking many laws (well, internal "laws" but there could be international intel impact and NDA lawsuits if you know what I mean :)
>>> 
>>> However, I think that the information provided is sufficient to illustrate this issue and it should be trivial for VRT to create necessary pcaps for their testing, if that is what you want.
>>> 
>>> I can cook up a pcap but as earlier implied, I don't think it is is necessary for me to provide this at this point and I certainly do not want to insult VRT by spoon-feeding them data that they can easily and more completely create on their own with minimal effort.
>>> 
>>> I believe that I have outlined the issue and provided the necessary data to accurately describe the problem.  Even without a pcap, I think the info provided is sufficient for a good discussion.  Has anyone else experienced this challenge with their snort installs?  If so, how do you deal with this nuance of HTTP and the current state of http-inspect?
>>> 
>>> Please let me know if you are unclear on anything or need more information regarding this issue.
>>> 
>>> Thanks all.
>>> 
>>> -Mike Cox
>>> 
>>> 
>>> 
>>> On Tue, Oct 16, 2012 at 4:51 PM, Joel Esler <jesler at ...435...> wrote:
>>> Can you send me a pcap?
>>> 
>>> --
>>> Joel Esler
>>> Senior Research Engineer, VRT
>>> OpenSource Community Manager
>>> Sourcefire
>>> 
>>> On Oct 16, 2012, at 5:18 PM, Mike Cox <mike.cox52 at ...2420...> wrote:
>>> 
>>>> I've noticed that in some multipart/form-data POSTs, data that is normally in the HTTP header gets sent in the body of the message and not parsed by http-inspect as part of the http_header buffer.  Specifically, the headers "Content-Type", "Content-Disposition", and "Content-Transfer-Encoding", although there could be others.  For example:
>>>> 
>>>> POST /blackhole/safe.php HTTP/1.1
>>>> Host: snort.org
>>>> Content-Type: multipart/form-data, boundary=---dG91Y2hteXNub3J0
>>>> Content-Length: 8675309
>>>> 
>>>> ---dG91Y2hteXNub3J0
>>>> Content-Disposition: form-data; name="name"
>>>> 
>>>> Joshua
>>>> ---dG91Y2hteXNub3J0
>>>> Content-Disposition: form-data; name="play_a_game"
>>>> 
>>>> True
>>>> ---dG91Y2hteXNub3J0
>>>> Content-Disposition: form-data; name="file"; filename="GLOBAL_THERMONUCLEAR_WAR.pdf"
>>>> Content-Type: application/pdf
>>>> Content-Transfer-Encoding: binary
>>>> ...
>>>> 
>>>> So a snort rule looking for a specific filename in a Content-Disposition header wouldn't match if it were written as you would expect it to be written.  For example, this wouldn't match the above:
>>>> 
>>>> alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"Bad PDF File Upload"; flow:established,to_server; content:"Content-Disposition"; http_header; content:"filename="; distance:0; http_header; content:".pdf"; distance:0; within:100; http_header; sid:1234567;)
>>>> 
>>>> What is the best way to match this and not incur the overhead of using global content matches?  Is there a plan for the http-inspect pre-processor to account for this?
>>>> 
>>>> Thanks.
>>>> 
>>>> -Mike Cox
>>>> ------------------------------------------------------------------------------
>>>> Everyone hates slow websites. So do we.
>>>> Make your web apps faster with AppDynamics
>>>> Download AppDynamics Lite for free today:
>>>> http://p.sf.net/sfu/appdyn_sfd2d_oct_______________________________________________
>>>> Snort-sigs mailing list
>>>> Snort-sigs at lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/snort-sigs
>>>> http://www.snort.org
>>>> 
>>>> 
>>>> Please visit http://blog.snort.org for the latest news about Snort!
>>> 
>>> 
>> 
>> 
> 
> 
> <post-multipart.pcap><post-singlepart.pcap>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.snort.org/pipermail/snort-sigs/attachments/20121025/1524445f/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: header.png
Type: image/png
Size: 108597 bytes
Desc: not available
URL: <https://lists.snort.org/pipermail/snort-sigs/attachments/20121025/1524445f/attachment.png>


More information about the Snort-sigs mailing list