[Snort-devel] Fast-pattern matcher does not honor ignore_data in the SMTP Preprocessor

Joel Esler jesler at ...402...
Wed Feb 22 11:16:27 EST 2012


It's currently slated for 2.9.3, yes.  

As for your suggestions, they are good.  We currently are doing a complete revamp on a couple critical pieces of Snort, (smtp not being one at this time).  But we'll take a look at this.

J

On Feb 21, 2012, at 8:22 PM, Joshua Kinard wrote:

> On 11/28/2011 08:24, Joel Esler wrote:
> 
>> This has been entered into the system as well, just FYI.  It's been a crazy week around here with trying to get 2.9.2 out and the Holiday.
>> 
>> Joel
>> 
>> On Nov 20, 2011, at 9:05 PM, Joshua Kinard wrote:
>> 
>>> 
>>> Hi snort-devel,
>>> 
>>> I discovered a bug between the SMTP preprocessor and the fast_pattern
>>> matcher when 'ignore_data' is turned on.  The issue stems from using a a
>>> rule that has only a single content match with a pattern found only in the
>>> e-mail body.  It seems that the fast-pattern matcher will find this pattern
>>> in the overall packet and flag an event before the SMTP preprocessor has a
>>> chance to fully normalize/decode the packet and apply the 'ignore_data' option.
>>> 
>>> I've attached an SMTP example session from twin Postfix MTAs running on a
>>> Linux VM (over IANA reserved IPs on loopback, all local).  The sender is
>>> "jack" from host "formenos.aman.arda" to "jill" at "tirion.aman.arda" (props
>>> to anyone who gets the references).  The message body uses text generated
>>> from the great Lorem Ipsum generator.  The file attachment is 768K of
>>> /dev/urandom data to simulate a bog-standard SMTP session of message + MIME
>>> attachment.
>>> 
>>> This is my sample SMTP Preprocessor config on snort-2.9.2.  Assume I have
>>> the other preprocessors set to sane defaults:
>>> 
>>> preprocessor smtp:                  \
>>>       normalize all               \
>>>       inspection_type stateful    \
>>>       ports { 25 587 }            \
>>>       ignore_data                 \
>>>       ignore_tls_data             \
>>>       no_alerts                   \
>>>       b64_decode_depth 0          \
>>>       max_mime_mem 2097152
>>> 
>>> 
>>> Given the above, this rule below WILL alert on the specified pattern (found
>>> in the third Lorem Ipsum paragraph, last sentence), regardless of the
>>> 'ignore_data' directive:
>>> 
>>> alert tcp any any -> any 25 (                    \
>>>       msg:"Lorem Ipsum?";                      \
>>>       flow:established,to_server;              \
>>>       content:"sollicitudin"; nocase;          \
>>>       sid:42000009; rev:1; priority:1;         \
>>>       classtype:bad-unknown;                   \
>>> )
>>> 
>>> This is because the fast-pattern matcher finds "sollicitudin" in the
>>> unreassembled packets.  I believe the SMTP preprocessor has some done kind
>>> of processing by this point (I stepped through the code using gdb/ddd, and
>>> the alert happens after the 8th TCP packet), but I guess a check somewhere
>>> misses or fast-pattern simply doesn't handle the 'ignore_data' directive
>>> properly.
>>> 
>>> 
>>> A workaround I found was to short-circuit the fast-pattern matcher with a
>>> content match you ALWAYS expect to be in an SMTP packet, like "Received:":
>>> 
>>> alert tcp any any -> any 25 (                    \
>>>       msg:"Lorem Ipsum?";                      \
>>>       flow:established,to_server;              \
>>>       content:"Received|3a|"; fast_pattern;    \
>>>       content:"sollicitudin"; nocase;          \
>>>       sid:42000009; rev:1; priority:1;         \
>>>       classtype:bad-unknown;                   \
>>> )
>>> 
>>> With this, we short-circuit fast-pattern because it will find "Received:",
>>> then the SMTP preprocessor does what it does, and the second content match
>>> fails as expected because we want to ignore the DATA portion of the e-mail.
>>> 
>>> It's a hack, though, and you're giving up performance by the short-circuit.
>>> I honestly don't know a good way around this with the current design of the
>>> SMTP preprocessor.  Other dynamic preprocessors, like SIP, DCE, the new GTP
>>> ones, all provide functions to tell the engine where to look.  My guess is
>>> an equivalent 'smtp_header' and 'smtp_body' set of rule options might be
>>> needed (while 'file_data' continues to point at decoded MIME data), so that
>>> the fast-pattern matcher can be pointed to the appropriate decode buffer and
>>> not generate potential false positives.
> 
> 
> Curious to know if there's been any updates on this bug.  Is this slated to
> be fixed in 2.9.3?  Or has any further thought been given to crating
> 'smtp_header' and 'smtp_body' rule options to better control the
> fast-pattern matcher?
> 
> I've looked at the code in the SMTP preprocessor and isolated the specific
> checks that indicate why fast-pattern doesn't honor ignore_data.  I suspect
> that, basing off of the new modbus_data and dnp3_data options, replacing the
> bits for ignore_data with instructions to copy the SMTP header to one decode
> buffer and the body to another for use within in a rule might not be too
> much of an issue if I get some free time.  Any tips on other code points to
> look at (outside of adding the actual rule option hooks and such)?
> 
> Example -- we avoid the fast-pattern bug and positional checks:
> alert tcp any any -> any 25 (msg:"Spam E-mail from Bob";
> flow:established,to_server; smtp_header; content:"From|3a 20|Bob Bob
> <bob at ...3259...>"; smtp_body;
> content:"http|3a|//clicktracker.bob.com/spam?id="; sid:xxxxxxxx; rev:1;
> classtype:policy-violation; )
> 
> Arguably, having these two options would make the SMTP Preprocessor far more
> useful.  They would move the doe_ptr, so they would go before a content, not
> after.  Even better would be creating buffers for From, To, and Subject,
> then dump the rest of the header into the smtp_header buffer.
> 
> 
> Thanks!
> 
> -- 
> Joshua Kinard
> Gentoo/MIPS
> kumba at ...2185...
> 4096R/D25D95E3 2011-03-28
> 
> "The past tempts us, the present confuses us, the future frightens us.  And
> our lives slip away, moment by moment, lost in that vast, terrible in-between."
> 
> --Emperor Turhan, Centauri Republic
> 





More information about the Snort-devel mailing list