[Snort-devel] Fast-pattern matcher does not honor ignore_data in the SMTP Preprocessor

Joshua Kinard kumba at ...2185...
Tue Feb 21 20:22:50 EST 2012

On 11/28/2011 08:24, Joel Esler wrote:

> This has been entered into the system as well, just FYI.  It's been a crazy week around here with trying to get 2.9.2 out and the Holiday.
> Joel
> On Nov 20, 2011, at 9:05 PM, Joshua Kinard wrote:
>> Hi snort-devel,
>> I discovered a bug between the SMTP preprocessor and the fast_pattern
>> matcher when 'ignore_data' is turned on.  The issue stems from using a a
>> rule that has only a single content match with a pattern found only in the
>> e-mail body.  It seems that the fast-pattern matcher will find this pattern
>> in the overall packet and flag an event before the SMTP preprocessor has a
>> chance to fully normalize/decode the packet and apply the 'ignore_data' option.
>> I've attached an SMTP example session from twin Postfix MTAs running on a
>> Linux VM (over IANA reserved IPs on loopback, all local).  The sender is
>> "jack" from host "formenos.aman.arda" to "jill" at "tirion.aman.arda" (props
>> to anyone who gets the references).  The message body uses text generated
>> from the great Lorem Ipsum generator.  The file attachment is 768K of
>> /dev/urandom data to simulate a bog-standard SMTP session of message + MIME
>> attachment.
>> This is my sample SMTP Preprocessor config on snort-2.9.2.  Assume I have
>> the other preprocessors set to sane defaults:
>> preprocessor smtp:                  \
>>        normalize all               \
>>        inspection_type stateful    \
>>        ports { 25 587 }            \
>>        ignore_data                 \
>>        ignore_tls_data             \
>>        no_alerts                   \
>>        b64_decode_depth 0          \
>>        max_mime_mem 2097152
>> Given the above, this rule below WILL alert on the specified pattern (found
>> in the third Lorem Ipsum paragraph, last sentence), regardless of the
>> 'ignore_data' directive:
>> alert tcp any any -> any 25 (                    \
>>        msg:"Lorem Ipsum?";                      \
>>        flow:established,to_server;              \
>>        content:"sollicitudin"; nocase;          \
>>        sid:42000009; rev:1; priority:1;         \
>>        classtype:bad-unknown;                   \
>> )
>> This is because the fast-pattern matcher finds "sollicitudin" in the
>> unreassembled packets.  I believe the SMTP preprocessor has some done kind
>> of processing by this point (I stepped through the code using gdb/ddd, and
>> the alert happens after the 8th TCP packet), but I guess a check somewhere
>> misses or fast-pattern simply doesn't handle the 'ignore_data' directive
>> properly.
>> A workaround I found was to short-circuit the fast-pattern matcher with a
>> content match you ALWAYS expect to be in an SMTP packet, like "Received:":
>> alert tcp any any -> any 25 (                    \
>>        msg:"Lorem Ipsum?";                      \
>>        flow:established,to_server;              \
>>        content:"Received|3a|"; fast_pattern;    \
>>        content:"sollicitudin"; nocase;          \
>>        sid:42000009; rev:1; priority:1;         \
>>        classtype:bad-unknown;                   \
>> )
>> With this, we short-circuit fast-pattern because it will find "Received:",
>> then the SMTP preprocessor does what it does, and the second content match
>> fails as expected because we want to ignore the DATA portion of the e-mail.
>> It's a hack, though, and you're giving up performance by the short-circuit.
>> I honestly don't know a good way around this with the current design of the
>> SMTP preprocessor.  Other dynamic preprocessors, like SIP, DCE, the new GTP
>> ones, all provide functions to tell the engine where to look.  My guess is
>> an equivalent 'smtp_header' and 'smtp_body' set of rule options might be
>> needed (while 'file_data' continues to point at decoded MIME data), so that
>> the fast-pattern matcher can be pointed to the appropriate decode buffer and
>> not generate potential false positives.

Curious to know if there's been any updates on this bug.  Is this slated to
be fixed in 2.9.3?  Or has any further thought been given to crating
'smtp_header' and 'smtp_body' rule options to better control the
fast-pattern matcher?

I've looked at the code in the SMTP preprocessor and isolated the specific
checks that indicate why fast-pattern doesn't honor ignore_data.  I suspect
that, basing off of the new modbus_data and dnp3_data options, replacing the
bits for ignore_data with instructions to copy the SMTP header to one decode
buffer and the body to another for use within in a rule might not be too
much of an issue if I get some free time.  Any tips on other code points to
look at (outside of adding the actual rule option hooks and such)?

Example -- we avoid the fast-pattern bug and positional checks:
alert tcp any any -> any 25 (msg:"Spam E-mail from Bob";
flow:established,to_server; smtp_header; content:"From|3a 20|Bob Bob
<bob at ...3259...>"; smtp_body;
content:"http|3a|//clicktracker.bob.com/spam?id="; sid:xxxxxxxx; rev:1;
classtype:policy-violation; )

Arguably, having these two options would make the SMTP Preprocessor far more
useful.  They would move the doe_ptr, so they would go before a content, not
after.  Even better would be creating buffers for From, To, and Subject,
then dump the rest of the header into the smtp_header buffer.


Joshua Kinard
kumba at ...2185...
4096R/D25D95E3 2011-03-28

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 834 bytes
Desc: OpenPGP digital signature
URL: <https://lists.snort.org/pipermail/snort-devel/attachments/20120221/4722667f/attachment.sig>

More information about the Snort-devel mailing list