[Snort-devel] SMTP preprocessor: packet reassembly / fails to detect switch to TLS (STARTTLS)

Bram bram-fabeg at ...3414...
Fri Aug 30 03:09:24 EDT 2013


Hi,


There appears to be a problem in the SMTP preprocessor: it fails to  
correctly detect the switch to TLS (via the STARTTLS command).
This failure only happens when packets are received out of order/when  
a packet got lost (and was retransmitted later).

This was detected due to some false positives: it failed to detect the  
switch to TLS which means that some alerts were still generated.
Attached are several dump files, explanation of them will follow later  
in this mail.
These dump files were created using raw sockets and use 'port 5555'  
and not 'port 25' --> be sure to specify port 5555 for the  
'stream5_tcp' preprocessor and the 'smtp' preprocessor.

All tests were done using the config file:
         dynamicpreprocessor directory  
/usr/lib/ips/lib/snort_dynamicpreprocessor/
         preprocessor stream5_global: track_tcp yes, \
            track_udp no, \
            track_icmp no

         preprocessor stream5_tcp: ports both 25 5555

         preprocessor smtp: ports { 5555 25 465 587 691 } \
             inspection_type stateful \
             b64_decode_depth 0 \
             qp_decode_depth 0 \
             bitenc_decode_depth 0 \
             uu_decode_depth 0 \
             log_mailfrom \
             log_rcptto \
             log_filename \
             log_email_hdrs \
             normalize cmds \
             normalize_cmds { ATRN AUTH BDAT CHUNKING DATA DEBUG EHLO  
EMAL ESAM ESND ESOM ETRN EVFY } \
             normalize_cmds { EXPN HELO HELP IDENT MAIL NOOP ONEX QUEU  
QUIT RCPT RSET SAML SEND SOML } \
             normalize_cmds { STARTTLS TICK TIME TURN TURNME VERB VRFY  
X-ADAT X-DRCP X-ERCP X-EXCH50 } \
             normalize_cmds { X-EXPS X-LINK2STATE XADR XAUTH XCIR  
XEXCH50 XGEN XLICENSE XQUE XSTA XTRN XUSR } \
             max_command_line_len 512 \
             max_header_line_len 1000 \
             max_response_line_len 512 \
             alt_max_command_line_len 260 { MAIL } \
             alt_max_command_line_len 300 { RCPT } \
             alt_max_command_line_len 500 { HELP HELO ETRN EHLO } \
             alt_max_command_line_len 255 { EXPN VRFY ATRN SIZE BDAT  
DEBUG EMAL ESAM ESND ESOM EVFY IDENT NOOP RSET } \
             alt_max_command_line_len 246 { SEND SAML SOML AUTH TURN  
ETRN DATA RSET QUIT ONEX QUEU STARTTLS TICK TIME TURNME VERB X-EXPS  
X-LINK2STATE XADR XAUTH XCIR XEXCH50 XGEN XLICENSE XQUE XSTA XTRN XUSR  
} \
             valid_cmds { ATRN AUTH BDAT CHUNKING DATA DEBUG EHLO EMAL  
ESAM ESND ESOM ETRN EVFY } \
             valid_cmds { EXPN HELO HELP IDENT MAIL NOOP ONEX QUEU  
QUIT RCPT RSET SAML SEND SOML } \
             valid_cmds { STARTTLS TICK TIME TURN TURNME VERB VRFY  
X-ADAT X-DRCP X-ERCP X-EXCH50 } \
             valid_cmds { X-EXPS X-LINK2STATE XADR XAUTH XCIR XEXCH50  
XGEN XLICENSE XQUE XSTA XTRN XUSR } \
             xlink2state { enabled }

         alert ( msg: "SMTP_RESPONSE_OVERFLOW"; sid: 3; gid: 124; rev:  
1; metadata: rule-type preproc, service smtp, policy security-ips  
drop; )
         alert ( msg: "SMTP_UNKNOWN_CMD"; sid: 5; gid: 124; rev: 1;  
metadata: rule-type preproc, service smtp ; )
         alert ( msg: "SMTP_ILLEGAL_CMD"; sid: 6; gid: 124; rev: 1;  
metadata: rule-type preproc, service smtp ;  )

         output alert_fast: stdout


* First two dump files: 'smtp_packet_loss.cap' and 'smtp_no_packet_loss.cap'

The dump file 'smtp_packet_loss.cap' is based on an actual SMTP  
session for which some packets were received out of order (or were  
lost in transit and resend later).
The dump file 'smtp_no_packet_loss.cap' contains the same data as  
'smtp_packet_loss.cap' except the packets were reordered.

Running it:
         $ /ub/pkg/ips/bin/snort -v -l /var/log -c /etc/ips/snort.conf  
--daq-dir /lib/daq/ -r /tmp/smtp_no_packet_loss.cap 2>&1 | grep '124:'

         $ /ub/pkg/ips/bin/snort -v -l /var/log -c /etc/ips/snort.conf  
--daq-dir /lib/daq/ -r /tmp/smtp_packet_loss.cap 2>&1 | grep '124:'
         08/28-12:44:54.238096  [**] [124:3:1] (smtp) Attempted  
response buffer overflow: 567 chars [**] [Priority: 0] {TCP}  
192.168.173.1:5555 -> 192.168.173.153:5556
         => this is a false positive, it matches on SSL data (probably  
the server certificate)...


* Looking at the code of 'SnortSMTP'  
(dynamic-preprocessors/smtp/snort_smtp.c) shows:

No reassembly is ever done for packets coming from the server
Reassembly is done for packets coming for the client except when the  
state is set to 'STATE_TLS_CLIENT_PEND'.

This means that it goes wrong when:
- packets from the server are received out of order/are resend/...
- the packet immediately following the 'STARTTLS' command, normally  
the 'SSL CLIENT HELLO', is a resend of a previous packet or when the  
'SSL CLIENT HELLO' packet is split in two and the second segment is  
received before the first.

To trigger this additional dumps were created.
The only 'real' traffic that was observed is the one of  
'smtp_packet_loss.cap'.
The traffic in the other dumps was not (yet) observed on an actual system.

Dumps created:

- 'smtp_starttls.cap': the 'STARTTLS' command is resend: can happen in  
real session
- 'smtp_ehlo.cap': the 'EHLO' command is resend after the 'STARTTLS'  
command: this can theoretically happen in real sessions but seems a  
bit unlikely
- 'smtp_smtp_client_hello1.cap': the 'SSL CLIENT HELLO' packet is  
split in two, segments arrive in the 'correct' order: this can  
theoretically happen in real sessions but seems a bit unlikely given  
that the 'SSL CLIENT HELLO' is typically small
- 'smtp_smtp_client_hello2.cap': the 'SSL CLIENT HELLO' packet is  
split in two, the 2nd segment is received before the 1st segment: this  
can theoretically happen in real sessions but seems a bit unlikely  
given that the 'SSL CLIENT HELLO' is typically small



* Running it under gdb

Since it's easier to see what is happening I decided to run all dumps  
under gdb.
Note: line numbers are based on snort 2.9.5.3.

Breakpoints:
- 1: snort_smtp.c:2369: switch from state 'STATE_TLS_CLIENT_PEND' to  
'STATE_TLS_SERVER_PEND' -> TLS from client is OK
- 2: snort_smtp.c:2374: switch from state 'STATE_TLS_CLIENT_PEND' to  
'STATE_COMMAND' -> TLS from client is NOT OK
- 3: snort_smtp.c:2082: switch from state 'STATE_TLS_SERVER_PEND' to  
'STATE_TLS_DATA' -> TLS from client and server is OK (-> all further  
data ignored)
- 4: snort_smtp.c:2086: switch from state 'STATE_TLS_SERVER_PEND' to  
'STATE_COMMAND' -> TLS from server is NOT OK

Dumps:

- 'smtp_no_packet_loss.cap':
Breakpoints 1 and 3 reached: TLS correctly detected.

- 'smtp_packet_loss.cap':
Breakpoints 1 and 4 reached: TLS from client is OK, TLS from server is  
rejected by snort (accepted by the actual client)

- 'smtp_starttls.cap':
Breakpoints 2, 1 and 3 reached: TLS correctly detected.

- 'smtp_ehlo.cap':
Breakpoint 2 reached: TLS from client not OK

- 'smtp_client_hello1.cap'
Breakpoints 1 and 3 reached: TLS correctly detected.

- 'smtp_client_hello2.cap':
Breakpoint 2 reached: TLS from client not OK


Most of these problems can be explained due to the reassembly not being done.
There is one special case tho, for the dump 'smtp_starttls.cap' it  
does detect the switch to TLS.
Looking at the output above shows that it first reaches breakpoint 2  
and then breakpoint 1 which is unexpected.
What I would have expected is that it reached breakpoint 2 and never  
reaches breakpoint 1.
What this indicates to me (not investigated) is that the duplicate  
'STARTTLS' packet was processed somewhere..
I would have expected it to be ignored since it's a  
retransmissions/duplicate packet.. (same sequence number)
This could indicates that there is another problem (somewhere)...


Best regards,

Bram


----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smtp_packet_loss.cap
Type: application/octet-stream
Size: 3991 bytes
Desc: not available
URL: <https://lists.snort.org/pipermail/snort-devel/attachments/20130830/53abfc67/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smtp_no_packet_loss.cap
Type: application/octet-stream
Size: 3983 bytes
Desc: not available
URL: <https://lists.snort.org/pipermail/snort-devel/attachments/20130830/53abfc67/attachment-0001.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smtp_starttls.cap
Type: application/octet-stream
Size: 4195 bytes
Desc: not available
URL: <https://lists.snort.org/pipermail/snort-devel/attachments/20130830/53abfc67/attachment-0002.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smtp_ehlo.cap
Type: application/octet-stream
Size: 4303 bytes
Desc: not available
URL: <https://lists.snort.org/pipermail/snort-devel/attachments/20130830/53abfc67/attachment-0003.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smtp_client_hello1.cap
Type: application/octet-stream
Size: 4147 bytes
Desc: not available
URL: <https://lists.snort.org/pipermail/snort-devel/attachments/20130830/53abfc67/attachment-0004.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smtp_client_hello2.cap
Type: application/octet-stream
Size: 4155 bytes
Desc: not available
URL: <https://lists.snort.org/pipermail/snort-devel/attachments/20130830/53abfc67/attachment-0005.obj>


More information about the Snort-devel mailing list