<div dir="ltr"><div style="font-size:12.8000001907349px">Hello all,</div><div style="font-size:12.8000001907349px"><br></div><div style="font-size:12.8000001907349px">I have been testing the new experimental preprocessor called File Services in order to get an event every time a file go through our network. To carry on these tests I have used two pcap files. The first one is a 1GB-size pcap with a great number of files and the second one is a short pcap generated on my computer when I downloaded a GIF file.</div><div style="font-size:12.8000001907349px"><br></div><div style="font-size:12.8000001907349px">My snort.conf file is configured like this at the end:</div><div style="font-size:12.8000001907349px"><br></div><div style="font-size:12.8000001907349px"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">include file_magic.conf<br>preprocessor file_inspect: type_id, signature, \<br>             capture_queue_size 5000, \<br>             capture_disk /home/file_capture/tmp/</blockquote><div><br></div><div>In both cases files are captured by the preprocessor, as you can see below (1GB pcap output):<br></div></div><div style="font-size:12.8000001907349px"><br></div><div style="font-size:12.8000001907349px"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">===============================================================================<br>Action Stats:<br>     Alerts:            0 (  0.000%)<br>     Logged:            0 (  0.000%)<br>     Passed:            0 (  0.000%)<br>Limits:<br>      Match:            0<br>      Queue:            0<br>        Log:            0<br>      Event:            0<br>      Alert:            0<br>Verdicts:<br>      Allow:      8418451 ( 97.482%)<br>      Block:            0 (  0.000%)<br>    Replace:            0 (  0.000%)<br>  Whitelist:       217492 (  2.518%)<br>  Blacklist:            0 (  0.000%)<br>     Ignore:            0 (  0.000%)<br>===============================================================================</blockquote></div><div style="font-size:12.8000001907349px"><br></div><div style="font-size:12.8000001907349px"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><span style="font-size:12.8000001907349px">===============================================================================<br></span><span style="font-size:12.8000001907349px">File Preprocessor Statistics<br></span><span style="font-size:12.8000001907349px">  Total file type callbacks:            576        <br></span><span style="font-size:12.8000001907349px">  Total file signature callbacks:       578        <br></span><span style="font-size:12.8000001907349px">  Total files would saved to disk:      574        <br></span><span style="font-size:12.8000001907349px">  Total files saved to disk:            320        <br></span><span style="font-size:12.8000001907349px">  Total file data saved to disk:        483039    bytes<br></span><span style="font-size:12.8000001907349px">  Total files duplicated:               254        <br></span><span style="font-size:12.8000001907349px">  Total files reserving failed:         2          <br></span><span style="font-size:12.8000001907349px">  Total file capture min:               0          <br></span><span style="font-size:12.8000001907349px">  Total file capture max:               2          <br></span><span style="font-size:12.8000001907349px">  Total file capture memcap:            0          <br></span><span style="font-size:12.8000001907349px">  Total files reading failed:           0          <br></span><span style="font-size:12.8000001907349px">  Total file agent memcap failures:     0          <br></span><span style="font-size:12.8000001907349px">  Total files sent:                     0          <br></span><span style="font-size:12.8000001907349px">  Total file data sent:                 0          <br></span><span style="font-size:12.8000001907349px">  Total file transfer failures:         0          <br></span><span style="font-size:12.8000001907349px">===============================================================================<br></span><span style="font-size:12.8000001907349px">File type stats:<br></span><span style="font-size:12.8000001907349px">         Type              Download   (Bytes)      Upload     (Bytes)<br></span><span style="font-size:12.8000001907349px">          GZ( 33)          2          5580056      0          0          <br></span><span style="font-size:12.8000001907349px">         SWF( 52)          1          65991        0          0          <br></span><span style="font-size:12.8000001907349px">         GIF( 62)          7          16516        0          0          <br></span><span style="font-size:12.8000001907349px">         GIF( 63)          275        151718       0          0          <br></span><span style="font-size:12.8000001907349px">         PNG( 69)          266        256724       0          0          <br></span><span style="font-size:12.8000001907349px">        JPEG( 70)          2          35566        0          0          <br></span><span style="font-size:12.8000001907349px">         BMP(148)          2          4204         0          0          <br></span><span style="font-size:12.8000001907349px">         ICO(149)          21         187894       0          0          <br></span><span style="font-size:12.8000001907349px">            Total          576        6298669      0          0          </span><span style="font-size:12.8000001907349px"><br></span><span style="font-size:12.8000001907349px">File signature stats:<br></span><span style="font-size:12.8000001907349px">         Type              Download   Upload <br></span><span style="font-size:12.8000001907349px">          GZ( 33)          2          0          <br></span><span style="font-size:12.8000001907349px">         SWF( 52)          1          0          <br></span><span style="font-size:12.8000001907349px">         GIF( 62)          7          0          <br></span><span style="font-size:12.8000001907349px">         GIF( 63)          275        0          <br></span><span style="font-size:12.8000001907349px">         PNG( 69)          266        0          <br></span><span style="font-size:12.8000001907349px">        JPEG( 70)          2          0          <br></span><span style="font-size:12.8000001907349px">         BMP(148)          2          0          <br></span><span style="font-size:12.8000001907349px">         ICO(149)          21         0          <br></span><span style="font-size:12.8000001907349px">            Total          576        0          </span><span style="font-size:12.8000001907349px"><br></span><span style="font-size:12.8000001907349px">File type verdicts:<br></span><span style="font-size:12.8000001907349px">        UNKNOWN:           576        <br></span><span style="font-size:12.8000001907349px">            LOG:           0          <br></span><span style="font-size:12.8000001907349px">           STOP:           0          <br></span><span style="font-size:12.8000001907349px">          BLOCK:           0          <br></span><span style="font-size:12.8000001907349px">         REJECT:           0          <br></span><span style="font-size:12.8000001907349px">        PENDING:           0          <br></span><span style="font-size:12.8000001907349px">   STOP CAPTURE:           0          <br></span><span style="font-size:12.8000001907349px">          Total:           576        </span><span style="font-size:12.8000001907349px"><br></span><span style="font-size:12.8000001907349px">File signature verdicts:<br></span><span style="font-size:12.8000001907349px">        UNKNOWN:           578        <br></span><span style="font-size:12.8000001907349px">            LOG:           0          <br></span><span style="font-size:12.8000001907349px">           STOP:           0          <br></span><span style="font-size:12.8000001907349px">          BLOCK:           0          <br></span><span style="font-size:12.8000001907349px">         REJECT:           0          <br></span><span style="font-size:12.8000001907349px">        PENDING:           0          <br></span><span style="font-size:12.8000001907349px">   STOP CAPTURE:           0          <br></span><span style="font-size:12.8000001907349px">          Total:           578        </span><span style="font-size:12.8000001907349px"><br></span><span style="font-size:12.8000001907349px">Total files processed:             68985      <br></span><span style="font-size:12.8000001907349px">Total files data processed:        97156439  bytes <br></span><span style="font-size:12.8000001907349px">Total files buffered:              576        <br></span><span style="font-size:12.8000001907349px">Total files released:              574        <br></span><span style="font-size:12.8000001907349px">Total files freed:                 2          <br></span><span style="font-size:12.8000001907349px">Total files captured:              574        <br></span><span style="font-size:12.8000001907349px">Total files within one packet:     561        <br></span><span style="font-size:12.8000001907349px">Total buffers allocated:           641        <br></span><span style="font-size:12.8000001907349px">Total buffers freed:               64         <br></span><span style="font-size:12.8000001907349px">Total buffers released:            577        <br></span><span style="font-size:12.8000001907349px">Maximum file buffers used:         64         <br></span><span style="font-size:12.8000001907349px">Total buffers free errors:         0          <br></span><span style="font-size:12.8000001907349px">Total buffers release errors:      0          <br></span><span style="font-size:12.8000001907349px">Total memcap failures:             0          <br></span><span style="font-size:12.8000001907349px">Total memcap failures at reserve:  0          <br></span><span style="font-size:12.8000001907349px">Total reserve failures:            0          <br></span><span style="font-size:12.8000001907349px">Total file capture size min:       0          <br></span><span style="font-size:12.8000001907349px">Total file capture size max:       0          <br></span><span style="font-size:12.8000001907349px">Total capture max before reserve:  2          <br></span><span style="font-size:12.8000001907349px">Total file signature max:          0          <br></span><span style="font-size:12.8000001907349px">Maximum buffers can allocate:      3196       <br></span><span style="font-size:12.8000001907349px">Number of buffers in use:          0          <br></span><span style="font-size:12.8000001907349px">Number of buffers in free list:    2619       <br></span><span style="font-size:12.8000001907349px">Number of buffers in release list: 577        <br></span><span style="font-size:12.8000001907349px">===============================================================================</span></blockquote><div style="font-size:12.8000001907349px"><br></div><div style="font-size:12.8000001907349px">Following the instructions given as examples inside the file README.file, I have included the following rules to get an alert every time Snort detects a file:</div><div style="font-size:12.8000001907349px"><br></div><div style="font-size:12.8000001907349px"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">alert (msg: "GIF file"; gid:146; sid:63; rev:1; metadata: rule-type preproc;)<br>alert (msg: "GIF file"; gid:147; sid:1; rev:1; metadata: rule-type preproc;)</blockquote><div><br></div></div><div style="font-size:12.8000001907349px">After that, no alert showed up.</div><div style="font-size:12.8000001907349px"><br></div><div style="font-size:12.8000001907349px">I went deep inside the code to find out what the reason is and found the following piece of code that confused me:</div><div style="font-size:12.8000001907349px"><br></div><div style="font-size:12.8000001907349px">snort/src/dynamic-preprocessors/file/file_agent.c:601-614</div><div style="font-size:12.8000001907349px"><br></div><div style="font-size:12.8000001907349px"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">/*<br> * File type callback when file type is identified<br> *<br> * For file capture or file signature, FILE_VERDICT_PENDING must be returned<br> */<br>static File_Verdict file_agent_type_callback(void* p, void* ssnptr,<br>        uint32_t file_type_id, bool upload, uint32_t file_id)<br>{<br>    file_inspect_stats.file_types_total++;<br>    if (file_signature_enabled || file_capture_enabled)<br>        return FILE_VERDICT_UNKNOWN;<br>    else<br>        return FILE_VERDICT_LOG;<br>}</blockquote><div style="font-size:12.8000001907349px"><br></div><div style="font-size:12.8000001907349px">You can read on the description that FILE_VERDICT_PENDING must be returned when file capture OR file signature is enabled, but what really the code does is to return FILE_VERDICT_UNKNOWN when capture or signature are enabled.</div><div style="font-size:12.8000001907349px"><br></div><div style="font-size:12.8000001907349px">After see that, I have modified the snort.conf by carrying on the following changes:</div><div style="font-size:12.8000001907349px"><br></div><div style="font-size:12.8000001907349px">Replace this:</div><div style="font-size:12.8000001907349px"><br></div><div style="font-size:12.8000001907349px"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">preprocessor file_inspect: type_id, signature, \<br>             capture_queue_size 5000, \<br>             capture_disk /home/file_capture/tmp/</blockquote></div></div><div style="font-size:12.8000001907349px"><br></div><div style="font-size:12.8000001907349px">By:</div><div style="font-size:12.8000001907349px"><br></div><div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">preprocessor file_inspect: type_id</blockquote><div style="font-size:small"><br></div><div style="font-size:small">This way I forced to go through the ELSE and return a FILE_VERDICT_LOG. After this change, and using the same two alert rules, we run snort, getting alerts like these below:</div><div style="font-size:small"><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">03/16-12:50:22.350000  [**] [146:63:1] GIF [**] [Priority: 0] {TCP} <a href="http://192.168.202.78/" target="_blank">192.168.202.78:80</a> -> <a href="http://192.168.203.61:38976/" target="_blank">192.168.203.61:38976</a><br>03/16-12:50:22.350000  [**] [146:63:1] GIF [**] [Priority: 0] {TCP} <a href="http://192.168.202.78/" target="_blank">192.168.202.78:80</a> -> <a href="http://192.168.203.61:38976/" target="_blank">192.168.203.61:38976</a><br>03/16-12:50:22.350000  [**] [146:63:1] GIF [**] [Priority: 0] {TCP} <a href="http://192.168.202.78/" target="_blank">192.168.202.78:80</a> -> <a href="http://192.168.203.61:38977/" target="_blank">192.168.203.61:38977</a></blockquote><div><div style="font-size:small">...</div><div style="font-size:small"><br></div><div style="font-size:small"> and getting the following output at the end:<br></div><div style="font-size:small"><br></div><div><div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">===============================================================================<br>Action Stats:<br>     Alerts:          275 (  0.003%)<br>     Logged:          275 (  0.003%)<br>     Passed:            0 (  0.000%)<br>Limits:<br>      Match:            0<br>      Queue:            0<br>        Log:            0<br>      Event:            0<br>      Alert:            0<br>Verdicts:<br>      Allow:      8418514 ( 97.482%)<br>      Block:            0 (  0.000%)<br>    Replace:            0 (  0.000%)<br>  Whitelist:       217429 (  2.518%)<br>  Blacklist:            0 (  0.000%)<br>     Ignore:            0 (  0.000%)<br>===============================================================================</blockquote></div><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">===============================================================================<br>File Preprocessor Statistics<br>  Total file type callbacks:            576        <br>  Total file signature callbacks:       0          <br>  Total files would saved to disk:      0          <br>  Total files saved to disk:            0          <br>  Total file data saved to disk:        0         bytes<br>  Total files duplicated:               0          <br>  Total files reserving failed:         0          <br>  Total file capture min:               0          <br>  Total file capture max:               0          <br>  Total file capture memcap:            0          <br>  Total files reading failed:           0          <br>  Total file agent memcap failures:     0          <br>  Total files sent:                     0          <br>  Total file data sent:                 0          <br>  Total file transfer failures:         0          <br>===============================================================================<br>File type stats:<br>         Type              Download   (Bytes)      Upload     (Bytes)<br>          GZ( 33)          2          0            0          0          <br>         SWF( 52)          1          0            0          0          <br>         GIF( 62)          7          0            0          0          <br>         GIF( 63)          275        0            0          0          <br>         PNG( 69)          266        0            0          0          <br>        JPEG( 70)          2          0            0          0          <br>         BMP(148)          2          0            0          0          <br>         ICO(149)          21         0            0          0          <br>            Total          576        0            0          0          <br>File signature stats:<br>         Type              Download   Upload <br>            Total          0          0          <br>File type verdicts:<br>        UNKNOWN:           0          <br>            LOG:           576        <br>           STOP:           0          <br>          BLOCK:           0          <br>         REJECT:           0          <br>        PENDING:           0          <br>   STOP CAPTURE:           0          <br>          Total:           576        <br>File signature verdicts:<br>        UNKNOWN:           0          <br>            LOG:           0          <br>           STOP:           0          <br>          BLOCK:           0          <br>         REJECT:           0          <br>        PENDING:           0          <br>   STOP CAPTURE:           0          <br>          Total:           0          <br>Total files processed:             68987      <br>Total files data processed:        42751396  bytes <br>Total files buffered:              0          <br>Total files released:              0          <br>Total files freed:                 0          <br>Total files captured:              0          <br>Total files within one packet:     0          <br>Total buffers allocated:           0          <br>Total buffers freed:               0          <br>Total buffers released:            0          <br>Maximum file buffers used:         0          <br>Total buffers free errors:         0          <br>Total buffers release errors:      0          <br>Total memcap failures:             0          <br>Total memcap failures at reserve:  0          <br>Total reserve failures:            0          <br>Total file capture size min:       0          <br>Total file capture size max:       0          <br>Total capture max before reserve:  0          <br>Total file signature max:          0          <br>===============================================================================</blockquote></div><div style="font-size:small"><br></div><div style="font-size:small">As you can see, in the "File type verdicts" section I got all the files with verdict LOG. Also, I got 275 alerts that match the 275 GIF files detected by Snort.</div><div style="font-size:small"><br></div><div style="font-size:small">I am not sure if this is the expected behavior of this feature or maybe I am not configuring Snort properly.</div><div style="font-size:small"><br></div><div style="font-size:small">Am I doing something wrong or configuring the preprocessor in a wrong way?</div><div style="font-size:small"><br></div><div style="font-size:small">Thanks for your help and best Regards,</div></div></div></div></div></div></div></div></div></div></div><div><div class="gmail_signature"><div dir="ltr"><div><br></div><div>Pablo Cantos</div><div><a href="http://redborder.org" target="_blank">redborder.org</a> / <a href="mailto:pcantos@...3500..." target="_blank">pcantos@...3500...</a></div></div></div></div>
</div>