[Snort-users] Getting alerts for every file Snort detects and File Services preprocessor

Victor Roemer viroemer at ...589...
Fri Mar 27 15:49:25 EDT 2015


Pablo,

Sorry for the delay. The "file_identify" preprocessor is designed to 
work more as a "reputation" based system. I think you will get better 
millage by using the new "file_type" rule keyword in a plain-old snort rule.

Something like this:

alert tcp $EXTERNAL_NET any -> $HOME_NET any (msg:"GIF file downloaded"; 
flow:to_client,established; file_type:GIF; sid:1000000;)
alert tcp $EXTERNAL_NET any -> $HOME_NET any (msg:"GIF file downloaded"; 
flow:to_client,established; file_type:PNG; sid:1000001;)


You'll still need to have "file_magic.conf" included in your Snort 
configuration, but you will not need the file_identify preprocessor.

~Victor

On 03/17/15 4:57, Pablo Cantos Polaino wrote:
> Hello all,
>
> I already sent this issue to snort-devel with the same subject since I am
> not sure if either I am configuring Snort in the right way or there is an
> internal malfunction to fix.
>
> I have been testing the new experimental preprocessor called File Services
> in order to get an event every time a file go through our network. To carry
> on these tests I have used two pcap files. The first one is a 1GB-size pcap
> with a great number of files and the second one is a short pcap generated
> on my computer when I downloaded a GIF file.
>
> My snort.conf file is configured like this at the end:
>
> include file_magic.conf
>> preprocessor file_inspect: type_id, signature, \
>>               capture_queue_size 5000, \
>>               capture_disk /home/file_capture/tmp/
>
> In both cases files are captured by the preprocessor, as you can see below
> (1GB pcap output):
>
> ===============================================================================
>> Action Stats:
>>       Alerts:            0 (  0.000%)
>>       Logged:            0 (  0.000%)
>>       Passed:            0 (  0.000%)
>> Limits:
>>        Match:            0
>>        Queue:            0
>>          Log:            0
>>        Event:            0
>>        Alert:            0
>> Verdicts:
>>        Allow:      8418451 ( 97.482%)
>>        Block:            0 (  0.000%)
>>      Replace:            0 (  0.000%)
>>    Whitelist:       217492 (  2.518%)
>>    Blacklist:            0 (  0.000%)
>>       Ignore:            0 (  0.000%)
>>
>> ===============================================================================
>
> ===============================================================================
>> File Preprocessor Statistics
>>    Total file type callbacks:            576
>>    Total file signature callbacks:       578
>>    Total files would saved to disk:      574
>>    Total files saved to disk:            320
>>    Total file data saved to disk:        483039    bytes
>>    Total files duplicated:               254
>>    Total files reserving failed:         2
>>    Total file capture min:               0
>>    Total file capture max:               2
>>    Total file capture memcap:            0
>>    Total files reading failed:           0
>>    Total file agent memcap failures:     0
>>    Total files sent:                     0
>>    Total file data sent:                 0
>>    Total file transfer failures:         0
>>
>> ===============================================================================
>> File type stats:
>>           Type              Download   (Bytes)      Upload     (Bytes)
>>            GZ( 33)          2          5580056      0          0
>>           SWF( 52)          1          65991        0          0
>>           GIF( 62)          7          16516        0          0
>>           GIF( 63)          275        151718       0          0
>>           PNG( 69)          266        256724       0          0
>>          JPEG( 70)          2          35566        0          0
>>           BMP(148)          2          4204         0          0
>>           ICO(149)          21         187894       0          0
>>              Total          576        6298669      0          0
>> File signature stats:
>>           Type              Download   Upload
>>            GZ( 33)          2          0
>>           SWF( 52)          1          0
>>           GIF( 62)          7          0
>>           GIF( 63)          275        0
>>           PNG( 69)          266        0
>>          JPEG( 70)          2          0
>>           BMP(148)          2          0
>>           ICO(149)          21         0
>>              Total          576        0
>> File type verdicts:
>>          UNKNOWN:           576
>>              LOG:           0
>>             STOP:           0
>>            BLOCK:           0
>>           REJECT:           0
>>          PENDING:           0
>>     STOP CAPTURE:           0
>>            Total:           576
>> File signature verdicts:
>>          UNKNOWN:           578
>>              LOG:           0
>>             STOP:           0
>>            BLOCK:           0
>>           REJECT:           0
>>          PENDING:           0
>>     STOP CAPTURE:           0
>>            Total:           578
>> Total files processed:             68985
>> Total files data processed:        97156439  bytes
>> Total files buffered:              576
>> Total files released:              574
>> Total files freed:                 2
>> Total files captured:              574
>> Total files within one packet:     561
>> Total buffers allocated:           641
>> Total buffers freed:               64
>> Total buffers released:            577
>> Maximum file buffers used:         64
>> Total buffers free errors:         0
>> Total buffers release errors:      0
>> Total memcap failures:             0
>> Total memcap failures at reserve:  0
>> Total reserve failures:            0
>> Total file capture size min:       0
>> Total file capture size max:       0
>> Total capture max before reserve:  2
>> Total file signature max:          0
>> Maximum buffers can allocate:      3196
>> Number of buffers in use:          0
>> Number of buffers in free list:    2619
>> Number of buffers in release list: 577
>>
>> ===============================================================================
>
> Following the instructions given as examples inside the file README.file, I
> have included the following rules to get an alert every time Snort detects
> a file:
>
> alert (msg: "GIF file"; gid:146; sid:63; rev:1; metadata: rule-type
>> preproc;)
>>
> After that, no alert showed up.
>
> I went deep inside the code to find out what the reason is and found the
> following piece of code that confused me:
>
> snort/src/dynamic-preprocessors/file/file_agent.c:601-614
>
> /*
>>   * File type callback when file type is identified
>>   *
>>   * For file capture or file signature, FILE_VERDICT_PENDING must be
>> returned
>>   */
>> static File_Verdict file_agent_type_callback(void* p, void* ssnptr,
>>          uint32_t file_type_id, bool upload, uint32_t file_id)
>> {
>>      file_inspect_stats.file_types_total++;
>>      if (file_signature_enabled || file_capture_enabled)
>>          return FILE_VERDICT_UNKNOWN;
>>      else
>>          return FILE_VERDICT_LOG;
>> }
>
> You can read on the description that FILE_VERDICT_PENDING must be returned
> when file capture OR file signature is enabled, but what really the code
> does is to return FILE_VERDICT_UNKNOWN when capture or signature are
> enabled.
>
> After see that, I have modified the snort.conf by carrying on the following
> changes:
>
> Replace this:
>
> preprocessor file_inspect: type_id, signature, \
>>               capture_queue_size 5000, \
>>               capture_disk /home/file_capture/tmp/
>
> By:
>
> preprocessor file_inspect: type_id
>
>
> This way I forced to go through the ELSE and return a FILE_VERDICT_LOG.
> After this change, and using the same two alert rules, we run snort,
> getting alerts like these below:
>
> 03/16-12:50:22.350000  [**] [146:63:1] GIF [**] [Priority: 0] {TCP}
>> 192.168.202.78:80 <http://192.168.202.78/> -> 192.168.203.61:38976
>> 03/16-12:50:22.350000  [**] [146:63:1] GIF [**] [Priority: 0] {TCP}
>> 192.168.202.78:80 <http://192.168.202.78/> -> 192.168.203.61:38976
>> 03/16-12:50:22.350000  [**] [146:63:1] GIF [**] [Priority: 0] {TCP}
>> 192.168.202.78:80 <http://192.168.202.78/> -> 192.168.203.61:38977
> ...
>
>   and getting the following output at the end:
>
> ===============================================================================
>> Action Stats:
>>       Alerts:          275 (  0.003%)
>>       Logged:          275 (  0.003%)
>>       Passed:            0 (  0.000%)
>> Limits:
>>        Match:            0
>>        Queue:            0
>>          Log:            0
>>        Event:            0
>>        Alert:            0
>> Verdicts:
>>        Allow:      8418514 ( 97.482%)
>>        Block:            0 (  0.000%)
>>      Replace:            0 (  0.000%)
>>    Whitelist:       217429 (  2.518%)
>>    Blacklist:            0 (  0.000%)
>>       Ignore:            0 (  0.000%)
>>
>> ===============================================================================
>
>
>> ===============================================================================
>> File Preprocessor Statistics
>>    Total file type callbacks:            576
>>    Total file signature callbacks:       0
>>    Total files would saved to disk:      0
>>    Total files saved to disk:            0
>>    Total file data saved to disk:        0         bytes
>>    Total files duplicated:               0
>>    Total files reserving failed:         0
>>    Total file capture min:               0
>>    Total file capture max:               0
>>    Total file capture memcap:            0
>>    Total files reading failed:           0
>>    Total file agent memcap failures:     0
>>    Total files sent:                     0
>>    Total file data sent:                 0
>>    Total file transfer failures:         0
>>
>> ===============================================================================
>> File type stats:
>>           Type              Download   (Bytes)      Upload     (Bytes)
>>            GZ( 33)          2          0            0          0
>>           SWF( 52)          1          0            0          0
>>           GIF( 62)          7          0            0          0
>>           GIF( 63)          275        0            0          0
>>           PNG( 69)          266        0            0          0
>>          JPEG( 70)          2          0            0          0
>>           BMP(148)          2          0            0          0
>>           ICO(149)          21         0            0          0
>>              Total          576        0            0          0
>> File signature stats:
>>           Type              Download   Upload
>>              Total          0          0
>> File type verdicts:
>>          UNKNOWN:           0
>>              LOG:           576
>>             STOP:           0
>>            BLOCK:           0
>>           REJECT:           0
>>          PENDING:           0
>>     STOP CAPTURE:           0
>>            Total:           576
>> File signature verdicts:
>>          UNKNOWN:           0
>>              LOG:           0
>>             STOP:           0
>>            BLOCK:           0
>>           REJECT:           0
>>          PENDING:           0
>>     STOP CAPTURE:           0
>>            Total:           0
>> Total files processed:             68987
>> Total files data processed:        42751396  bytes
>> Total files buffered:              0
>> Total files released:              0
>> Total files freed:                 0
>> Total files captured:              0
>> Total files within one packet:     0
>> Total buffers allocated:           0
>> Total buffers freed:               0
>> Total buffers released:            0
>> Maximum file buffers used:         0
>> Total buffers free errors:         0
>> Total buffers release errors:      0
>> Total memcap failures:             0
>> Total memcap failures at reserve:  0
>> Total reserve failures:            0
>> Total file capture size min:       0
>> Total file capture size max:       0
>> Total capture max before reserve:  0
>> Total file signature max:          0
>>
>> ===============================================================================
>
> As you can see, in the "File type verdicts" section I got all the files
> with verdict LOG. Also, I got 275 alerts that match the 275 GIF files
> detected by Snort.
>
> I am not sure if this is the expected behavior of this feature or maybe I
> am not configuring Snort properly.
>
> Am I doing something wrong or configuring the preprocessor in a wrong way?
>
> Thanks for your help and best Regards,
>
> Pablo Cantos
> redborder.org / pcantos at ...16842...
>
>
> This body part will be downloaded on demand.
>
>
> This body part will be downloaded on demand.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.snort.org/pipermail/snort-users/attachments/20150327/13a375e3/attachment.html>


More information about the Snort-users mailing list