[Snort-users] Getting alerts for every file Snort detects and File Services preprocessor

Jaime Nebrera jnebrera at ...16842...
Fri Mar 27 16:04:03 EDT 2015


Hi Víctor,

Pablo's work is geared towards a more ambitious goal:

1) Intercept the file and send it to a S3 based storage platform for
further analysis or whatever

2) Provide as much context about such interception as possible, like from
were to whom, URL, email, etc

This we hope to open source and make it public as soon as it is usable in
our repository www.github.com/redborder
El 27/03/2015 20:50, "Victor Roemer" <viroemer at ...589...> escribió:

>  Pablo,
>
> Sorry for the delay. The "file_identify" preprocessor is designed to work
> more as a "reputation" based system. I think you will get better millage by
> using the new "file_type" rule keyword in a plain-old snort rule.
>
> Something like this:
>
> alert tcp $EXTERNAL_NET any -> $HOME_NET any (msg:"GIF file downloaded";
> flow:to_client,established; file_type:GIF; sid:1000000;)
> alert tcp $EXTERNAL_NET any -> $HOME_NET any (msg:"GIF file downloaded";
> flow:to_client,established; file_type:PNG; sid:1000001;)
>
>
> You'll still need to have "file_magic.conf" included in your Snort
> configuration, but you will not need the file_identify preprocessor.
>
> ~Victor
>
> On 03/17/15 4:57, Pablo Cantos Polaino wrote:
>
> Hello all,
>
> I already sent this issue to snort-devel with the same subject since I am
> not sure if either I am configuring Snort in the right way or there is an
> internal malfunction to fix.
>
> I have been testing the new experimental preprocessor called File Services
> in order to get an event every time a file go through our network. To carry
> on these tests I have used two pcap files. The first one is a 1GB-size pcap
> with a great number of files and the second one is a short pcap generated
> on my computer when I downloaded a GIF file.
>
> My snort.conf file is configured like this at the end:
>
> include file_magic.conf
>
>  preprocessor file_inspect: type_id, signature, \
>              capture_queue_size 5000, \
>              capture_disk /home/file_capture/tmp/
>
>
>
> In both cases files are captured by the preprocessor, as you can see below
> (1GB pcap output):
>
> ===============================================================================
>
>  Action Stats:
>      Alerts:            0 (  0.000%)
>      Logged:            0 (  0.000%)
>      Passed:            0 (  0.000%)
> Limits:
>       Match:            0
>       Queue:            0
>         Log:            0
>       Event:            0
>       Alert:            0
> Verdicts:
>       Allow:      8418451 ( 97.482%)
>       Block:            0 (  0.000%)
>     Replace:            0 (  0.000%)
>   Whitelist:       217492 (  2.518%)
>   Blacklist:            0 (  0.000%)
>      Ignore:            0 (  0.000%)
>
> ===============================================================================
>
>
>
> ===============================================================================
>
>  File Preprocessor Statistics
>   Total file type callbacks:            576
>   Total file signature callbacks:       578
>   Total files would saved to disk:      574
>   Total files saved to disk:            320
>   Total file data saved to disk:        483039    bytes
>   Total files duplicated:               254
>   Total files reserving failed:         2
>   Total file capture min:               0
>   Total file capture max:               2
>   Total file capture memcap:            0
>   Total files reading failed:           0
>   Total file agent memcap failures:     0
>   Total files sent:                     0
>   Total file data sent:                 0
>   Total file transfer failures:         0
>
> ===============================================================================
> File type stats:
>          Type              Download   (Bytes)      Upload     (Bytes)
>           GZ( 33)          2          5580056      0          0
>          SWF( 52)          1          65991        0          0
>          GIF( 62)          7          16516        0          0
>          GIF( 63)          275        151718       0          0
>          PNG( 69)          266        256724       0          0
>         JPEG( 70)          2          35566        0          0
>          BMP(148)          2          4204         0          0
>          ICO(149)          21         187894       0          0
>             Total          576        6298669      0          0
> File signature stats:
>          Type              Download   Upload
>           GZ( 33)          2          0
>          SWF( 52)          1          0
>          GIF( 62)          7          0
>          GIF( 63)          275        0
>          PNG( 69)          266        0
>         JPEG( 70)          2          0
>          BMP(148)          2          0
>          ICO(149)          21         0
>             Total          576        0
> File type verdicts:
>         UNKNOWN:           576
>             LOG:           0
>            STOP:           0
>           BLOCK:           0
>          REJECT:           0
>         PENDING:           0
>    STOP CAPTURE:           0
>           Total:           576
> File signature verdicts:
>         UNKNOWN:           578
>             LOG:           0
>            STOP:           0
>           BLOCK:           0
>          REJECT:           0
>         PENDING:           0
>    STOP CAPTURE:           0
>           Total:           578
> Total files processed:             68985
> Total files data processed:        97156439  bytes
> Total files buffered:              576
> Total files released:              574
> Total files freed:                 2
> Total files captured:              574
> Total files within one packet:     561
> Total buffers allocated:           641
> Total buffers freed:               64
> Total buffers released:            577
> Maximum file buffers used:         64
> Total buffers free errors:         0
> Total buffers release errors:      0
> Total memcap failures:             0
> Total memcap failures at reserve:  0
> Total reserve failures:            0
> Total file capture size min:       0
> Total file capture size max:       0
> Total capture max before reserve:  2
> Total file signature max:          0
> Maximum buffers can allocate:      3196
> Number of buffers in use:          0
> Number of buffers in free list:    2619
> Number of buffers in release list: 577
>
> ===============================================================================
>
>
>
> Following the instructions given as examples inside the file README.file, I
> have included the following rules to get an alert every time Snort detects
> a file:
>
> alert (msg: "GIF file"; gid:146; sid:63; rev:1; metadata: rule-type
>
>  preproc;)
>
>
>
> After that, no alert showed up.
>
> I went deep inside the code to find out what the reason is and found the
> following piece of code that confused me:
>
> snort/src/dynamic-preprocessors/file/file_agent.c:601-614
>
> /*
>
>   * File type callback when file type is identified
>  *
>  * For file capture or file signature, FILE_VERDICT_PENDING must be
> returned
>  */
> static File_Verdict file_agent_type_callback(void* p, void* ssnptr,
>         uint32_t file_type_id, bool upload, uint32_t file_id)
> {
>     file_inspect_stats.file_types_total++;
>     if (file_signature_enabled || file_capture_enabled)
>         return FILE_VERDICT_UNKNOWN;
>     else
>         return FILE_VERDICT_LOG;
> }
>
>
>
> You can read on the description that FILE_VERDICT_PENDING must be returned
> when file capture OR file signature is enabled, but what really the code
> does is to return FILE_VERDICT_UNKNOWN when capture or signature are
> enabled.
>
> After see that, I have modified the snort.conf by carrying on the following
> changes:
>
> Replace this:
>
> preprocessor file_inspect: type_id, signature, \
>
>               capture_queue_size 5000, \
>              capture_disk /home/file_capture/tmp/
>
>
>
> By:
>
> preprocessor file_inspect: type_id
>
>
> This way I forced to go through the ELSE and return a FILE_VERDICT_LOG.
> After this change, and using the same two alert rules, we run snort,
> getting alerts like these below:
>
> 03/16-12:50:22.350000  [**] [146:63:1] GIF [**] [Priority: 0] {TCP}
>
>  192.168.202.78:80 <http://192.168.202.78/> <http://192.168.202.78/> -> 192.168.203.61:38976
> 03/16-12:50:22.350000  [**] [146:63:1] GIF [**] [Priority: 0] {TCP}192.168.202.78:80 <http://192.168.202.78/> <http://192.168.202.78/> -> 192.168.203.61:38976
> 03/16-12:50:22.350000  [**] [146:63:1] GIF [**] [Priority: 0] {TCP}192.168.202.78:80 <http://192.168.202.78/> <http://192.168.202.78/> -> 192.168.203.61:38977
>
>
> ...
>
>  and getting the following output at the end:
>
> ===============================================================================
>
>  Action Stats:
>      Alerts:          275 (  0.003%)
>      Logged:          275 (  0.003%)
>      Passed:            0 (  0.000%)
> Limits:
>       Match:            0
>       Queue:            0
>         Log:            0
>       Event:            0
>       Alert:            0
> Verdicts:
>       Allow:      8418514 ( 97.482%)
>       Block:            0 (  0.000%)
>     Replace:            0 (  0.000%)
>   Whitelist:       217429 (  2.518%)
>   Blacklist:            0 (  0.000%)
>      Ignore:            0 (  0.000%)
>
> ===============================================================================
>
>
>
>
> ===============================================================================
> File Preprocessor Statistics
>   Total file type callbacks:            576
>   Total file signature callbacks:       0
>   Total files would saved to disk:      0
>   Total files saved to disk:            0
>   Total file data saved to disk:        0         bytes
>   Total files duplicated:               0
>   Total files reserving failed:         0
>   Total file capture min:               0
>   Total file capture max:               0
>   Total file capture memcap:            0
>   Total files reading failed:           0
>   Total file agent memcap failures:     0
>   Total files sent:                     0
>   Total file data sent:                 0
>   Total file transfer failures:         0
>
> ===============================================================================
> File type stats:
>          Type              Download   (Bytes)      Upload     (Bytes)
>           GZ( 33)          2          0            0          0
>          SWF( 52)          1          0            0          0
>          GIF( 62)          7          0            0          0
>          GIF( 63)          275        0            0          0
>          PNG( 69)          266        0            0          0
>         JPEG( 70)          2          0            0          0
>          BMP(148)          2          0            0          0
>          ICO(149)          21         0            0          0
>             Total          576        0            0          0
> File signature stats:
>          Type              Download   Upload
>             Total          0          0
> File type verdicts:
>         UNKNOWN:           0
>             LOG:           576
>            STOP:           0
>           BLOCK:           0
>          REJECT:           0
>         PENDING:           0
>    STOP CAPTURE:           0
>           Total:           576
> File signature verdicts:
>         UNKNOWN:           0
>             LOG:           0
>            STOP:           0
>           BLOCK:           0
>          REJECT:           0
>         PENDING:           0
>    STOP CAPTURE:           0
>           Total:           0
> Total files processed:             68987
> Total files data processed:        42751396  bytes
> Total files buffered:              0
> Total files released:              0
> Total files freed:                 0
> Total files captured:              0
> Total files within one packet:     0
> Total buffers allocated:           0
> Total buffers freed:               0
> Total buffers released:            0
> Maximum file buffers used:         0
> Total buffers free errors:         0
> Total buffers release errors:      0
> Total memcap failures:             0
> Total memcap failures at reserve:  0
> Total reserve failures:            0
> Total file capture size min:       0
> Total file capture size max:       0
> Total capture max before reserve:  0
> Total file signature max:          0
>
> ===============================================================================
>
>
>
> As you can see, in the "File type verdicts" section I got all the files
> with verdict LOG. Also, I got 275 alerts that match the 275 GIF files
> detected by Snort.
>
> I am not sure if this is the expected behavior of this feature or maybe I
> am not configuring Snort properly.
>
> Am I doing something wrong or configuring the preprocessor in a wrong way?
>
> Thanks for your help and best Regards,
>
> Pablo Cantosredborder.org / pcantos at ...16842...
>
>
>
> This body part will be downloaded on demand.
>
>
>
> This body part will be downloaded on demand.
>
>
>
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming The Go Parallel Website,
> sponsored
> by Intel and developed in partnership with Slashdot Media, is your hub for
> all
> things parallel software development, from weekly thought leadership blogs
> to
> news, videos, case studies, tutorials and more. Take a look and join the
> conversation now. http://goparallel.sourceforge.net/
> _______________________________________________
> Snort-users mailing list
> Snort-users at lists.sourceforge.net
> Go to this URL to change user options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users
>
> Please visit http://blog.snort.org to stay current on all the latest
> Snort news!
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.snort.org/pipermail/snort-users/attachments/20150327/b9cc59a9/attachment.html>


More information about the Snort-users mailing list