[Snort-devel] [snort-devel] Chainning pre-processors

Hui Cao hcao at ...402...
Thu Dec 5 13:42:47 EST 2013


Can you give me pcap and your configurations to take a  look?

Best,
Hui.

On 12/05/2013 01:40 PM, Emiliano Fausto wrote:
> Hui,
>
> I was really trying to get Stream5 working and I couldn't, I'm pretty 
> sure that the problem is not related to Stream5, but I just can't get 
> the preprocessor to rebuilt TCP segments.
>
> In wireshark I can see there are lot of TCP segmented packets and if I 
> select to reassemble the HTTP ones, it does it well. Backing to SRNOT, 
> for some reason the statistics that the Stream5 are showing are that 
> there where some TCP sessions, there were discarded TCP packets, etc. 
> But in the line: "TCP Rebuilt packets" shows a 0.
>
> I tried with HTTP so that I can have a known protocol to start with as 
> you suggested me, and tried to download a wallpaper, which (due to its 
> size) was delivered to me in several packets.
>
> I changed the snort.conf configuration file, to look like this:
>
> preprocessor stream5_global: track_tcp yes, \
>   track_udp no, \
>   track_icmp no, \
>     max_tcp 262144, \
>   max_active_responses 2, \
>   min_response_seconds 5, \
>   show_rebuilt_packets
> preprocessor stream5_tcp: policy linux, \
>   overlap_limit 10, small_segments 3 bytes 150, timeout 180, \
>   ports both all
>
> I also played a little bit with my own preprocessor to print the Flags 
> that the packet brings, and they change based on the priority I assign 
> to the AddPreproc(). But I tried different ones, from FIRST to LAST 
> and none of them matched the & FLAG_REBUILT_STREAM flag.
>
> I was looking at the stream5 documentation (the README file), also 
> searched different threads in the snort users and devel community, but 
> none of them helped me.
>
> Do you know where can I get more information or examples on how to use 
> properly the Stream5 preprocess, or if I'm missing something else to 
> make it works? maybe create a generic rule, or something like that?
>
> I really thank you in advance for all your help on this,
> Emiliano.
>
>
>
> 2013/12/5 Hui Cao <hcao at ...402... <mailto:hcao at ...402...>>
>
>     Make sure you have frag3 enabled. You can try the application
>     preprocessors (such as http, smtp, ssl, sip etc) to see how it
>     works. If the packets are reassembled, you can see that from snort
>     exit stats for stream etc.
>
>     Stream will reassemble the payloads of TCP.
>
>     Best,
>     Hui.
>
>
>     On 12/05/2013 08:26 AM, Emiliano Fausto wrote:
>>     Hello Hui,
>>
>>     you are totally right. I was confused, I didn't have to assemble
>>     fragmented IP packets.
>>
>>     So, I configured the Stream5 preprocessor in order it to take
>>     into account the port of my application (it's 9090).
>>
>>     I let my AddPreproc() with priority set to PRIORITY_APPLICATION
>>     so that it's registered after the Stream5.
>>
>>     Then, I tried to get any packet with (p->flags &
>>     FLAG_REBUILT_STREAM), but none of them matched, it seems that the
>>     Stream5 preprocessor isn't reassembling my TCP segments.
>>
>>     The SNORT configuration file is set like this:
>>
>>     preprocessor stream5_global: track_tcp yes, \
>>         track_udp yes, \
>>         track_icmp no, \
>>         max_tcp 262144, \
>>         max_udp 131072, \
>>         max_active_responses 2, \
>>         min_response_seconds 5
>>     preprocessor stream5_tcp: policy windows, detect_anomalies,
>>     require_3whs 180, \
>>         overlap_limit 10, small_segments 3 bytes 150, timeout 180, \
>>         ports client 21 22, \
>>         ports both 9090
>>     preprocessor stream5_udp: timeout 180
>>
>>     My app works on port 9090.
>>
>>     Maybe, the problem is not related to stream5, maybe the problem
>>     is originated because the one in charge of doing the segments
>>     reassembly is my preprocessor, at the application level, and not
>>     the stream5 preprocessor at the TCP level.
>>
>>     More on this, into wireshark I can see all the packets of the
>>     protocol TCP segmented, and if I go to
>>     preferences-->protocols-->TCP it has an option that says: "allow
>>     dissectors to reassemble TCP". It means that the reassemble isn't
>>     being done at the TCP layer, but into the proper application.
>>
>>     Do you know if SNORT has something of this? Or the Stream5 should
>>     reassemble the payloads of the TCP packets?
>>
>>     Thanks a lot in advance,
>>     Emiliano.
>>
>>
>>     2013/12/4 Hui Cao <hcao at ...402... <mailto:hcao at ...402...>>
>>
>>         Frag3 will deal with IP fragmentation. You might want
>>         reassembled data from TCP. This means your preprocessor
>>         should be after stream and you should check
>>         (SFSnortPacket*)ipacketp)->flags & FLAG_REBUILT_STREAM. Snort
>>         will only reassemble up to the flush point defined (typically
>>         16K). You can't expect snort will give all the data from one
>>         reassembled packet.
>>
>>         Fragmentation from wireshark might mean TCP segmentation. You
>>         might get full IP packets.
>>
>>         Best,
>>         Hui.
>>         On 12/04/2013 03:17 PM, Emiliano Fausto wrote:
>>>         Hui,
>>>
>>>         yes, I understood what you told me about filtering just the
>>>         de-fragmented packets, that's why I added these 3 lines:
>>>
>>>         SFSnortPacket *p = (SFSnortPacket *)pkt;
>>>         int fla = (p->flags & FLAG_REBUILT_FRAG);
>>>         _dpd.logMsg("Flags set? %d",fla);
>>>
>>>         But I generate fragmented packets (which for instance
>>>         Wireshark detects and show as fragmented and the possibility
>>>         of reassembly them) and I'm always getting with each one the
>>>         output: "Flags set? 0".
>>>
>>>         I receive all the fragmented packets, but not the last one
>>>         reassembled.
>>>
>>>         Regards,
>>>         Emiliano.
>>>
>>>
>>>         2013/12/4 Hui Cao <hcao at ...402...
>>>         <mailto:hcao at ...402...>>
>>>
>>>             It will still get all packets including the raw packets.
>>>             You need to
>>>             use packet flags to filter them.
>>>
>>>             Best,
>>>             Hui.
>>>
>>>             On Wed, Dec 4, 2013 at 3:00 PM, Emiliano Fausto
>>>             <emiliano.fausto at ...2499...
>>>             <mailto:emiliano.fausto at ...2499...>> wrote:
>>>             > Yes,
>>>             >
>>>             > that's what I thought, but for some reason the TCP
>>>             packets keep on coming
>>>             > fragmented to my preprocessor.
>>>             >
>>>             > No I took off all the preprocessors and just let the
>>>             frag3 and mines, I'll
>>>             > try to figure out if they are called in the correct
>>>             order, but they are not
>>>             > being chained.
>>>             >
>>>             > Thanks,
>>>             > Emiliano.
>>>             >
>>>             >
>>>             > 2013/12/4 Hui Cao <hcao at ...402...
>>>             <mailto:hcao at ...402...>>
>>>             >>
>>>             >> It looks good to me.
>>>             >>
>>>             >> Best,
>>>             >> Hui.
>>>             >>
>>>             >> On Wed, Dec 4, 2013 at 2:44 PM, Emiliano Fausto
>>>             >> <emiliano.fausto at ...2499...
>>>             <mailto:emiliano.fausto at ...2499...>> wrote:
>>>             >> > Hi Hui,
>>>             >> >
>>>             >> > I've seen that I was using PRIORITY_TRANSPORT,
>>>             which is lower than the
>>>             >> > PRIORITY_NETWORK that uses frag3.
>>>             >> >
>>>             >> > Anyway, I put the priority: PRIORITY_LAST to my own
>>>             preprocessor, but
>>>             >> > when
>>>             >> > the TCP packets keep arriving fragmented to my
>>>             preprocessor.
>>>             >> >
>>>             >> > Is there anything else I should take into account?
>>>             >> >
>>>             >> > Thanks in advance,
>>>             >> > Emiliano.
>>>             >> >
>>>             >> >
>>>             >> > 2013/12/4 Hui Cao <hcao at ...402...
>>>             <mailto:hcao at ...402...>>
>>>             >> >>
>>>             >> >> In sr/preprocids.h
>>>             >> >>
>>>             >> >> Best,
>>>             >> >> Hui.
>>>             >> >>
>>>             >> >> On 12/04/2013 02:36 PM, Emiliano Fausto wrote:
>>>             >> >>
>>>             >> >> Great,
>>>             >> >>
>>>             >> >> so, the pre-processors are "chained" by default,
>>>             and the order that
>>>             >> >> SNORT
>>>             >> >> follows to call them is set by the PRIORITY variable.
>>>             >> >>
>>>             >> >> Do you know where's defined this PRIORITY
>>>             variable? Because I saw that
>>>             >> >> the
>>>             >> >> frag3 is being registered with PRIORITY_NETWORK,
>>>             so I'd like to set the
>>>             >> >> priority of my own preprocessor as
>>>             (PRIORITY_NETWORK -1).
>>>             >> >>
>>>             >> >> Thanks in advance,
>>>             >> >> Emiliano
>>>             >> >>
>>>             >> >>
>>>             >> >> 2013/12/4 Hui Cao <hcao at ...402...
>>>             <mailto:hcao at ...402...>>
>>>             >> >>>
>>>             >> >>> sc means snort configuration. We use PRIORITY to
>>>             sort the processing.
>>>             >> >>> All
>>>             >> >>> processors enabled will be called and processed
>>>             based on priority. You
>>>             >> >>> have
>>>             >> >>> to rely on the code to figure out what exactly
>>>             snort does.
>>>             >> >>>
>>>             >> >>> The checking is correct. You will only process
>>>             rebuilt packets.
>>>             >> >>>
>>>             >> >>> Best,
>>>             >> >>> Hui.
>>>             >> >>> On 12/04/2013 02:19 PM, Emiliano Fausto wrote:
>>>             >> >>>
>>>             >> >>> Hello Hui,
>>>             >> >>>
>>>             >> >>> thanks a lot for your answer.
>>>             >> >>>
>>>             >> >>> Right now I have registered my preprocessor
>>>             (let's call it
>>>             >> >>> examplePreprocess as you said, because right now
>>>             I'm using the one
>>>             >> >>> provided
>>>             >> >>> with the DPX) with this line:
>>>             >> >>>
>>>             >> >>> _dpd.addPreproc(ExampleProcess,
>>>             PRIORITY_TRANSPORT, 10000,
>>>             >> >>> PROTO_BIT__TCP);
>>>             >> >>>
>>>             >> >>> So, the only change is to add previous to the
>>>             parameter
>>>             >> >>> ExampleProcess,
>>>             >> >>> the "sc". What does it mean? Do you know if
>>>             there's any documentation
>>>             >> >>> about
>>>             >> >>> this chaining preprocesses?
>>>             >> >>>
>>>             >> >>> So, checking the flags, should be:
>>>             >> >>>
>>>             >> >>> (SFSnortPacket*)tcppacket)->flags & FLAG_REBUILT_FRAG
>>>             >> >>>
>>>             >> >>> right?
>>>             >> >>>
>>>             >> >>> Thanks again!
>>>             >> >>> Emiliano.
>>>             >> >>>
>>>             >> >>>
>>>             >> >>>
>>>             >> >>> Then, I'll have to register my own preprocessor
>>>             where?
>>>             >> >>>
>>>             >> >>>
>>>             >> >>> 2013/12/4 Hui Cao <hcao at ...402...
>>>             <mailto:hcao at ...402...>>
>>>             >> >>>>
>>>             >> >>>> Yes, it is possible. You can register you
>>>             preprocessor like this:
>>>             >> >>>>
>>>             >> >>>> _dpd.addPreproc( sc, ExampleProcess,
>>>             PRIORITY_TRANSPORT, You_PP_ID,
>>>             >> >>>> PROTO_BIT__IP );
>>>             >> >>>>
>>>             >> >>>> Remember check the following flag in your
>>>             ExampleProcess:
>>>             >> >>>>
>>>             >> >>>> (SFSnortPacket*)ipacketp)->flags & FLAG_REBUILT_FRAG
>>>             >> >>>>
>>>             >> >>>> Best,
>>>             >> >>>> Hui.
>>>             >> >>>>
>>>             >> >>>>
>>>             >> >>>> On 12/04/2013 12:52 PM, Emiliano Fausto wrote:
>>>             >> >>>>
>>>             >> >>>> Hi everybody,
>>>             >> >>>>
>>>             >> >>>> I'm creating a new preprocessor which needs to
>>>             have the whole content
>>>             >> >>>> in
>>>             >> >>>> a packet which was fragmented.
>>>             >> >>>>
>>>             >> >>>> So I thought of using the frag3 preprocessor to
>>>             re-assembly the
>>>             >> >>>> packets,
>>>             >> >>>> and then, when this reassembly is done, sent it
>>>             to my own
>>>             >> >>>> preprocessor.
>>>             >> >>>>
>>>             >> >>>> Do you know if this is possible? May I have the
>>>             output of frag3 being
>>>             >> >>>> the input of my own preprocessor?
>>>             >> >>>>
>>>             >> >>>> Regards,
>>>             >> >>>> Emiliano.
>>>             >> >>>>
>>>             >> >>>>
>>>             >> >>>>
>>>             >> >>>>
>>>             >> >>>>
>>>             ------------------------------------------------------------------------------
>>>             >> >>>> Sponsored by Intel(R) XDK
>>>             >> >>>> Develop, test and display web and hybrid apps
>>>             with a single code
>>>             >> >>>> base.
>>>             >> >>>> Download it for free now!
>>>             >> >>>>
>>>             >> >>>>
>>>             >> >>>>
>>>             http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk
>>>             >> >>>>
>>>             >> >>>>
>>>             >> >>>>
>>>             >> >>>> _______________________________________________
>>>             >> >>>> Snort-devel mailing list
>>>             >> >>>> Snort-devel at lists.sourceforge.net
>>>             <mailto:Snort-devel at lists.sourceforge.net>
>>>             >> >>>>
>>>             https://lists.sourceforge.net/lists/listinfo/snort-devel
>>>             >> >>>> Archive:
>>>             >> >>>>
>>>             http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel
>>>             >> >>>>
>>>             >> >>>> Please visit http://blog.snort.org for the
>>>             latest news about Snort!
>>>             >> >>>>
>>>             >> >>>>
>>>             >> >>>>
>>>             >> >>>>
>>>             >> >>>>
>>>             >> >>>>
>>>             ------------------------------------------------------------------------------
>>>             >> >>>> Sponsored by Intel(R) XDK
>>>             >> >>>> Develop, test and display web and hybrid apps
>>>             with a single code
>>>             >> >>>> base.
>>>             >> >>>> Download it for free now!
>>>             >> >>>>
>>>             >> >>>>
>>>             >> >>>>
>>>             http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk
>>>             >> >>>> _______________________________________________
>>>             >> >>>> Snort-devel mailing list
>>>             >> >>>> Snort-devel at lists.sourceforge.net
>>>             <mailto:Snort-devel at lists.sourceforge.net>
>>>             >> >>>>
>>>             https://lists.sourceforge.net/lists/listinfo/snort-devel
>>>             >> >>>> Archive:
>>>             >> >>>>
>>>             http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel
>>>             >> >>>>
>>>             >> >>>> Please visit http://blog.snort.org for the
>>>             latest news about Snort!
>>>             >> >>>
>>>             >> >>>
>>>             >> >>>
>>>             >> >>
>>>             >> >>
>>>             >> >
>>>             >
>>>             >
>>>
>>>
>>
>>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.snort.org/pipermail/snort-devel/attachments/20131205/ba872579/attachment.html>


More information about the Snort-devel mailing list