[Snort-users] high packet loss - low throughput

Livio Ricciulli livio at ...15149...
Tue Jul 23 13:30:11 EDT 2013


One easy way to check is to do:

tcpdump -i <iface> -n -c 1000

using the the myrcom libpcap/nic.

Then look at the length field of what tcpdump reports. If there are 
lengths  greater than the MTU of you NIC, then something is doing
reassembly.

Livio.

On 07/23/2013 01:43 AM, Michal Purzynski wrote:
> On 7/23/13 2:08 AM, Livio Ricciulli wrote:
>> Does the Myricom card/pcap library do reassembly for you? If so, you 
>> need to adjust the snort snaplen to match the myrcom maximum frame 
>> size with the -P flag.
>> Snort, by default only looks at the first 1514 bytes per packet.
>
> Thanks, emailed the Myricom support.
>
> Also:
>
> Offload parameters for eth5:
> rx-checksumming: off
> tx-checksumming: off
> scatter-gather: off
> tcp-segmentation-offload: off
> udp-fragmentation-offload: off
> generic-segmentation-offload: off
> generic-receive-offload: off
> large-receive-offload: off
> rx-vlan-offload: off
> tx-vlan-offload: off
> ntuple-filters: off
> receive-hashing: off
>
> So we should be fine, but it's worth double checking.
>
>> On 07/22/2013 04:10 PM, Michal Purzynski wrote:
>>> OK, so
>>>
>>> I've just done some more snort tuning and here are the results. I 
>>> tend to share things like this - maybe someone will benefit? You 
>>> never know :)
>>>
>>> 1. changing the number of snort processes from 20 to 10 actualy 
>>> helped. The amount of traffic a single instance has to crush is 
>>> bigger, but it seems like a scheduler contention took cycles away 
>>> from all instances, so it was a 'everybody loss' situation.
>>> 2. of course when you don't have enough instances that's bad, too. 
>>> Need to find a sweet spot.
>>> 3. The best results I got were between 120 and 180Mbit/sec. Not bad 
>>> for a 2.0Ghz CPU after all.
>>>
>>> Than I've switched to the Myricom card with vendor provided pcap 
>>> library. There's no native DAQ (yet).
>>>
>>> Snort was able to do over 633Mbit/sec with a packet loss around 
>>> 0.5%. Awesome. Actualy two instances managed to crush over 1.2Gbit.
>>>
>>> I'm now running four instances, traffic spikes at 2.5Gbit (avg 
>>> around 2Gbit) with no or very low packet loss.
>>>
>>> Scared to think what it could look like without any libpcap 
>>> emulation like the FPGA based cards do. With a card like this and a 
>>> faster CPU Joel might be actualy right that it's possible to get 
>>> more than 1Gbit/sec on a single core ;)
>>>
>>> On 7/21/13 6:08 PM, Joel Esler wrote:
>>>> If Snort sees traffic more than once, it will analyze it as many 
>>>> times as it sees it.
>>>>
>>>> The SSL preprocessing should discard an ignore a session after it 
>>>> determines the legitimate certificate exchange,
>>>> But like I said, it sounds like there is something else going on here.
>>>>
>>>> Sent from my iPhone
>>>>
>>>> On Jul 21, 2013, at 9:33 AM, Michal Purzynski <michal at ...16244... 
>>>> <mailto:michal at ...16244...>> wrote:
>>>>
>>>>> On 7/21/13 2:03 PM, Joel Esler wrote:
>>>>>> Yes, performance that low seems incorrect. I don't think it's 
>>>>>> Snort with numbers that low.
>>>>>>
>>>>>>
>>>>> Also, a question for the more experienced. I have a simple setup - 
>>>>> load balancers in front of everything, doing L7 and terminating 
>>>>> SSL. Snort gets a copy of all the traffic and that means it can see:
>>>>> 1. traffic from Internet to load balancers
>>>>> 2. traffic from LB to the backend servers
>>>>> 3. traffic from the backend to LB
>>>>> 4. traffic from the LB to the Internet
>>>>>
>>>>> It's clear it can see the same traffic twice, sometimes enrypted 
>>>>> sometimes decrypted (SSL preprocessor enabled, so the encrypted 
>>>>> traffic is being ignored).
>>>>>
>>>>> Question: does it make sense to leave it like this or should I 
>>>>> only direct the "internal" traffic to snort? You know, the one 
>>>>> between the LB <-> backend?
>>>>>
>>>>>> Sent from my iPad
>>>>>>
>>>>>> On Jul 21, 2013, at 6:16 AM, Michal Purzynski <michal at ...16244... 
>>>>>> <mailto:michal at ...16244...>> wrote:
>>>>>>
>>>>>>> On 7/21/13 2:22 AM, Joel Esler wrote:
>>>>>>>> On Jul 20, 2013, at 6:46 PM, Michal Purzynski <michal at ...16244... 
>>>>>>>> <mailto:michal at ...16244...>> wrote:
>>>>>>>>
>>>>>>>>> The sourcefire company claims to achieve 1Gbit/sec per CPU 
>>>>>>>>> core. I find
>>>>>>>>> it actualy hard to believe as the "empty" snort used to do around
>>>>>>>>> 250-300Mbit/sec per core here. Empty as in no rules at all.
>>>>>>>>
>>>>>>>> Even more.  But we have a dedicated appliance specifically 
>>>>>>>> tuned with special drivers to run Snort very fast.  You are 
>>>>>>>> doing this, I assume on commodity hardware, on a stock OS, 
>>>>>>>> running many things (Security Onion)
>>>>>>>>
>>>>>>>>
>>>>>>> Not really, SO is so wonderful you can enable and disable 
>>>>>>> functionality on demand, and so I've done. The box is running 
>>>>>>> snort and netsniff-ng only, has around 20 processes of snort (24 
>>>>>>> execution threads with HT enabled).
>>>>>>>
>>>>>>> Still - 45Mbit/sec per instance with packet loss is 
>>>>>>> disappointing. And 100 would be too.
>>>>>>>
>>>>>>> Also, I'm running Intel and pf_ring, can try a Myricom (and not 
>>>>>>> pf_ring). I won't try anything more expensive like FPGA 
>>>>>>> accelerated cards, since I find them too limited and having no 
>>>>>>> real advantage over Myricom and a lot of downsides.
>>>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> See everything from the browser to the database with AppDynamics
>>> Get end-to-end visibility with application monitoring from AppDynamics
>>> Isolate bottlenecks and diagnose root cause in seconds.
>>> Start your free trial of AppDynamics Pro today!
>>> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
>>>
>>>
>>> _______________________________________________
>>> Snort-users mailing list
>>> Snort-users at lists.sourceforge.net
>>> Go to this URL to change user options or unsubscribe:
>>> https://lists.sourceforge.net/lists/listinfo/snort-users
>>> Snort-users list archive:
>>> http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users
>>>
>>> Please visithttp://blog.snort.org  to stay current on all the latest Snort news!
>>
>>
>>
>> ------------------------------------------------------------------------------
>> See everything from the browser to the database with AppDynamics
>> Get end-to-end visibility with application monitoring from AppDynamics
>> Isolate bottlenecks and diagnose root cause in seconds.
>> Start your free trial of AppDynamics Pro today!
>> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
>>
>>
>> _______________________________________________
>> Snort-users mailing list
>> Snort-users at lists.sourceforge.net
>> Go to this URL to change user options or unsubscribe:
>> https://lists.sourceforge.net/lists/listinfo/snort-users
>> Snort-users list archive:
>> http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users
>>
>> Please visithttp://blog.snort.org  to stay current on all the latest Snort news!
>
>
>
> ------------------------------------------------------------------------------
> See everything from the browser to the database with AppDynamics
> Get end-to-end visibility with application monitoring from AppDynamics
> Isolate bottlenecks and diagnose root cause in seconds.
> Start your free trial of AppDynamics Pro today!
> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
>
>
> _______________________________________________
> Snort-users mailing list
> Snort-users at lists.sourceforge.net
> Go to this URL to change user options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users
>
> Please visit http://blog.snort.org to stay current on all the latest Snort news!

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.snort.org/pipermail/snort-users/attachments/20130723/a346453a/attachment.html>


More information about the Snort-users mailing list