[Snort-devel] Hyperscan pattern matcher integration for Snort 2.9.8.2

Viiret, Justin justin.viiret at ...3635...
Mon Jun 13 20:27:18 EDT 2016


Hi Vladimir,

Thanks for trying it out!

Depending on the pattern set, Hyperscan’s compilation process can take time – and with a  ruleset of that size, Snort will build many (perhaps thousands of) Hyperscan databases. We do track and try to minimize compile time; if you’re able to share your pattern set off-list we can take a look at it and see if there are improvements that can be made.

Hyperscan does have a straightforward serialization API, so I’m sure it would be possible to use it to cache previously compiled databases as you suggest. Most of the work would be in hashing the input patterns and ensuring that you load the correct database, plus the configuration and filesystem handling for the caching mechanism. This is too large a change to Snort for us to make as the Hyperscan team, though.

Best regards,
    Justin


From: Vladimir Kunschikov [mailto:kunschikov at ...2499...]
Sent: Friday, June 10, 2016 6:54 PM
To: Viiret, Justin
Cc: snort-devel at lists.sourceforge.net
Subject: Re: [Snort-devel] Hyperscan pattern matcher integration for Snort 2.9.8.2

Very nice patch. I applied it to snort-2.9.8.0 without any problems. The throughput of the Snort improved for sure.
 Is there a way to speed up loading of the ruleset with 'search-method hyperscan' option enabled?
 My set of the rules is quite a big one - over 20 thousands of rules. Can hyperscan objects be serialized at first run and then loaded at subsequent runs of the Snort? Is it hard to implement such serialization by usage of the Hyperscan library API?

On Thu, Jun 9, 2016 at 3:57 AM, Viiret, Justin <justin.viiret at ...3635...<mailto:justin.viiret at ...3635...>> wrote:
Hi all,

Hyperscan is a high-performance regular expression matching library from Intel, released as open source software under a 3-clause BSD license. Although there is already some use of Hyperscan in Snort++, we at Intel have also received requests for a Hyperscan integration into Snort 2.9.x.

Accordingly, we have put together a patch against Snort 2.9.8.2 which adds support for using the Hyperscan library to accelerate some of the pattern matching tasks in Snort. This integrates Hyperscan library usage into three places:

1. A new multi-pattern literal matcher (MPSE module) called "hyperscan".
2. A faster single-pattern content string matcher, replacing the Boyer-Moore approach used by default.
3. A prefilter for PCRE, where Hyperscan is used as a prefilter check for regex options before PCRE is run. Expressions that are expensive to evaluate in PCRE may be avoided entirely depending on the result of the prefilter.

You can find the patch here, including a README with instructions for its use:

    https://01.org/hyperscan/downloads/hyperscan-integration-snort-2.9.8.2

You can find more information about the Hyperscan library here:

Website: https://01.org/hyperscan
Github: https://github.com/01org/hyperscan

Please get in touch if you have any feedback on the patch!

Best regards,
    Justin


------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
_______________________________________________
Snort-devel mailing list
Snort-devel at lists.sourceforge.net<mailto:Snort-devel at lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/snort-devel
Archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel

Please visit http://blog.snort.org for the latest news about Snort!

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.snort.org/pipermail/snort-devel/attachments/20160614/720ceee7/attachment.html>


More information about the Snort-devel mailing list