[Snort-devel] Barnyard2 Kafka

Jaime Nebrera jnebrera at ...3348...
Wed May 22 10:04:08 EDT 2013

Dear all,

The redBorder team is pleased to announce the availability of the Beta 
release of Barnyard2-Kafka plugin in our Github repository 
https://github.com/redBorder/ under GPL license.

This is an extension of Barnyard2 2-1.13 official release to add the 
following capabilities:

* Ability to send Snort events using an Apache Kafka messaging system 
* Preprocessing of certain Unified2 fields in order to provide enhanced 
meta data information
- Geolocation of IPs based on Maxmind libraries
- IP translation based on /etc/hosts & /etc/barnyard_networks information

In future releases we hope to extend the meta data fields provided (fe 
services information extracted from /etc/services) but for now we 
believe this is ok. This patch is usable, but beta quality, use at your 
own risk. Of course, we would really appreciate any help to extend the 
number of Unified2 fields supported as well as testing in real 
scenarios. We have based our contribution in CSV and SQL Barnyard2 plugins.

Apache Kafka is a new messaging system several orders of magnitude 
faster than AMPQ or similar. By using this framework, we will be able to 
more easily plug Snort events into a BigData environment. Just two ideas 
in this regard, it would enable to save Snort events in a Hadoop 
(http://hadoop.apache.org/) cluster as well as preprocess them using 
Twitter's Storm (http://storm-project.net/)

As for redBorder project, we are working on the real time management of 
the events for the GUI as well as a scale out capable correlation 
engine, that will not only process events generated by Snort but also 
from other elements in our framework. More information here 

Of course, we would like to thank our sponsors and clients for 
supporting us into making this public. Also, the Barnyard2 and Snort 
developers for their great software. We just hope this patch helps the 


PS.- I work for the company developing redBorder


1 Using alert_json barnyard2 plugin
If you want to use alert_json barnyard2 plugin, you have to put it in 
barnyard2.conf file.
The format of the argument passed to the plugin is:

output alert_json: kafka://<host>:<port>@<topic>

Where host, port and topic are the kafka host, port and topic (not 
zookeepers one).

2 Host and network in readable format:
Alert_json can can print a human readable string plus the default host 
string. For example, if you
have the hostname “foo PC” associated with the “” ip in 
/etc/hosts file, alert_json will
print “foo PC” plus “” and the number representation of the 

In the same way, alert_json can print the destination or source network 
of the packet. You have to make
an entry in “/etc/barnyard_networks” indicating this. For example, the 
entry “ foo
network” will make alert_json print the network name plus the network id.

In case alert_json does not locate the network in the file, it will 
print “” instead.

3 GeoIP
Alert_json can locate the region of the IP too. You just have to have 
libGeoIP installed and compile
the sources with GEO_IP macro defined (it's defined by default). Also, 
you have to put the database
in “/usr/local/share/GeoIP/GeoIP.dat”

If you want “alert_json” print geo-localization information too, you 
have to compile barnyard with
geo-ip support:

./configure --enable-geo-ip

Jaime Nebrera -jnebrera at ...3348...
Consultor TI - ENEO Tecnologia SL
C/ Manufactura 2, Edificio Euro, Oficina 3N
Mairena del Aljarafe - 41927 - Sevilla
Telf.- 955 60 11 60 / 619 04 55 18

More information about the Snort-devel mailing list