[Snort-users] SNORT DROPPING PACKETS

Phil Wood cpw at ...440...
Wed Jan 2 16:36:05 EST 2002


On Wed, Jan 02, 2002 at 03:38:11PM -0600, Crow, Owen wrote:
> Phil,
> 
> I've finally gotten time to play with this a bit.  Comparing my FreeBSD 4.4
> and my RH7.2 installs yields different results.
> 
> On FreeBSD, I'm just using the stock libpcap that comes with 4.4.  On the
> Redhat 7.2 system, I've downloaded and installed libcap-2002.01.02 and Snort
> 1.8.3.
> 
> If I start up Snort the same way on both hosts and then setup a loop to send
> a SIGUSR1 to the snort process every 5 seconds, I get strange results on the
> Redhat system.  Each SIGUSR1 seems to reset the packet statistics:
> 
> Snort analyzed 38145 out of 40324 packets, dropping 2179(5.404%) packets
> Snort analyzed 43496 out of 47194 packets, dropping 3698(7.836%) packets
> Snort analyzed 40218 out of 42558 packets, dropping 2340(5.498%) packets
> Snort analyzed 45146 out of 46521 packets, dropping 1375(2.956%) packets
> Snort analyzed 40368 out of 41268 packets, dropping 900(2.181%) packets
> 
> On FreeBSD, this just increases (as expected):
> Snort analyzed 778344 out of 971391 packets, dropping 193047(19.873%)
> packets
> Snort analyzed 802363 out of 1001364 packets, dropping 199001(19.873%)
> packets
> Snort analyzed 829366 out of 1037311 packets, dropping 207945(20.047%)
> packets
> Snort analyzed 853693 out of 1067773 packets, dropping 214080(20.049%)
> packets
> Snort analyzed 879381 out of 1101073 packets, dropping 221692(20.134%)
> packets
> 

> Any ideas?  My Redhat has been updated with all the latest packages for 7.2
> (using APT4RPM), kernel is "stock" 2.4.7-10.

I'd like to help, but this first cut will be quick:

  1. It looks like freebsd system call does not clear kernel stats
     whereas the linux one does.  I could modify this behavior, but don't
     know which api I prefer.  I kind of like the clear behavior, cause
     it means I could run a program for a long time and never get a wrap,
     and let the application do really long arithmetic.

  2. Are the FreeBSD and Linux runs concurrent with a USR1 every N seconds?
     cause the differences are monumental.

  3. I'd make sure and run the tests with no filter as in "".   

  4. I've run into problems when building different versions related to
     re running the configure program each time to make sure that the
     proper pcap includes and libraries are applied.  This is especially
     true with shared libs.

  5. I've run a tcpdump with basically the libpcap changes indicated in my
     previous message and compared the results with the actual interface
     statitistics provided by /proc/net/dev.  Usually, I'm off by a small
     delta of packets due to the fact that I'm doing a cat /proc/net/dev
     before and after, like so:

     Try this around your favorite linux libpcap program:

     (don't worry about PCAP_FRAMES, it is used by my libpcap to change
      the size of the ring buffer without modifying tcpdump)

      #!/bin/sh

      face=$1
      
      if [ -z "$PCAP_FRAMES" ]; then export PCAP_FRAMES=10000; fi
      
      set `awk '/'$face':/ {print $2" "$10}' < /proc/net/dev`
      pkt0=`expr $1 + $2`
      time tcpdump -i $face -w /dev/null -c $PCAP_FRAMES
      set `awk '/'$face':/ {print $2" "$10}' < /proc/net/dev`
      pkt1=`expr $1 + $2`
      echo "/proc/net/dev:$face saw `expr $pkt1 - $pkt0` packets."
 
    This will show the actual # of packets "in + out" on the inter[face]
    in question during the tcpdump run.  (which is why I mention to use
    an "all packets" filter.)


  6. As for a patch, I was premature to release a pointer to my modified 
     libpcap.  I've got an issue (totally bogus stats!) which only happens
     on one system.  I'm thinking I have a disk going south, but little
     evidence yet.  Until I know for sure, I'm holding back on any sharing
     of beta code.

  If and when you ever want to try my library (as-is), I'll include a modified
  tcpdump that dumps statistics at a specified interval, using some stats
  I built into libpcap.  It dumps stats to stderr while collecting packets.
  I put the above wrapper around the run to see if my stats are in the
  ball park (including, of course pcap_stats).  Looks like this:

   # PACKETS=100000 ./teststats

   real    0m12.921s
   user    0m0.110s
   sys     0m0.270s
   statistics in /tmp/ee-1000-max-68-17.stats
   raw data in /tmp/ee-1000-max-68-17.ring

     packet time       received by filter
                       |    ignore (comes into play if not using kernel filter)
                       |    | specious (don't want to see this, algorithm prob)
                       |    | | bytes received by filter
                       |    | | |       ring size (might remove, since its set)
                       |    | | |       |     ring index
                       |    | | |       |     |     max packets received of ring
                       |    | | |       |     |     |prior to having to poll
                       |    | | |       |     |     |  number of polls (syscall)
                       |    | | |       |     |     |  |    dropped packets
                       |    | | |       |     |     |  |    |
   S:1010014643.596991 7644 0 0 4833780 32750 23661 10 4444 0
   S:1010014644.596368 7530 0 0 4600061 32750 31191 10 4383 0
   S:1010014645.595467 7287 0 0 4254035 32750 5728   9 4223 0
   S:1010014646.594601 7795 0 0 4780128 32750 13523  9 4477 0
   S:1010014647.593702 7120 0 0 4400179 32750 20643  9 4186 0
   S:1010014648.592894 7238 0 0 4508644 32750 27881  9 4295 0
   S:1010014649.591924 6619 0 0 4048638 32750 1750  11 3870 0

   100000 packets received by filter
   0 packets dropped by kernel

   eth2 saw 100004 packets. <- generated from diff of /proc/net/dev
                               which matches well with 100000 packets received.

> 
> I also tried this with a Debian 2.2 system by adding the patch in #3 below
> to the 0.6.2 libpcap output, but never got any dropped packets.  I didn't
> want to use the daily because this is the system were the daily caused snort

Ah, the daily is the current release from tcpdump.

Well, if /usr/include/linux/if_packet.h has PACKET_STATISTICS and you
have chosen the correct options when building the kernel, you might get
the attached patch to work.

Let me know how it goes.

> and tcpdump to core dump.  Can you provide a patch file for the necessary
> changes to 0.6.2?
> 
> Regards,
> Owen Crow
> Systems Programmer (Unix)
> BMC Software, Inc.
> 
> -----Original Message-----
> From: Phil Wood [mailto:cpw at ...440...]
> Sent: Sunday, December 23, 2001 6:42 PM
> To: Crow, Owen
> Cc: 'Greg Herlein'; snort-users at lists.sourceforge.net
> Subject: Re: [Snort-users] SNORT DROPPING PACKETS
> 
> 
> IF YOU DO NOT DO LINUX, JUST DELETE THIS MESSAGE
> 
> Ok, if I might be so brazen, I'm quite familiar with the linux/pcap_stats
> issue.
> 
> I published a patch to tcpdump.org libpcap this month to fix the problem
> you are refering to.  Some facts.
> 
>   1. ftp://ftp.ee.lbl.gov/libpcap.tar.Z
> 
>      Does NOT know about dropped packets.  This is the extent of it's
>      abilities:
> 
>         int
>         pcap_stats(pcap_t *p, struct pcap_stat *ps)
>         {
>         
>                 *ps = p->md.stat;
>                 return (0);
>         }
> 
>   2. http://www.tcpdump.org/release/libpcap-0.6.2.tar.gz
> 
>      Does NOT know about dropped packets.  This is the extent of it's
>      abilities:
>      
>         int
>         pcap_stats(pcap_t *handle, struct pcap_stat *stats)
>         {
>             *stats = handle->md.stat;
>             return 0;
>         }
> 
>   3. http://www.tcpdump.org/daily/libpcap-current.tar.gz
> 
>      Does know about dropped packets!  It did not know earlier in December
>      prior to the fix I so boldly presented to
> snort-users at lists.sourceforge.net
>      and tcpdump-workers at ...4406...  My contribution was improved by
>      Guy Harris of tcpdump.org.
> 
>         int
>         pcap_stats(pcap_t *handle, struct pcap_stat *stats)
>         {
>         #ifdef HAVE_TPACKET_STATS
>         	struct tpacket_stats kstats;
>         	socklen_t len = sizeof (struct tpacket_stats);
>         
>         	/*
>         	 * Try to get the packet counts from the kernel.
>         	 */
>         	if (getsockopt(handle->fd, SOL_PACKET, PACKET_STATISTICS,
>         			&kstats, &len) > -1) {
>         		/*
>         		 * In "linux/net/packet/af_packet.c", at least in
> the
>         		 * 2.4.9 kernel, "tp_packets" is incremented for
> every
>         		 * packet that passes the packet filter *and* is
>         		 * successfully queued on the socket; "tp_drops" is
>         		 * incremented for every packet dropped because
> there's
>         		 * not enough free space in the socket buffer.
>         		 *
>         		 * When the statistics are returned for a
> PACKET_STATISTICS
>         		 * "getsockopt()" call, "tp_drops" is added to
> "tp_packets",
>         		 * so that "tp_packets" counts all packets handed to
>         		 * the PF_PACKET socket, including packets dropped
> because
>         		 * there wasn't room on the socket buffer - but not
>         		 * including packets that didn't pass the filter.
>         		 *
>         		 * In the BSD BPF, the count of received packets is
>         		 * incremented for every packet handed to BPF,
> regardless
>         		 * of whether it passed the filter.
>         		 *
>         		 * We can't make "pcap_stats()" work the same on
> both
>         		 * platforms, but the best approximation is to
> return
>         		 * "tp_packets" as the count of packets and
> "tp_drops"
>         		 * as the count of drops.
>         		 */
>         		handle->md.stat.ps_recv = kstats.tp_packets;
>         		handle->md.stat.ps_drop = kstats.tp_drops;
>         	}
>         	else
>         	{
>         		/*
>         		 * If the error was EOPNOTSUPP, fall through, so
> that
>         		 * if you build the library on a system with
>         		 * "struct tpacket_stats" and run it on a system
>         		 * that doesn't, it works as it does if the library
>         		 * is built on a system without "struct
> tpacket_stats".
>         		 */
>         		if (errno != EOPNOTSUPP) {
>         			snprintf(handle->errbuf, PCAP_ERRBUF_SIZE,
>         			    "pcap_stats: %s", pcap_strerror(errno));
>         			return -1;
>         		}
>         	}
>         #endif
>         	/*
>         	 * On systems where the PACKET_STATISTICS "getsockopt()"
> argument
>         	 * is supported on PF_PACKET sockets:
>         	 *
>         	 *	"ps_recv" counts only packets that *passed* the
> filter,
>         	 *	not packets that didn't pass the filter.  This
> includes
>         	 *	packets later dropped because we ran out of buffer
> space.
>         	 *
>         	 *	"ps_drop" counts packets dropped because we ran out
> of
>         	 *	buffer space.  It doesn't count packets dropped by
> the
>         	 *	interface driver.  It counts only packets that
> passed
>         	 *	the filter.
>         	 *
>         	 *	Both statistics include packets not yet read from
> the
>         	 *	kernel by libpcap, and thus not yet seen by the
> application.
>         	 *
>         	 * On systems where the PACKET_STATISTICS "getsockopt()"
> argument
>         	 * is not supported on PF_PACKET sockets:
>         	 *
>         	 *	"ps_recv" counts only packets that *passed* the
> filter,
>         	 *	not packets that didn't pass the filter.  It does
> not
>         	 *	count packets dropped because we ran out of buffer
>         	 *	space.
>         	 *
>         	 *	"ps_drop" is not supported.
>         	 *
>         	 *	"ps_recv" doesn't include packets not yet read from
>         	 *	the kernel by libpcap.
>         	 */
>         	*stats = handle->md.stat;
>         	return 0;
>         }
> 
>      4. http://home.lanl.gov/cpw/libpcap-0.6.tar.gz
> 
>         This is a pcap hacked to include the ring buffer stuff
>         created by Alexey Kuznetsov.  It will compile on both 2.2 and
>         2.4.  However, you would need patches to the 2.2 kernel which I do
>         not have at my finger tips.  On 2.4 kernel if you install the
>         shared libraries, you should be able to get tcpdump to use the ring
>         buffer with a few judicious environment variables.  THIS IS BETA.
>         So, if you do use it, please report problems to cornett at ...4407...  
> 
>         Anyone want to try and build snort with it?  I'd appreciate
>         a few of you brave sorts trying it out.  Read the files in
>         the libpcap directory, README.linux and README.ring.
> 
> On Sun, Dec 23, 2001 at 12:45:23PM -0600, Crow, Owen wrote:
> > Sorry, the thing that I haven't gotten to work on Linux is an accurate
> count
> > of dropped packets.  That's not clear the way I wrote it.  The SIGUSR1
> thing
> > still prints out the stats.
> > 
> > I'm a Debian user mostly, but after upgrading the kernel and recompiling
> > libpcap and snort (and tcpdump, too) I always got 0% packets dropped on a
> > fully saturated 100Mbit pipe.  I tried investigating, but there seems to
> be
> > very little info on getting this to work accurately in 2.4 (not in any
> FAQs
> > I could find, etc.)  I tried the latest CVS of libpcap, but it just made
> > snort core dump.  If anyone knows how to make it work, please fess up.
> > Answers containing "switch to Redhat/SuSE" will be ignored.
> > 
> > Peace,
> > Owen
> > 
> > -----Original Message-----
> > From: Greg Herlein [mailto:gherlein at ...3379...]
> > Sent: Sunday, December 23, 2001 11:17 AM
> > To: Crow, Owen
> > Cc: 'Bartholomew Simpson'; snort-users at lists.sourceforge.net
> > Subject: RE: [Snort-users] SNORT DROPPING PACKETS
> > 
> > 
> > > I was surprised to find this isn't listed in the manual or the FAQ list.
> > > It's been covered repeatedly on the list.
> > > 
> > > On Unix systems send SIGUSR1 to the Snort process:
> > > 
> > > 	kill -USR1 <snort_pid>
> > > 
> > > I don't know how on Win32 systems.  Linux systems require patching to
> make
> > > this work as far as I can tell.  I never found out how on a 2.4 system,
> so
> > I
> > > switched to FreeBSD.  
> > 
> > This is the kind of anti-linux FUD that sometimes really annoys
> > me.  Of course sending signals works on linux - you don't need a
> > patch.
> > 
> > It works fine - I just did it on a stock SuSE 2.4.4 kernel and a
> > 2.4.14 kernel - both production machines.
> > 
> > Nothing against FreeBSD - that's a fine OS as well.  But geesh,
> > check your facts about linux.
> > 
> > Now Win32 - that's another story.  I won't go there.
> > 
> > Greg
> > 
> > _______________________________________________
> > Snort-users mailing list
> > Snort-users at lists.sourceforge.net
> > Go to this URL to change user options or unsubscribe:
> > https://lists.sourceforge.net/lists/listinfo/snort-users
> > Snort-users list archive:
> > http://www.geocrawler.com/redir-sf.php3?list=snort-users
> 
> -- 
> Phil Wood, cpw at ...440...

-- 
Phil Wood, cpw at ...440...

-------------- next part --------------
diff -Naur -b libpcap-0.6.2/pcap-linux.c libpcap-0.6.2.x/pcap-linux.c
--- libpcap-0.6.2/pcap-linux.c	Thu Jan 18 03:59:56 2001
+++ libpcap-0.6.2.x/pcap-linux.c	Thu Jan  3 00:30:24 2002
@@ -75,11 +75,11 @@
 #include <net/if_arp.h>
 
 #ifdef HAVE_NETPACKET_PACKET_H
-# include <netpacket/packet.h>
 
  /*
   * We assume this means we really do have PF_PACKET sockets.
   */
+#include <linux/if_packet.h>
 # define HAVE_PF_PACKET_SOCKETS
 #else
  /*
@@ -449,11 +449,96 @@
 
 /*
  *  Get the statistics for the given packet capture handle.
- *  FIXME: Currently does not report the number of dropped packets.
+ *  Reports the number of dropped packets iff the kernel supports
+ *  the PACKET_STATISTICS "getsockopt()" argument (2.4 and later
+ *  kernels, and 2.2[.x] kernels with Alexey Kuznetzov's turbopacket
+ *  patches); otherwise, that information isn't available, and we lie
+ *  and report 0 as the count of dropped packets.
  */
 int
 pcap_stats(pcap_t *handle, struct pcap_stat *stats)
 {
+#ifdef PACKET_STATISTICS
+	struct tpacket_stats kstats;
+	socklen_t len = sizeof (struct tpacket_stats);
+
+	/*
+	 * Try to get the packet counts from the kernel.
+	 */
+	if (getsockopt(handle->fd, SOL_PACKET, PACKET_STATISTICS,
+			&kstats, &len) > -1) {
+		/*
+		 * In "linux/net/packet/af_packet.c", at least in the
+		 * 2.4.9 kernel, "tp_packets" is incremented for every
+		 * packet that passes the packet filter *and* is
+		 * successfully queued on the socket; "tp_drops" is
+		 * incremented for every packet dropped because there's
+		 * not enough free space in the socket buffer.
+		 *
+		 * When the statistics are returned for a PACKET_STATISTICS
+		 * "getsockopt()" call, "tp_drops" is added to "tp_packets",
+		 * so that "tp_packets" counts all packets handed to
+		 * the PF_PACKET socket, including packets dropped because
+		 * there wasn't room on the socket buffer - but not
+		 * including packets that didn't pass the filter.
+		 *
+		 * In the BSD BPF, the count of received packets is
+		 * incremented for every packet handed to BPF, regardless
+		 * of whether it passed the filter.
+		 *
+		 * We can't make "pcap_stats()" work the same on both
+		 * platforms, but the best approximation is to return
+		 * "tp_packets" as the count of packets and "tp_drops"
+		 * as the count of drops.
+		 */
+		handle->md.stat.ps_recv = kstats.tp_packets;
+		handle->md.stat.ps_drop = kstats.tp_drops;
+	}
+	else
+	{
+		/*
+		 * If the error was EOPNOTSUPP, fall through, so that
+		 * if you build the library on a system with
+		 * "struct tpacket_stats" and run it on a system
+		 * that doesn't, it works as it does if the library
+		 * is built on a system without "struct tpacket_stats".
+		 */
+		if (errno != EOPNOTSUPP) {
+			snprintf(handle->errbuf, PCAP_ERRBUF_SIZE,
+			    "pcap_stats: %s", pcap_strerror(errno));
+			return -1;
+		}
+	}
+#endif
+	/*
+	 * On systems where the PACKET_STATISTICS "getsockopt()" argument
+	 * is supported on PF_PACKET sockets:
+	 *
+	 *	"ps_recv" counts only packets that *passed* the filter,
+	 *	not packets that didn't pass the filter.  This includes
+	 *	packets later dropped because we ran out of buffer space.
+	 *
+	 *	"ps_drop" counts packets dropped because we ran out of
+	 *	buffer space.  It doesn't count packets dropped by the
+	 *	interface driver.  It counts only packets that passed
+	 *	the filter.
+	 *
+	 *	Both statistics include packets not yet read from the
+	 *	kernel by libpcap, and thus not yet seen by the application.
+	 *
+	 * On systems where the PACKET_STATISTICS "getsockopt()" argument
+	 * is not supported on PF_PACKET sockets:
+	 *
+	 *	"ps_recv" counts only packets that *passed* the filter,
+	 *	not packets that didn't pass the filter.  It does not
+	 *	count packets dropped because we ran out of buffer
+	 *	space.
+	 *
+	 *	"ps_drop" is not supported.
+	 *
+	 *	"ps_recv" doesn't include packets not yet read from
+	 *	the kernel by libpcap.
+	 */
 	*stats = handle->md.stat;
 	return 0;
 }


More information about the Snort-users mailing list