[Snort-users] snort.stats key-value mapping (Solved)

Juan Jesus Prieto jjprieto at ...16842...
Tue Apr 14 07:33:01 EDT 2015


Hi, thanks Karolis.

Best regards.

El mar, 14-04-2015 a las 11:32 +0300, Karolis escribió:
> Hi Juan,
> 
> 
> 
> On the snort.stats change date in dpkg.log found this: "upgrade
> securityonion-snort 2.9.5.6-0ubuntu0securityonion1
> 2.9.7.0-0ubuntu0securityonion4"
> 
> 
> Statistics prior upgrade:
> #time,pkt_drop_percent,wire_mbits_per_sec.realtime,alerts_per_second,kpackets_wire_per_sec.realtime,avg_bytes_per_wire_packet,patmatch_percent,syns_per_second,synacks_per_second,new_sessions_per_second,deleted_sessions_per_second,total_sessions,max_sessions,stream_flushes_per_second,stream_faults,stream_timeouts,frag_creates_per_second,frag_completes_per_second,frag_inserts_per_second,frag_deletes_per_second,frag_autofrees_per_second,frag_flushes_per_second,current_frags,max_frags,frag_timeouts,frag_faults,iCPUs,usr[0],sys[0],idle[0],wire_mbits_per_sec.realtime,ipfrag_mbits_per_sec.realtime,ipreass_mbits_per_sec.realtime,rebuilt_mbits_per_sec.realtime,mbits_per_sec.realtime,avg_bytes_per_wire_packet,avg_bytes_per_ipfrag_packet,avg_bytes_per_ipreass_packet,avg_bytes_per_rebuilt_packet,avg_bytes_per_packet,kpackets_wire_per_sec.realtime,kpackets_ipfrag_per_sec.realtime,kpackets_ipreass_per_sec.realtime,kpackets_rebuilt_per_sec.realtime,kpackets_per_sec.realtime,pkt_stats.pkts_recv,pkt_stats.pkts_drop,total_blocked_packets,new_udp_sessions_per_second,deleted_udp_sessions_per_second,total_udp_sessions,max_udp_sessions,max_tcp_sessions_interval,curr_tcp_sessions_initializing,curr_tcp_sessions_established,curr_tcp_sessions_closing,tcp_sessions_midstream_per_second,tcp_sessions_closed_per_second,tcp_sessions_timedout_per_second,tcp_sessions_pruned_per_second,tcp_sessions_dropped_async_per_second,current_attribute_hosts,attribute_table_reloads,mpls_mbits_per_sec.realtime,avg_bytes_per_mpls_packet,kpackets_per_sec_mpls.realtime,total_tcp_filtered_packets,total_udp_filtered_packets,ip4::trim,ip4::tos,ip4::df,ip4::rf,ip4::ttl,ip4::opts,icmp4::echo,ip6::ttl,ip6::opts,icmp6::echo,tcp::syn_opt,tcp::opt,tcp::pad,tcp::rsv,tcp::ns,tcp::urg,tcp::urp,tcp::trim,tcp::ecn_pkt,tcp::ecn_ssn,tcp::ts_ecr,tcp::ts_nop,tcp::ips_data,tcp::block,total_injected_packets,frag3_mem_in_use,stream5_mem_in_use,total_alerts_per_second
> 
> 
> Statistics  after upgrade:
> #time,pkt_drop_percent,wire_mbits_per_sec.realtime,alerts_per_second,kpackets_wire_per_sec.realtime,avg_bytes_per_wire_packet,patmatch_percent,syns_per_second,synacks_per_second,new_sessions_per_second,deleted_sessions_per_second,total_sessions,max_sessions,stream_flushes_per_second,stream_faults,stream_timeouts,frag_creates_per_second,frag_completes_per_second,frag_inserts_per_second,frag_deletes_per_second,frag_autofrees_per_second,frag_flushes_per_second,current_frags,max_frags,frag_timeouts,frag_faults,iCPUs,usr[0],sys[0],idle[0],wire_mbits_per_sec.realtime,ipfrag_mbits_per_sec.realtime,ipreass_mbits_per_sec.realtime,rebuilt_mbits_per_sec.realtime,mbits_per_sec.realtime,avg_bytes_per_wire_packet,avg_bytes_per_ipfrag_packet,avg_bytes_per_ipreass_packet,avg_bytes_per_rebuilt_packet,avg_bytes_per_packet,kpackets_wire_per_sec.realtime,kpackets_ipfrag_per_sec.realtime,kpackets_ipreass_per_sec.realtime,kpackets_rebuilt_per_sec.realtime,kpackets_per_sec.realtime,pkt_stats.pkts_recv,pkt_stats.pkts_drop,total_blocked_verdicts,new_udp_sessions_per_second,deleted_udp_sessions_per_second,total_udp_sessions,max_udp_sessions,max_tcp_sessions_interval,curr_tcp_sessions_initializing,curr_tcp_sessions_established,curr_tcp_sessions_closing,tcp_sessions_midstream_per_second,tcp_sessions_closed_per_second,tcp_sessions_timedout_per_second,tcp_sessions_pruned_per_second,tcp_sessions_dropped_async_per_second,current_attribute_hosts,attribute_table_reloads,mpls_mbits_per_sec.realtime,avg_bytes_per_mpls_packet,kpackets_per_sec_mpls.realtime,total_tcp_filtered_packets,total_udp_filtered_packets,num_normalizations,ip4::trim,ip4::tos,ip4::df,ip4::rf,ip4::ttl,ip4::opts,icmp4::echo,ip6::ttl,ip6::opts,icmp6::echo,tcp::syn_opt,tcp::opt,tcp::pad,tcp::rsv,tcp::ns,tcp::urp,tcp::ecn_pkt,tcp::ecn_ssn,tcp::ts_ecr,tcp::ts_nop,tcp::ips_data,tcp::block,tcp::req_urg,tcp::req_pay,tcp::req_urp,tcp::trim_syn,tcp::trim_rst,tcp::trim_win,tcp::trim_mss,would_ip4::trim,would_ip4::tos,would_ip4::df,would_ip4::rf,would_ip4::ttl,would_ip4::opts,would_icmp4::echo,would_ip6::ttl,would_ip6::opts,would_icmp6::echo,would_tcp::syn_opt,would_tcp::opt,would_tcp::pad,would_tcp::rsv,would_tcp::ns,would_tcp::urp,would_tcp::ecn_pkt,would_tcp::ecn_ssn,would_tcp::ts_ecr,would_tcp::ts_nop,would_tcp::ips_data,would_tcp::block,would_tcp::req_urg,would_tcp::req_pay,would_tcp::req_urp,would_tcp::trim_syn,would_tcp::trim_rst,would_tcp::trim_win,would_tcp::trim_mss,total_injected_packets,frag3_mem_in_use,stream5_mem_in_use,total_alerts_per_second
> 
> 
> Karolis
> 
> 
> 
> On Tue, Apr 14, 2015 at 10:22 AM, Juan Jesus Prieto
> <jjprieto at ...16842...> wrote:
> 
>         Hi Karolis,
>         
>           What version of snort are you testing? I would like to check
>         the source code for the perfmonitor preprocessor.
>         
>         Regrads.
>         
>         
>         
>         El lun, 13-04-2015 a las 20:29 +0300, Karolis escribió: 
>         
>         > Hi Juan,
>         > 
>         > 
>         > I have found  the root cause of the problem. Snort all the
>         > time outputted key-value pairs correctly.  It seems that
>         > snort upgrade changed the number of statistics monitored. I
>         > have formed the array exactly as you are "head'ed" the keys
>         > and "tail'ed" the latest values whats why they do not
>         > correlate anymore. I will modify the script so it reads keys
>         > from the end of the file to avoid such problems in the
>         > future.
>         > 
>         > 
>         > Karolis
>         > 
>         > 
>         > 
>         > On Mon, Apr 13, 2015 at 10:59 AM, Juan Jesus Prieto
>         > <jjprieto at ...16842...> wrote:
>         > 
>         >         Hi Karolis,
>         >         
>         >           Could you attach a stat file content example?
>         >         every key should be accompanied with their
>         >         corresponding value, one on one.
>         >         
>         >         Regards. 
>         >         
>         >         
>         >         El jue, 09-04-2015 a las 19:46 +0300, Karolis
>         >         escribió: 
>         >         
>         >         > Hi Juan,
>         >         > 
>         >         > 
>         >         > Thanks for reply. I have got same associative
>         >         > array but can I rely on it? 
>         >         > As I mentioned there are 96 keys and 131 values in
>         >         > the snort.stats file.
>         >         > How do you know that first 96 keys correspond to
>         >         > the first 96 values
>         >         > on one to one relationship and only the last
>         >         > values misses keys?
>         >         > Can it be what there are gaps in key value pairs
>         >         > eg. key 10 correspond to value 12?
>         >         > 
>         >         > 
>         >         > Karolis 
>         >         > 
>         >         > 
>         >         > 
>         >         > 
>         >         > 
>         >         > On Mon, Apr 6, 2015 at 11:14 AM, Juan Jesus Prieto
>         >         > <jjprieto at ...16842...> wrote:
>         >         > 
>         >         >         Hi Karolis,
>         >         >         
>         >         >           The manual is out-of-date at this point.
>         >         >         I use scripting for dinamically map this
>         >         >         pairs. For example:
>         >         >         
>         >         >         
>         >         >         # declare -A v; \
>         >         >         keys=( $(head /var/log/snort/snort.stats -n2 | tail -n1 | sed 's/^#//' | tr ',' ' ') ); \
>         >         >         count=0; \
>         >         >         for n in $(tail /var/log/snort/snort.stats -n1 | tr ',' ' '); do \
>         >         >            v[${keys[$count]}]=$n; \
>         >         >            count=$(($count+1)); \
>         >         >            done; \
>         >         >         echo "stream5_mem_in_use: ${v['stream5_mem_in_use']}"; \
>         >         >         echo "curr_tcp_sessions_established: ${v['curr_tcp_sessions_established']}"
>         >         >         stream5_mem_in_use: 13950060
>         >         >         curr_tcp_sessions_established: 5195
>         >         >         
>         >         >         
>         >         >         
>         >         >         This small script will map into a hash
>         >         >         (named 'v') all pairs key/value and
>         >         >         present last values from stats file
>         >         >         (stream5_mem_in_use and
>         >         >         curr_tcp_sessions_established in this
>         >         >         example).
>         >         >         
>         >         >         Another option is to use my snmp passthrou
>         >         >         agent:
>         >         >         
>         >         >         https://github.com/redBorder/rb_snmp_pass
>         >         >         
>         >         >         You will need to adapt it for your case. 
>         >         >         
>         >         >         
>         >         >         El mar, 31-03-2015 a las 10:03 +0300,
>         >         >         Karolis escribió: 
>         >         >         
>         >         >         > Hi,
>         >         >         > 
>         >         >         > I am trying to map perfmonitor
>         >         >         > preprocessors statistics keys to values.
>         >         >         > 
>         >         >         > 
>         >         >         > config:
>         >         >         > preprocessor perfmonitor: time 300
>         >         >         > file /nsm/sensor_data/"sensor-name"/snort.stats pktcnt 10000
>         >         >         > 
>         >         >         > 
>         >         >         > 
>         >         >         > snort manual  states "There are over 100
>         >         >         > individual statistics included. A header
>         >         >         > line is output at startup and rollover
>         >         >         > that labels each column." although only
>         >         >         > 75 keys are listed.
>         >         >         > 
>         >         >         > 
>         >         >         > snort.stats file has 96 keys  and 131
>         >         >         > values. 
>         >         >         > 
>         >         >         > 
>         >         >         > How can I correctly map keys to values?
>         >         >         > 
>         >         >         > 
>         >         >         > Karolis
>         >         >         > 
>         >         >         > 
>         >         >         > 
>         >         >         > 
>         >         >         > 
>         >         >         > 
>         >         >         > 
>         >         >         > ------------------------------------------------------------------------------
>         >         >         > Dive into the World of Parallel Programming The Go Parallel Website, sponsored
>         >         >         > by Intel and developed in partnership with Slashdot Media, is your hub for all
>         >         >         > things parallel software development, from weekly thought leadership blogs to
>         >         >         > news, videos, case studies, tutorials and more. Take a look and join the 
>         >         >         > conversation now. http://goparallel.sourceforge.net/
>         >         >         > _______________________________________________ Snort-users mailing list Snort-users at lists.sourceforge.net Go to this URL to change user options or unsubscribe: https://lists.sourceforge.net/lists/listinfo/snort-users Snort-users list archive: http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users Please visit http://blog.snort.org to stay current on all the latest Snort news!
>         >         >         
>         >         >         
>         >         >         
>         >         >         
>         >         >         ------------------------------------------------------------------------------
>         >         >         BPM Camp - Free Virtual Workshop May 6th
>         >         >         at 10am PDT/1PM EDT
>         >         >         Develop your own process in accordance
>         >         >         with the BPMN 2 standard
>         >         >         Learn Process modeling best practices with
>         >         >         Bonita BPM through live exercises
>         >         >         http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
>         >         >         source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
>         >         >         _______________________________________________
>         >         >         Snort-users mailing list
>         >         >         Snort-users at lists.sourceforge.net
>         >         >         Go to this URL to change user options or
>         >         >         unsubscribe:
>         >         >         https://lists.sourceforge.net/lists/listinfo/snort-users
>         >         >         Snort-users list archive:
>         >         >         http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users
>         >         >         
>         >         >         Please visit http://blog.snort.org to stay
>         >         >         current on all the latest Snort news! 
>         >         > 
>         >         > 
>         >         > 
>         >         > 
>         >         > ------------------------------------------------------------------------------
>         >         > BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
>         >         > Develop your own process in accordance with the BPMN 2 standard
>         >         > Learn Process modeling best practices with Bonita BPM through live exercises
>         >         > http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
>         >         > source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
>         >         > _______________________________________________ Snort-users mailing list Snort-users at lists.sourceforge.net Go to this URL to change user options or unsubscribe: https://lists.sourceforge.net/lists/listinfo/snort-users Snort-users list archive: http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users Please visit http://blog.snort.org to stay current on all the latest Snort news!
>         >         
>         >         
>         >         
>         >         
>         >         ------------------------------------------------------------------------------
>         >         BPM Camp - Free Virtual Workshop May 6th at 10am
>         >         PDT/1PM EDT
>         >         Develop your own process in accordance with the BPMN
>         >         2 standard
>         >         Learn Process modeling best practices with Bonita
>         >         BPM through live exercises
>         >         http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
>         >         source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
>         >         _______________________________________________
>         >         Snort-users mailing list
>         >         Snort-users at lists.sourceforge.net
>         >         Go to this URL to change user options or
>         >         unsubscribe:
>         >         https://lists.sourceforge.net/lists/listinfo/snort-users
>         >         Snort-users list archive:
>         >         http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users
>         >         
>         >         Please visit http://blog.snort.org to stay current
>         >         on all the latest Snort news! 
>         > 
>         > 
>         > 
>         > 
>         > ------------------------------------------------------------------------------
>         > BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
>         > Develop your own process in accordance with the BPMN 2 standard
>         > Learn Process modeling best practices with Bonita BPM through live exercises
>         > http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
>         > source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
>         > _______________________________________________ Snort-users mailing list Snort-users at lists.sourceforge.net Go to this URL to change user options or unsubscribe: https://lists.sourceforge.net/lists/listinfo/snort-users Snort-users list archive: http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users Please visit http://blog.snort.org to stay current on all the latest Snort news!
>         
>         
>         
>         
>         
>         ------------------------------------------------------------------------------
>         BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
>         Develop your own process in accordance with the BPMN 2
>         standard
>         Learn Process modeling best practices with Bonita BPM through
>         live exercises
>         http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
>         source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
>         _______________________________________________
>         Snort-users mailing list
>         Snort-users at lists.sourceforge.net
>         Go to this URL to change user options or unsubscribe:
>         https://lists.sourceforge.net/lists/listinfo/snort-users
>         Snort-users list archive:
>         http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users
>         
>         Please visit http://blog.snort.org to stay current on all the
>         latest Snort news!
> 
> 
> 
> 
> ------------------------------------------------------------------------------
> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
> Develop your own process in accordance with the BPMN 2 standard
> Learn Process modeling best practices with Bonita BPM through live exercises
> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
> _______________________________________________ Snort-users mailing list Snort-users at lists.sourceforge.net Go to this URL to change user options or unsubscribe: https://lists.sourceforge.net/lists/listinfo/snort-users Snort-users list archive: http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users Please visit http://blog.snort.org to stay current on all the latest Snort news!


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.snort.org/pipermail/snort-users/attachments/20150414/d793ab35/attachment.html>


More information about the Snort-users mailing list