[Snort-users] Intermittent Pulled Pork Error

Weir, Jason jason.weir at ...14916...
Fri Feb 18 08:26:06 EST 2011


Sorry to reply to my own message but have some more info..

With verbose logging enabled it outputs the time required for successful download..

From midnight last night to 6:00 am this morning here are download times for the 33 byte md5 file

12:00	17 seconds
1:00	45 seconds 
2:00	26 seconds
3:00	48 seconds
4:00	27 seconds
5:00	27 seconds
6:00	25 seconds

All of these would have failed if the timeout had been @ 15 seconds..

OK Nigel & Joel - back in your court - doubtful those are acceptable download times for such a small file..

-J

> -----Original Message-----
> From: Weir, Jason 
> Sent: Friday, February 18, 2011 8:16 AM
> To: 'JJC'; Joel Esler; Snort Users; Nigel Houghton
> Subject: RE: [Snort-users] Intermittent Pulled Pork Error
> 
> 
> JJ - et all...
> 
> On line 1326 of pulledpork.pl I changed the timeout from
> 
> $ua->timeout(15); to
> $ua->timeout(60);
> 
> It seems to have fixed the problem!!  Could this really be 
> just a latency issue?
> 
> -J
> 
> > -----Original Message-----
> > From: JJC [mailto:cummingsj at ...11827...] 
> > Sent: Thursday, February 17, 2011 4:23 PM
> > To: Weir, Jason
> > Cc: Joel Esler; Snort Users; Nigel Houghton
> > Subject: Re: [Snort-users] Intermittent Pulled Pork Error
> > 
> > 
> > I would also be curious if you used 0.6.0 Dev if that would show the
> > same issues.  As to the tarball stuff, PP automates the filenaming
> > when you are puling from snort.org.. so that's why you see the
> > difference from what you specified to what it's trying to pull...
> > 
> > JJC
> > 
> > On Thu, Feb 17, 2011 at 11:47 AM, Weir, Jason 
> > <jason.weir at ...14916...> wrote:
> > > OK - finally got some additional output..
> > >
> > > First off here is the rule_url line in pulledpork.conf
> > >
> > > 
> > rule_url=https://www.snort.org/reg-rules/|snortrules-snapshot-
> > edge.tar.g
> > > z|<OINKCODE>
> > >
> > > And here is the -vv output
> > >
> > > ****************************************************
> > >
> > > /etc/cron.hourly/pulledpork:
> > >
> > >    http://code.google.com/p/pulledpork/
> > >      _____ ____
> > >     `----,\    )
> > >      `--==\\  /    PulledPork v0.5.0 The Drowning Rat
> > >       `--==\\/
> > >     .-~~~~-.Y|\\_  Copyright (C) 2009-2010 JJ Cummings
> > >  @_/        /  66\_  cummingsj at ...11827...
> > >    |    \   \   _(")
> > >     \   /-| ||'--'  Rules give me wings!
> > >      \_\  \_\\
> > >  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > >
> > > Command Line Variable Debug:
> > >        Config Path is: /etc/snort/pulledpork.conf
> > >        Verbose Flag is Set
> > >        Extra Verbose Flag is Set
> > >        Logging Flag is Set
> > >        Text Rules only Flag is Set
> > > Config File Variable Debug /etc/snort/pulledpork.conf
> > >        snort_path = /usr/local/bin/snort
> > >        enablesid = /etc/snort/enablesid.conf
> > >        modifysid = /etc/snort/modifysid.conf
> > >        rule_path = /etc/snort/rules/snort.rules
> > >        ignore = deleted,experimental,local
> > >        rule_url = ARRAY(0xa31bbd0)
> > >        snort_version = 2.9.0.4
> > >        sid_changelog = /var/log/sid_changes.log
> > >        sid_msg = /etc/snort/sid-msg.map
> > >        config_path = /etc/snort/snort.conf
> > >        sostub_path = /usr/local/etc/snort/rules/so_rules.rules
> > >        temp_path = /tmp
> > >        distro = Debian-Lenny
> > >        version = 0.5.0
> > >        sorule_path = /usr/local/lib/snort_dynamicrules/
> > >        disablesid = /etc/snort/disablesid.conf
> > >        local_rules = /etc/snort/rules/local.rules
> > > ** GET
> > > 
> > https://www.snort.org/reg-rules/snortrules-snapshot-2904.tar.g
> > z.md5/<oin
> > > kcode> ==>
> > > SSL_connect:before/connect initialization
> > > SSL_connect:SSLv2/v3 write client hello A
> > > SSL_connect:SSLv3 read server hello A
> > > SSL_connect:SSLv3 read server certificate A
> > > SSL_connect:SSLv3 read server done A
> > > SSL_connect:SSLv3 write client key exchange A
> > > SSL_connect:SSLv3 write change cipher spec A
> > > SSL_connect:SSLv3 write finished A
> > > SSL_connect:SSLv3 flush data
> > > SSL_connect:SSLv3 read finished A
> > > 500 SSL read timeout:  (15s)
> > >        Error 500 when fetching
> > > 
> > 
> https://www.snort.org/reg-rules/snortrules-snapshot-2904.tar.gz.md5 at
> > > /usr/local/bin/pulledpork.pl line 390
> > >        main::md5file('f1377e308ed944bcd44aa273f3eb8bf446a388dc',
> > > 'snortrules-snapshot-2904.tar.gz', '/tmp/',
> > > 'https://www.snort.org/reg-rules/') called at
> > > /usr/local/bin/pulledpork.pl line 1386
> > > Checking latest MD5 for snortrules-snapshot-2904.tar.gz....
> > >        Fetching md5sum for: snortrules-snapshot-2904.tar.gz.md5
> > > Stopping Snort and Barnyard:.
> > > ****************************************************
> > >
> > > JJ - we also need a Debian-Squeeze distro option..
> > >
> > > -J
> > >
> > >> -----Original Message-----
> > >> From: Weir, Jason [mailto:jason.weir at ...14916...]
> > >> Sent: Thursday, February 17, 2011 1:38 PM
> > >> To: JJ Cummings; Joel Esler; Snort Users; Nigel Houghton
> > >> Subject: Re: [Snort-users] Intermittent Pulled Pork Error
> > >>
> > >>
> > >> I agree that it shouldn't be a PP problem but when 
> > oinkmaster works at
> > >> the same time it makes you wonder...
> > >>
> > >> I added -vv per JJ below..
> > >>
> > >> Now I'm trying to make it fail by running the script manually..
> > >>
> > >> It works without error every time..  I'll have to wait for 
> > cron to run
> > >> it and if it fails I'll provide the output..
> > >>
> > >> -J
> > >>
> > >>
> > >> > -----Original Message-----
> > >> > From: JJ Cummings [mailto:cummingsj at ...11827...]
> > >> > Sent: Thursday, February 17, 2011 12:35 PM
> > >> > To: Weir, Jason
> > >> > Cc: Joel Esler; Snort Users; Nigel Houghton
> > >> > Subject: Re: [Snort-users] Intermittent Pulled Pork Error
> > >> >
> > >> >
> > >> > That is correct, md5 check then download or not, depending on
> > >> > hash change... As to the intermittent failures, I don't see
> > >> > what could be causing this in PP but if we can get the extra
> > >> > verbose output, might prove useful... (-vv)
> > >> >
> > >> > Sent from the iRoad
> > >> >
> > >> > On Feb 17, 2011, at 5:29, "Weir, Jason" 
> > <jason.weir at ...14916...> wrote:
> > >> >
> > >> > > Unless I'm incorrect - I'm only pulling rules when the md5
> > >> > hash file has
> > >> > > changed... I do have PP checking every couple hours 
> > (cron) for an
> > >> > > updated md5.
> > >> > >
> > >> > > I know that's way more often then you push updates, but it
> > >> > should have
> > >> > > no effect on the file availability...
> > >> > >
> > >> > > FYI - overnight PP fetching the 2.9.0.4 rules failed half
> > >> the time,
> > >> > > another sensor still using oinkmaster fetching the 2.8.6.1
> > >> > rules worked
> > >> > > without error every time..
> > >> > >
> > >> > > So maybe this is a PP problem???
> > >> > >
> > >> > > -J
> > >> > >
> > >> > >> -----Original Message-----
> > >> > >> From: Joel Esler [mailto:jesler at ...1935...]
> > >> > >> Sent: Wednesday, February 16, 2011 10:04 PM
> > >> > >> To: Weir, Jason
> > >> > >> Cc: Nigel Houghton; Snort Users
> > >> > >> Subject: Re: [Snort-users] Intermittent Pulled Pork Error
> > >> > >>
> > >> > >>
> > >> > >> We shouldn't. We've notified the web-team. How often are you
> > >> > >> trying to pull rule updates?  Just out of curiosity.
> > >> > >>
> > >> > >> --
> > >> > >> Sent from my iPad
> > >> > >> Please excuse the brevity
> > >> > >>
> > >> > >> On Feb 16, 2011, at 4:04 PM, "Weir, Jason"
> > >> > >> <jason.weir at ...14916...> wrote:
> > >> > >>
> > >> > >>> Nigel,
> > >> > >>>
> > >> > >>> I changed the rules file name to
> > >> > snortrules-snapshot-edge.tar.gz as
> > >> > >>> indicated below and I'm intermittently still getting the
> > >> > 500 error..
> > >> > >>>
> > >> > >>> "Error 500 when fetching
> > >> > >>>
> > >> > >>
> > >> >
> > >> 
> > 
> https://www.snort.org/reg-rules/snortrules-snapshot-2904.tar.gz.md5 at
> > >> > >>> /usr/local/bin/pulledpork.pl line 390"
> > >> > >>>
> > >> > >>> Just tried it manually and it worked fine...  You guys
> > >> > >> having a delivery
> > >> > >>> problem?
> > >> > >>>
> > >> > >>> -J
> > >> > >>>
> > >> > >>>> -----Original Message-----
> > >> > >>>> From: Nigel Houghton [mailto:nhoughton at ...1935...]
> > >> > >>>> Sent: Wednesday, February 16, 2011 1:38 PM
> > >> > >>>> To: Weir, Jason
> > >> > >>>> Cc: Snort Users
> > >> > >>>> Subject: Re: [Snort-users] Intermittent Pulled Pork Error
> > >> > >>>>
> > >> > >>>>
> > >> > >>>> On Wed, 16 Feb 2011 13:32:45 -0500, Nigel Houghton wrote:
> > >> > >>>>> On Wed, 16 Feb 2011 13:05:09 -0500, Weir, Jason wrote:
> > >> > >>>>>> Doesn't happen all of the time...
> > >> > >>>>>>
> > >> > >>>>>> Error 500 when fetching
> > >> > >>>>>>
> > >> > >>>>
> > >> > >>
> > >> >
> > >> 
> > 
> https://www.snort.org/reg-rules/snortrules-snapshot-2904.tar.gz.md5 at
> > >> > >>>>>> /usr/local/bin/pulledpork.pl line 390
> > >> > >>>>>>
> > >> > >>>>>> -J
> > >> > >>>>>
> > >> > >>>>> That's not a PulledPork error, that's a website 
> error. The
> > >> > >>>> file isn't
> > >> > >>>>> there, which strictly speaking shouldn't be a 500 server
> > >> > >> error, but
> > >> > >>>>> since the application that handles looking for the file
> > >> > >>>> can't find it,
> > >> > >>>>> the server will return the application error instead of a
> > >> > >>>> 404 not found.
> > >> > >>>>>
> > >> > >>>>> With that said, I'll forward this to our Snort 
> web team for
> > >> > >>>>> investigation.
> > >> > >>>>
> > >> > >>>> Actually, no I won't. After looking at snort.org 
> I see that
> > >> > >>>> the 2.9.0.4
> > >> > >>>> rule set is not yet available for registered users. So,
> > >> > >> you'll get a
> > >> > >>>> 404 (or 500) for the rules file too.
> > >> > >>>>
> > >> > >>>> You can fix this for future use by using
> > >> > >>>> snortrules-snapshot-edge.tar.gz as the name of your rules
> > >> > >> file. That
> > >> > >>>> way, you will get the latest version of rules for either
> > >> > >>>> registered or
> > >> > >>>> subscriber rules automatically. Right now, for registered
> > >> > >> users this
> > >> > >>>> will be a 2.9.0.3 rule set. Which should work 
> with 2.9.0.4.
> > >> > >>>>
> > >> > >>>> Now, per the rules of the drinking game, I will 
> be taking a
> > >> > >>>> shot or two
> > >> > >>>> for replying to my own email.
> > >> > >>>>
> > >> > >>>> --
> > >> > >>>> Nigel Houghton
> > >> > >>>> Head Mentalist
> > >> > >>>> SF VRT Department of Intelligence Excellence
> > >> > >>>> http://vrt-blog.snort.org/ && http://labs.snort.org/
> 


_____________________________________________________________________________________________

Please visit www.nhrs.org to subscribe to NHRS email announcements and updates.


More information about the Snort-users mailing list