[rt-users] RTx::Shredder time gap between tickets wipeout

[Ruslan Zakirov]

I really have no idea. Could you run shredder CLI with Devel::DProf?

On 3/17/07, Konstantin Naryzhniy <konstantinn at parallels.com> wrote:
> Hi again!
>
> I tried turning postfix and apache off and run shredder again.
> What I found out is really strange.
>
> During this gap (again - it occurs right before each top level object is
> processed) there are no processes in mysql, i.e. the SHOW PROCESSLIST
> command in mysql shows sleeping connection from rt_user.
> However, when I run the top command in shell, there is a perl process that
> eats 50% of CPU!!! And it does that precisely during this gap (20 seconds
> average). When the Shredder warnings start appearing, the mysqld process
> takes over CPU and RAM.
>
> When the top level object is wiped out, before going to the next one,
> Shredder again somehow calls the perl process and so on.
>
> Ruslan, please, tell me why does this happen?
>
> I never mentioned that, but we do have a clone RT with only 500 tickets for
> scrip testing and on that instance the Shredder runs almost instantly.
>
> I would really appreciate your advice on 'perl' gap thing.
>
> Kostya.
>
>
> -----Original Message-----
> From: rt-users-bounces at lists.bestpractical.com
> [mailto:rt-users-bounces at lists.bestpractical.com] On Behalf Of Konstantin
> Naryzhniy
> Sent: Friday, March 16, 2007 7:09 PM
> To: 'Ruslan Zakirov'
> Cc: rt-users at lists.bestpractical.com
> Subject: RE: [rt-users] RTx::Shredder time gap between tickets wipeout
>
> So, well, do you suggest that I reinstall Shredder?
> Not quite sure how to uninstall it once it's up...
>
> If you commit things once - what event do "warning <object-id> wiped out"
> lines refer to in the log? And how many times the dump file is written per
> one top level object?
>
> I also plan to try running Shredder with Postfix and Apache off. I slightly
> hope that that'll speed things up a bit, since there's going to be no
> queries committed simultaneously with Shredder.
>
> I'll report on results of this.
>
> Ruslan, thanks again for your great tool!
>
> Kostya
>
> -----Original Message-----
> From: Ruslan Zakirov [mailto:ruslan.zakirov at gmail.com]
> Sent: Friday, March 16, 2007 6:56 PM
> To: Konstantin Naryzhniy
> Cc: rt-users at lists.bestpractical.com
> Subject: Re: [rt-users] RTx::Shredder time gap between tickets wipeout
>
> On 3/16/07, Konstantin Naryzhniy <konstantinn at parallels.com> wrote:
> > Hello Ruslan!
> > Thanks for your rapid reply.
> >
> > I have added one of the indices you've suggested (we do not have Custom
> > Fields yet and the ACL table is relatively small and doesn't appear in
> slow
> > queries log).
> >
> > I do not think that the delay is caused by the committing transaction.
> > Actually, the gap has to deal with the previous ticket, rather that the
> next
> > one. I mean that this delay first happens right after when I confirm the
> "Do
> > you want to proceed? [y/N]" prompt and before the first warning line
> > appears.
> Huh, that's kinda a ringing bell that's I've screwed up something and
> that's actually problem in the shredder.
>
> >
> > Could you please explain what happens before the first warning is sent to
> > output? I am no programmer to dig into the source code))
> >
> > Are the DELETE statement in the Shredder committed one by one or are they
> > committed in a single "package" after all the records in DB tables were
> > found?
> I commit things once per ticket or per user, depends on what top level
> objects are. Top level objects are objects you select with a plugin.
>
> >
> > thanks,
> > Kostya
>
> _______________________________________________
> http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users
>
> Community help: http://wiki.bestpractical.com
> Commercial support: sales at bestpractical.com
>
>
> Discover RT's hidden secrets with RT Essentials from O'Reilly Media.
> Buy a copy at http://rtbook.bestpractical.com
>
>


-- 
Best regards, Ruslan.