[rt-users] RTx::Shredder time gap between tickets wipeout

[Konstantin Naryzhniy]

Hello, guys!

 

First of all, I would like to thank everyone who developed and contributed
to RT.

This system is really great and the more I dig into it the more I love its
flexibility compared to rivals.

I have been reading through these lists for quite some time now, and finally
decided to post my question.

 

So, this one probably goes to Ruslan Zakirov, the author of the Shredder.

To make a long story short: RT 3.6.1, huge installation, 150000 (one hundred
and fifty thousand) tickets, 30% of them deleted spam, 20 queues, 80
privileged users.

Mysql 5.0.18, Postfix, Apache/2.2.0, 2.6.15-1.2054_FC5.

 

I started to use the Shredder to get rid of at least spam tickets. A single
ticket took about 6 minutes to delete. It took most of the time to wipe out
Cached Group Members.

After we upgraded to new hardware (C2D, 2G RAM) I went down to 2.5 minutes
per ticket. After that (still with me?) I played around with mysql_slow.log
and indices, as it has been suggested on this list before (if anybody's
interested I'll post exactly what I've done).

Now, the ticket deletion takes only 10-15 seconds. Great! 

However, when I launch rtx-shredder script without -force option from CLI,
so I can see the messages and prompts, there is a delay (a gap) in process
after wiping out previous ticket and starting a new one. On screen
consecutive lines look something like this:

 

[Wed Mar 14 15:47:19 2007] [warning]: RT::Attachment-158005 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Wed Mar 14 15:47:19 2007] [warning]: RT::Transaction-330341 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Wed Mar 14 15:47:19 2007] [warning]: RT::Transaction-334128 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Wed Mar 14 15:47:19 2007] [warning]: RT::Ticket-31041 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Wed Mar 14 15:47:43 2007] [warning]: RT::CachedGroupMember-438043 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Wed Mar 14 15:47:43 2007] [warning]: RT::Transaction-330378 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Wed Mar 14 15:47:43 2007] [warning]: RT::Group-167572 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

 

So, there is this gap of 20-30 seconds, which ends up in approx 17 hours to
delete 1000 tickets.

I wonder, what causes this gap? a dump writing time? If so, can I run the
Shredder without creating a dump?

 

I'll appreciate any help.

 

Thank you very much!

 

Kostya.

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.bestpractical.com/pipermail/rt-users/attachments/20070314/05e9f9d2/attachment.htm>