[rt-users] Speeding up CLI RT::Shredder

[Maxwell A. Rathbone]

Hello,

I'm in the same boat as many others I've seen post. We have 35k tickets 
in one of our queues that I'm trying to shred(shame on us for not 
automating this previously). I've found the web version of the Shredder 
to be god-awful slow. We're talking 10min+ just to shred ONE ticket. So 
I discovered the command-line /opt/rt3/sbin/rt-shredder utility. I was 
then able to shred ONE ticket in about 5 minutes. I found some 
optimization keys to add to the tables, which allowed me to them shred 
ONE ticket in about a minute. I then discovered(this really should be in 
the documentation!), that if you specify a timeframe with rt-shredder, 
you can get MUCH faster processing. I was able to get it down to 
21seconds for the shredding of ONE ticket.

I noticed it was spitting out warning messages each time it deletes 
something. I honestly do not care about the output as long as it is 
working as expected, so I hunted through the code and was able to 
disable the on-screen logging altogether. I'm now able to shred ONE 
ticket in about 8-10 seconds.

For those who are interested in about a 50% reduction in processing time 
for the CLI Shredder, edit the file:
/opt/rt3/lib/RT/Shredder/Rercord.pm

Look for this line:
    $RT::Logger->warning( $msg );

Comment it so it looks like this:
#    $RT::Logger->warning( $msg );

a WORLD of difference from the 10 minutes per ticket I originally was 
getting. Now it looks like to shred the 35k might actually take a 
palatable amount of time.

I wanted to share this useful information on the list so it is google 
searchable. I'm SURE others will find this helpful.

BTW, the command I'm using to shred(again, documentation is kinda poor) is:
./rt-shredder --plugin "Tickets=query,((status = 'deleted' OR status = 
'rejected') AND 
LastUpdated='2008-10-03');limit,100;with_linked,FALSE;apply_query_to_linked,FALSE" 
--force

Max