[rt-users] [Fwd: RTx::Shredder issue thats just developed]

[Ruslan Zakirov]

On 12/31/05, Adrian Carter <cartera at lei.net.au> wrote:
> Yeah sorry Ruslan, I completly missed that bit... I had been checking
> the doco but I'd somehow been scrolling right past that.
>
> I've added the change to RT_SiteConfig, and thats fixed it now it appears.
>
> Out of curiosity is the MySQL bug mentioned re Foreign keys still valid?
> and if so, whats the symptoms? I've been shredding lots of tickets, and
Symtom: script dies. Mainstream RT doesn't use FKs, but when I was
writing first version I wrote script that adds all possible FK to DB
and hit documented bug. So it's note for people who use FKs in thiere
RT DB.

> it seems to be going through them (each additional batch references
> higher value tickets, so its 'progressing') but the Attachments table
> still stays pretty massive...
>
> Sorry for wasting your time on a documented problem.. but if you've got
> some more info on that mysql bug that would be good (im just concerned
> its not cleaning up all of the 3.5.5 Attachments table properly, because
> some 4000 tickets shredded of the 100,000 in the system and the
> Attachments table hasn't shrunk one bit...)
Yeah, it's correct mysql behaviour because it's too expensive to cut
DB file in the middle while delete operation, I really don't remeber
if mysql reuse freed space, but I do know that mysql documentation
describe how to maintain this situation. For example PostgreSQL has
pg_vacuum script for this purposes, search in mysql docs and google.


>
> Thanks
>
> Ruslan Zakirov wrote:
>
> >On 12/31/05, Adrian Carter <cartera at lei.net.au> wrote:
> >
> >
> >>Hey all,
> >>
> >>   Been happily using RTx::Shredder with the 3.5.5 stuff no problems,
> >>slowly crunching through old tickets I need to purge (my attachments DB
> >>is 3.1G and growing!!!).
> >>
> >>   I've gotten through about 2000 of the 100,000 odd tickets in the
> >>system, and well, its just broken. The DB seems ok, as I can browse
> >>tickets ok, but now whenever I run shredder from the commandline, I get
> >>the errors below. As I said, its just started occuring after a previous
> >>shredder instance died with error stat 255.
> >>
> >>   What can I do to A: give you some more info (im happy for you to
> >>even jump onto the machine) or B: fix the problem :)
> >>
> >>   Thanks for any assitance Ruslan!
> >>
> >>       RT::Ticket-102905 object
> >>       RT::Ticket-102911 object
> >>       RT::Ticket-102915 object
> >>Do you want to proceed? [y/N] y
> >>[Wed Dec 28 12:42:50 2005] [crit]: Message: 'Dependecies list have reach
> >>its limit.
> >>See $RT::DependenciesLimit in RTx::Shredder docs.
> >>'
> >>
> >>
> >Didn't you understand what's writen in documentation? Shredder before
> >deleting objects find all dependencies which also must be deleted with
> >objects, I've limited list of dependencies by 1000. For example you
> >can have ticket with more than 1000 transactions (email loop). If you
> >see such error you need to check that ticket really has many
> >transactions, attachments.... if it's true then you reconfigure limit
> >in RT_SiteConfig and delete object, but if ticket has not so many
> >dependencies then you file bugreport and we cooperate to debug
> >problem.
> >
> >
> >
> >>Trace begun at /opt/rt3/local/lib/RTx/Shredder/Dependencies.pm line 87
> >>RTx::Shredder::Dependencies::_PushDependency('RTx::Shredder::Dependencies=HASH(0x9770b9c)',
> >>'Shredder', 'RTx::Shredder=HASH(0x8f5122c)', 'Flags', 1, 'BaseObj',
> >>'RT::Group=HASH(0x9922d4c)', 'TargetObj',
> >>'RT::CachedGroupMember=HASH(0x9b058dc)') called at
> >>/opt/rt3/local/lib/RTx/Shredder/Dependencies.pm line 53
> >>RTx::Shredder::Dependencies::_PushDependencies('RTx::Shredder::Dependencies=HASH(0x9770b9c)',
> >>'Shredder', 'RTx::Shredder=HASH(0x8f5122c)', 'Flags', 1, 'BaseObj',
> >>'RT::Group=HASH(0x9922d4c)', 'TargetObjs',
> >>'RT::CachedGroupMembers=HASH(0x9958888)') called at
> >>/opt/rt3/local/lib/RTx/Shredder/Dependencies.pm line 59
> >>RTx::Shredder::Dependencies::_PushDependencies('RTx::Shredder::Dependencies=HASH(0x9770b9c)',
> >>'BaseObj', 'RT::Group=HASH(0x9922d4c)', 'Flags', 1, 'TargetObjs',
> >>'ARRAY(0x924af24)', 'Shredder', 'RTx::Shredder=HASH(0x8f5122c)') called
> >>at /opt/rt3/local/lib/RTx/Shredder/Group.pm line 86
> >>RT::Group::__DependsOn('RT::Group=HASH(0x9922d4c)', 'Shredder',
> >>'RTx::Shredder=HASH(0x8f5122c)', 'Flags', 1, 'Dependencies',
> >>'RTx::Shredder::Dependencies=HASH(0x9770b9c)') called at
> >>/opt/rt3/local/lib/RTx/Shredder/Record.pm line 66
> >>RT::Record::Dependencies('RT::Group=HASH(0x9922d4c)', 'Shredder',
> >>'RTx::Shredder=HASH(0x8f5122c)') called at
> >>/opt/rt3/local/lib/RTx/Shredder/Record.pm line 207
> >>RT::Record::_Wipeout('RT::Group=HASH(0x9922d4c)', 'Shredder',
> >>'RTx::Shredder=HASH(0x8f5122c)') called at
> >>/opt/rt3/local/lib/RTx/Shredder/Group.pm line 20
> >>RT::Group::_Wipeout('RT::Group=HASH(0x9922d4c)', 'Shredder',
> >>'RTx::Shredder=HASH(0x8f5122c)') called at
> >>/opt/rt3/local/lib/RTx/Shredder/Record.pm line 197
> >>RT::Record::Wipeout('RT::Group=HASH(0x9922d4c)', 'Shredder',
> >>'RTx::Shredder=HASH(0x8f5122c)') called at
> >>/opt/rt3/local/lib/RTx/Shredder/Dependencies.pm line 144
> >>RTx::Shredder::Dependencies::Wipeout('RTx::Shredder::Dependencies=HASH(0x97709b0)',
> >>'WithoutFlags', 4112, 'Shredder', 'RTx::Shredder=HASH(0x8f5122c)')
> >>called at /opt/rt3/local/lib/RTx/Shredder/Record.pm line 214
> >>RT::Record::_Wipeout('RT::Ticket=HASH(0x980d0e0)', 'Shredder',
> >>'RTx::Shredder=HASH(0x8f5122c)') called at
> >>/opt/rt3/local/lib/RTx/Shredder/Record.pm line 197
> >>RT::Record::Wipeout('RT::Ticket=HASH(0x980d0e0)', 'Shredder',
> >>'RTx::Shredder=HASH(0x8f5122c)') called at
> >>/opt/rt3/local/lib/RTx/Shredder.pm line 348
> >>RTx::Shredder::Wipeout('RTx::Shredder=HASH(0x8f5122c)') called at
> >>./rtx-shredder line 100
> >>(/opt/rt3/lib/RT.pm:295)
> >>\
> >>
> >>--
> >>Adrian Carter
> >>Technical Manager
> >>Leading Edge Internet
> >>
> >>Web       http://www.lei.net.au http://support.lei.net.au
> >>Direct    +61 2 6163 6162  Support 1 300 662 415
> >>E-mail    cartera at lei.net.au
> >>
> >>
> >>
> >>
> >>--
> >>Adrian Carter
> >>Technical Manager
> >>Leading Edge Internet
> >>
> >>Web       http://www.lei.net.au http://support.lei.net.au
> >>Direct    +61 2 6163 6162  Support 1 300 662 415
> >>E-mail    cartera at lei.net.au
> >>
> >>_______________________________________________
> >>http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users
> >>
> >>Be sure to check out the RT Wiki at http://wiki.bestpractical.com
> >>
> >>Download a free sample chapter of RT Essentials from O'Reilly Media at http://rtbook.bestpractical.com
> >>
> >>WE'RE COMING TO YOUR TOWN SOON - RT Training in Amsterdam, Boston and
> >>San Francisco - Find out more at http://bestpractical.com/services/training.html
> >>
> >>
> >>
> >
> >
> >--
> >Best regards, Ruslan.
> >
> >
>
> --
> Adrian Carter
> Technical Manager
> Leading Edge Internet
>
> Web       http://www.lei.net.au http://support.lei.net.au
> Direct    +61 2 6163 6162  Support 1 300 662 415
> E-mail    cartera at lei.net.au
>
>


--
Best regards, Ruslan.