[rt-users] [Fwd: RTx::Shredder issue thats just developed]

[Ruslan Zakirov]

On 12/31/05, Adrian Carter <cartera at lei.net.au> wrote:
>  Cheers Ruslan,
>      I've done an "ALTER TABLE `Attachments` TYPE=InnoDB" and that seems to
> have shrunk a few hundred Mb's off of it... I got that via
> http://bugs.mysql.com/bug.php?id=1341 . It seems this has
> turned into a 'feature request' to get "fixed" so it should be resolved in
> the next few months.
As I understand this only recreates indexes, but real shrinking you
get from dump+drop+restore.

>
>      I'm continuing to shred happily with that Dependencies thing fixed. I
> dont know how I missed such a blatant comment in the Documentation. Sorry
> for wasting your time on that.
>
>      You might want to make mention of this "problem" with MySQL in the Doco
> though, incase someone else is being concerned like myself with disk
> utilisation. Saves potential concern in regard to Shredder not doing its job
> properly.
You're right, will add that.

>
>  Thanks for your help mate
>
>  Adrian
>
>
>  Ruslan Zakirov wrote:
>  On 12/31/05, Ruslan Zakirov <ruslan.zakirov at gmail.com> wrote:
>
>
>  On 12/31/05, Adrian Carter <cartera at lei.net.au> wrote:
>
>
>  Yeah sorry Ruslan, I completly missed that bit... I had been checking
> the doco but I'd somehow been scrolling right past that.
>
> I've added the change to RT_SiteConfig, and thats fixed it now it appears.
>
> Out of curiosity is the MySQL bug mentioned re Foreign keys still valid?
> and if so, whats the symptoms? I've been shredding lots of tickets, and
>
>  Symtom: script dies. Mainstream RT doesn't use FKs, but when I was
> writing first version I wrote script that adds all possible FK to DB
> and hit documented bug. So it's note for people who use FKs in thiere
> RT DB.
>
>
>
>  it seems to be going through them (each additional batch references
> higher value tickets, so its 'progressing') but the Attachments table
> still stays pretty massive...
>
> Sorry for wasting your time on a documented problem.. but if you've got
> some more info on that mysql bug that would be good (im just concerned
> its not cleaning up all of the 3.5.5 Attachments table properly, because
> some 4000 tickets shredded of the 100,000 in the system and the
> Attachments table hasn't shrunk one bit...)
>
>  Yeah, it's correct mysql behaviour because it's too expensive to cut
> DB file in the middle while delete operation, I really don't remeber
> if mysql reuse freed space, but I do know that mysql documentation
> describe how to maintain this situation. For example PostgreSQL has
> pg_vacuum script for this purposes, search in mysql docs and google.
>
>  Read this discussion
> http://www.issociate.de/board/index.php?t=msg&goto=338208&rid=0
>
>
>
>
>
>  Thanks
>
> Ruslan Zakirov wrote:
>
>
>
>  On 12/31/05, Adrian Carter <cartera at lei.net.au> wrote:
>
>
>
>
>  Hey all,
>
>  Been happily using RTx::Shredder with the 3.5.5 stuff no problems,
> slowly crunching through old tickets I need to purge (my attachments DB
> is 3.1G and growing!!!).
>
>  I've gotten through about 2000 of the 100,000 odd tickets in the
> system, and well, its just broken. The DB seems ok, as I can browse
> tickets ok, but now whenever I run shredder from the commandline, I get
> the errors below. As I said, its just started occuring after a previous
> shredder instance died with error stat 255.
>
>  What can I do to A: give you some more info (im happy for you to
> even jump onto the machine) or B: fix the problem :)
>
>  Thanks for any assitance Ruslan!
>
>  RT::Ticket-102905 object
>  RT::Ticket-102911 object
>  RT::Ticket-102915 object
> Do you want to proceed? [y/N] y
> [Wed Dec 28 12:42:50 2005] [crit]: Message: 'Dependecies list have reach
> its limit.
> See $RT::DependenciesLimit in RTx::Shredder docs.
> '
>
>
>
>  Didn't you understand what's writen in documentation? Shredder before
> deleting objects find all dependencies which also must be deleted with
> objects, I've limited list of dependencies by 1000. For example you
> can have ticket with more than 1000 transactions (email loop). If you
> see such error you need to check that ticket really has many
> transactions, attachments.... if it's true then you reconfigure limit
> in RT_SiteConfig and delete object, but if ticket has not so many
> dependencies then you file bugreport and we cooperate to debug
> problem.
>
>
>
>
>
>  Trace begun at
> /opt/rt3/local/lib/RTx/Shredder/Dependencies.pm line 87
> RTx::Shredder::Dependencies::_PushDependency('RTx::Shredder::Dependencies=HASH(0x9770b9c)',
> 'Shredder', 'RTx::Shredder=HASH(0x8f5122c)', 'Flags', 1,
> 'BaseObj',
> 'RT::Group=HASH(0x9922d4c)', 'TargetObj',
> 'RT::CachedGroupMember=HASH(0x9b058dc)') called at
> /opt/rt3/local/lib/RTx/Shredder/Dependencies.pm line 53
> RTx::Shredder::Dependencies::_PushDependencies('RTx::Shredder::Dependencies=HASH(0x9770b9c)',
> 'Shredder', 'RTx::Shredder=HASH(0x8f5122c)', 'Flags', 1,
> 'BaseObj',
> 'RT::Group=HASH(0x9922d4c)', 'TargetObjs',
> 'RT::CachedGroupMembers=HASH(0x9958888)') called at
> /opt/rt3/local/lib/RTx/Shredder/Dependencies.pm line 59
> RTx::Shredder::Dependencies::_PushDependencies('RTx::Shredder::Dependencies=HASH(0x9770b9c)',
> 'BaseObj', 'RT::Group=HASH(0x9922d4c)', 'Flags', 1, 'TargetObjs',
> 'ARRAY(0x924af24)', 'Shredder',
> 'RTx::Shredder=HASH(0x8f5122c)') called
> at /opt/rt3/local/lib/RTx/Shredder/Group.pm line 86
> RT::Group::__DependsOn('RT::Group=HASH(0x9922d4c)',
> 'Shredder',
> 'RTx::Shredder=HASH(0x8f5122c)', 'Flags', 1,
> 'Dependencies',
> 'RTx::Shredder::Dependencies=HASH(0x9770b9c)') called at
> /opt/rt3/local/lib/RTx/Shredder/Record.pm line 66
> RT::Record::Dependencies('RT::Group=HASH(0x9922d4c)', 'Shredder',
> 'RTx::Shredder=HASH(0x8f5122c)') called at
> /opt/rt3/local/lib/RTx/Shredder/Record.pm line 207
> RT::Record::_Wipeout('RT::Group=HASH(0x9922d4c)', 'Shredder',
> 'RTx::Shredder=HASH(0x8f5122c)') called at
> /opt/rt3/local/lib/RTx/Shredder/Group.pm line 20
> RT::Group::_Wipeout('RT::Group=HASH(0x9922d4c)',
> 'Shredder',
> 'RTx::Shredder=HASH(0x8f5122c)') called at
> /opt/rt3/local/lib/RTx/Shredder/Record.pm line 197
> RT::Record::Wipeout('RT::Group=HASH(0x9922d4c)', 'Shredder',
> 'RTx::Shredder=HASH(0x8f5122c)') called at
> /opt/rt3/local/lib/RTx/Shredder/Dependencies.pm line 144
> RTx::Shredder::Dependencies::Wipeout('RTx::Shredder::Dependencies=HASH(0x97709b0)',
> 'WithoutFlags', 4112, 'Shredder',
> 'RTx::Shredder=HASH(0x8f5122c)')
> called at /opt/rt3/local/lib/RTx/Shredder/Record.pm line
> 214
> RT::Record::_Wipeout('RT::Ticket=HASH(0x980d0e0)', 'Shredder',
> 'RTx::Shredder=HASH(0x8f5122c)') called at
> /opt/rt3/local/lib/RTx/Shredder/Record.pm line 197
> RT::Record::Wipeout('RT::Ticket=HASH(0x980d0e0)', 'Shredder',
> 'RTx::Shredder=HASH(0x8f5122c)') called at
> /opt/rt3/local/lib/RTx/Shredder.pm line 348
> RTx::Shredder::Wipeout('RTx::Shredder=HASH(0x8f5122c)')
> called at
> ./rtx-shredder line 100
> (/opt/rt3/lib/RT.pm:295)
> \
>
> --
> Adrian Carter
> Technical Manager
> Leading Edge Internet
>
> Web http://www.lei.net.au http://support.lei.net.au
> Direct +61 2 6163 6162 Support 1 300 662 415
> E-mail cartera at lei.net.au
>
>
>
>
> --
> Adrian Carter
> Technical Manager
> Leading Edge Internet
>
> Web http://www.lei.net.au http://support.lei.net.au
> Direct +61 2 6163 6162 Support 1 300 662 415
> E-mail cartera at lei.net.au
>
> _______________________________________________
> http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users
>
> Be sure to check out the RT Wiki at http://wiki.bestpractical.com
>
> Download a free sample chapter of RT Essentials from O'Reilly Media at
> http://rtbook.bestpractical.com
>
> WE'RE COMING TO YOUR TOWN SOON - RT Training in Amsterdam, Boston and
> San Francisco - Find out more at
> http://bestpractical.com/services/training.html
>
>
>
>
>  --
> Best regards, Ruslan.
>
>
>
>  --
> Adrian Carter
> Technical Manager
> Leading Edge Internet
>
> Web http://www.lei.net.au http://support.lei.net.au
> Direct +61 2 6163 6162 Support 1 300 662 415
> E-mail cartera at lei.net.au
>
>
>
>  --
> Best regards, Ruslan.
>
>
>
> --
> Best regards, Ruslan.
>
>
>  --
> Adrian Carter
> Technical Manager
> Leading Edge Internet
>
> Web http://www.lei.net.au http://support.lei.net.au
> Direct +61 2 6163 6162 Support 1 300 662 415
> E-mail cartera at lei.net.au
>
>


--
Best regards, Ruslan.