[rt-users] Severe performance problems suddenly in RT 3.8.2

[Tim Cutts]

On 18 Feb 2009, at 12:48 am, Ruslan Zakirov wrote:

> First time I see something like that. May be it's better to ask on
> mysql lists. And sure you should keep digging into a reason. My
> practice shows that there is no such thing as "suddenly" in IT. There
> is always some change that turn world up side down.
>
> optimize/analyze on InnoDB don't do the same job as on myisam tables,
> it's documented in mysql's docs. You can find way to rebuild indexes
> on a table and as the last resort use full re-load.

We've had a flash of realisation, and we think we know what the  
problem is.  This virtual machine has a snapshot on it.  This means  
that all writes subsequent to the snapshot being taken have been going  
to a replay log, rather than to the .vmdk file that the virtual  
machine lives in.  Presumably this is a much more complex scenario for  
I/O to operate in than when snapshots are not present.  I hadn't  
realised this was how VMware snapshots work.

So, I'm deleting the snapshots now, but this is a very time consuming  
process because it's having to replay a week's worth of RT writes back  
into the .vmdk file.  We'll let you know whether this solves the  
problem.

Tim


-- 
 The Wellcome Trust Sanger Institute is operated by Genome Research 
 Limited, a charity registered in England with number 1021457 and a 
 company registered in England with number 2742969, whose registered 
 office is 215 Euston Road, London, NW1 2BE.