[rt-users] Re: Performance issues with RT3 (all versions)

[Nate Duehr]

Hi Jon,

On Tuesday 03 August 2004 18:56, Jon Masters wrote:
> [ apologies for breaking threading for some folks here. Please CC me
> with replies as I am subscribed but not currently filing your list. ]

Done.

Hmm... I just started out with some "basics"  - noted that you understand how 
to do Unix admin -- some don't, and I've spent a lot of time in the last 
three years helping newbies even *load* Linux, so have built up some bad 
habits of dumbing everything down to the lowest common denominator.  Sorry 
about that!

> | and InnoDB turned on in MySQL (it's off by default if you upgraded to
> | Testing and didn't turn it on).
>
> Oh but I did. In fact I tried a number of different MySQL configurations
> and even tinkered with the indexing that RT had chosen. I am not now
> thinking that MySQL is at fault here but will run the slow query logging
> to provide some figures.

Yeah, I saw some of that thread - haven't run into MySQL being the culprit of 
slowdowns on my system that I can tell, anyway.  

I'll paste the appropriate parts of the my.cnf here, but I warn you (and 
anyone reading) these settings weren't gained via any scientific or 
knowledgeable method -- It was 2AM and I was tinkering and barely reading 
documentation and I'm not a MySQL guru or anything close to it.  (There are 
folks on this list who are MUCH better MySQL tweakers than I am, I'm sure.)  

These numbers were STRICTLY a result of a "hey, let's set these values 
insanely high and let the OS sort out the dang RAM issues!" in a 
mess-around-with-the-server session one night, very late -- and they seemed 
to work, so I left them in place.  

Heck, I might have some of these values so messed up they're causing me harm, 
I don't know... this RT setup here isn't a business or "production" system 
other than a group of ten or so volunteer Linux folks would be discouraged if 
it broke down.

skip-locking
skip-networking
key_buffer              = 256M
max_allowed_packet      = 10M
thread_stack            = 1M
query_cache_limit       = 128M
query_cache_size        = 512M
query_cache_type        = 1

Let's see... Debian "testing" branch... Kernel: 2.4.26-1-k7 (it does help to 
use the pre-compiled Debian kernel or your own compiled kernel that's built 
for your particular processor... how much I can't say, but it helped quite a 
bit on this Athlon for various other things it does besides RT).

Athlon 2500, 512MB RAM (PC2700), Dual 7200 RPM IDE disks on opposite IDE 
controllers (hdparm -t says the buffered read speed from the md device 
wanders from 8 MB/s to 12 MB/s... haven't ever bothered with any bonnie++ 
tests.)

> The processor in the production box is something over 1GHz with 1GB RAM:

Note: Athlon 2500 is a native clock speed of 1.8 GHz, if I remember correctly.

> I get people moaning on a daily basis that RT is various four letter
> words that I will refrain from posting here. It is gets annoying.

Just a side-thought -- have you investigated network overload issues or 
bottlenecks in the network getting to/from the RT machine during the 
timeframe people are complaining?  How many users do you typically have 
hitting it simultaneously?  I *rarely* see more than 2 users at a time on 
this system but it is doing a number of other things, probably the biggest 
one is it also runs spamc for a medium-load mail server.

> Loading times over 5-10 seconds are out of whack in any case.

Loading times of what -- the queue list, a ticket, a bunch of tickets in a 
search?  It will probably make a difference.  Also how often are "idling" 
clients reloading the main page?

> There is only one process running most of the time and that is whatever
> is running the perl - be it apache modperl or whatever.

I usually only see apache itself jump to the top of a quickly-updating (1 
second refresh) top screen, rarely do I see MySQL pop up there long enough to 
matter, and I never see anything named "perl", although of course, I know 
it's running.

Let's see... what else... um, this is RT 3.0.11 from Debian... can't remember 
if you said which version you were on.

I'm sitting here (yet another late night e-mail/hacking session) trying to 
think of other parameters to either share that might help you find your 
performance issue, or ones to ask you about.  Hmmm.  

Just for comparison, I did a search against our tickets that returned 2979 
tickets in a search list and I selected "unlimited" number of tickets 
displayed at one time.  It took 147 seconds for apache to finish handing it 
all out to my laptop over 802.11g wireless with konqueror showing an average 
of 12.0 Kb/s throughput to the browser during the transfer.   Apache and 
MySQL fought over the single CPU with apache always much heavier than mysql 
-- typical numbers were 80% apache, <20% mysql with other sundry apps running 
at the same time as well.  

Like I said, "usable" but definitely not "speedy".  It's really rare that 
anyone would ever need to show all those tickets at the same time in a search 
like that.  

For the more typical "show 50 tickets" the query was over and the data 
displayed in less than 3 seconds.  Closer to 2, but too fast for me to really 
measure easily.

Pulling up one of those tickets with a 43.5 K file attachment was also too 
fast to measure realistically.

Okay, I'm fading fast... time for bed... hopefully something here helps.  
Nothing here was done to the level of professionalism I have to deal with at 
work -- this box is just a hodgepodge of various tools and toys for home 
stuff.

--
Nate Duehr, nate at natetech.com