[rt-users] Performance and responsetime in RT 3.0.10

Sun May 2 13:27:14 EDT 2004

Palle Girgensohn wrote:
> --On söndag, maj 02, 2004 16.30.24 +0400 "Ruslan U. Zakirov" 
> <cubic at acronis.ru> wrote:
> 
>> Preface:
>> I run RT at home for tests on Duron600 with 196MB memory on
>> A1/mp1/mysql4.0.x with >3GB DB 40k tickets, 15k users...
>> and home page doesn't take much time to load about 3sec no more.
>>
>> Palle Girgensohn wrote:
>>
>>> Hi,
>>>
>>> We have terrible response times, at least 6-7 seconds, often more,
>>> sometimes enough to make fastcgi give up (max time set to 30 s).
>>
>> I have no experience with FCGI :(
> 
> 
> I run Apache, and since each process snatched > 70 megs when running 
> with mmod_perl, 10 apache processes soon ate 900 megs of RAM, the 
> machine has about 800 MB, so it became very slow. Hence, I switched to 
> fastcgi and just run one process for now. This was a major speedup.
'>70megs' - it's very strange for me, our apache/mp is only 25-30 meg 
and only under some conditions one process can be 200 and more.

> 
> 
>>> Please note that this is *not* mainly a problem with the database, it is
>>> more or less *only* perl that uses the CPU. My guess is you will get the
>>> same results if you check using e.g. top. I just checked logging in to
>>> our RT, and the after hitting the login button with my credentials typed
>>> in, it took approx 25 seconds to load the "Home" page. Hitting "Home"
>>> again loads the page in about 6-7 seconds, since things are cached in
>>> the session, I guess... Anyway, checking the database log, of the 25
>>> seconds spent only 3.5 s where spent running database queries. Top shows
>>> about 15% load on postgresql and 100% load on perl (2 CPUs, remember).
>>> Nothing else loads the machine.
>>
>> `top` isn't good method to analize perfomance and it doesn't show real
>> situation.
> 
> 
> That it quite true, but for procs running for some time, it does give 
> you a good clue, esp on FreeBSD where it does not eat cycles the way 
> I've seen on some other systems.
Did you read report about similar conditions under linux from Dan 
O'Neill? Situation can be same IMHO.

> 
>> Could you send full DB report for one request to home page?
> 
> 
> enclosed is all db queries with milliseconds duration, sorted on 
> duration. Total sum at the bottom. In "Comma Separated Values" format.
> 
>> Did you try MasonX::Profiler, RT_Config:
> 
> 
> Nope, but I could. What kind of parameters would I use?
@MasonParameters=(preamble=>'my $p = MasonX::Profiler->new($m,$r);');

and you should install MasonX::Profiler module. It'll give more info 
about where RT spins more while request.
> 
>> # @MasonParameters is the list of parameters for the constructor of
>> # HTML::Mason's Apache or CGI Handler.  This is normally only useful
>> # for debugging, eg. profiling individual components with
>> #     (preamble => 'my $p = MasonX::Profiler->new($m, $r);');
>> @MasonParameters = () unless (@MasonParameters);
>>
>>
>>>
>>> We don't have a very fancy machine, which might explain the slowness to
>>> a certain degree. Two 450 MHz P3s.
>>
>> I wrote test machine specs which run RT for me at home.
>>
>>>
>>> One thing about the postgresql queries is bad, there's a mail thread
>>> about this a while back, but I believe it is postgresql specific - the
>>> SearchBuilder uses lower() on IDs, so indices are not used. Still only
>>> 10-15 % of the time is spent in the database, so there is something
>>> else, something in the perl code.
>>
>> Does DBIx::SB use 'lower' for you? May be ILIKE, because this fix to SB
>> only in latest SVN and not in 0.99v as I remember.
>> In Pg you can create index on lower(column). iirc Pg schema on RT SVN
>> allready modified.
> 
> 
> Main point is that there is a bug where it wants to use ILIKE for 
> primary key and ID:s, and this is not very clever. Jesse admitted that 
> this was wrong, but I have no idea how to fix it. It bumps queries from 
> 10 ms to 30 ms, it is not really a big deal.
This should be fixed in DBIx::SB, but it's not so easy. DBIx::SB doesn't 
know anything about field types so can't make decision where use 
ILIKE/LOWER and where not to make query case insensetive. Pg doesn't 
optimize ILIKE or LOWER too if field is numeric.

> 
> /Palle
> 
> 
> 
> 
>>> I've had this discussion a few times before on the list, but I'm not
>>> sure there is any way to fix this? Perhaps it is just the design of RT?
>>> Hopefully throwing faster hardware at RT will help, but it is a pity.
>>>
>>> I have FreeBSD 4.9, Postgresql-7.4.2, Perl 5.8.3, RT 3.0.10
>>>
>>> /Palle
>>>
>>> --On fredag, april 30, 2004 14.13.32 +0200 Hilde Therese Lauvset
>>> <Hilde.Lauvset at cc.uit.no> wrote:
>>>
>>>>
>>>>
>>>>
>>>>
>>>>   _____
>>>>
>>>> Fra: Hilde Therese Lauvset
>>>> Sendt: 30. april 2004 14:07
>>>> Til: 'rt-users at bestpractical.com'
>>>> Emne: Performance and responsetime in RT 3.0.10
>>>>
>>>>
>>>>
>>>> Hi,
>>>>
>>>>
>>>>
>>>> We think RT's response time is very low. It take about 6-7 seconds to
>>>> load a new page every time. I was wondering if anybody has the same
>>>> response time; better or worse. I have tried to do some research on how
>>>> to better the performance and has done a few changes, but with no luck.
>>>>
>>>>
>>>>
>>>> Is there an idea to use indexing in the mysql database? Or isn't the
>>>> database a problem at all?
>>>>
>>>>
>>>>
>>>> If anyone has an idea of what we can do, so please give us a hint.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Here are our settings:
>>>>
>>>>
>>>>
>>>> Machine: 2 CPU with 1 GHz and 1 GB RAM
>>>>
>>>>
>>>>
>>>> RT-3.0.10
>>>>
>>>> Apache 2.0
>>>>
>>>> Fastcgi 2.4.2
>>>>
>>>> Mysql 4
>>>>
>>>> --------------------------------------------------------
>>>>
>>>> httpd.conf:
>>>>
>>>>
>>>>
>>>> LoadModule fastcgi_module modules/mod_fastcgi.so
>>>>
>>>> AddHandler fastcgi-script fcgi
>>>>
>>>> FastCgiServer /opt/rt3/bin/mason_handler.fcgi
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> NameVirtualHost xxx.xxx.x.x:80
>>>>
>>>>
>>>>
>>>> <VirtualHost xxx.xxx.x.x:80>
>>>>
>>>>         ScriptAlias / /opt/rt3/bin/mason_handler.fcgi/
>>>>
>>>>         ServerName blabla
>>>>
>>>>         DocumentRoot /opt/rt3/share/html
>>>>
>>>>         AddDefaultCharset UTF-8
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> </VirtualHost>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> # hack to fix graphics with fastcgi
>>>>
>>>> NameVirtualHost xxx.xxx.x.x:81
>>>>
>>>>
>>>>
>>>> <VirtualHost xxx.xx.x.x:81>
>>>>
>>>>         DocumentRoot /opt/rt3/share/html/NoAuth/images
>>>>
>>>> </VirtualHost>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> PerlSetVar MasonCodeCacheMaxSize 20000000
>>>>
>>>> PerlSetVar MasonStaticSource 1
>>>>
>>>>
>>>>
>>>> If you find something strange in my httpd.conf file please tell me :-)
>>>>
>>>>
>>>>
>>>> Hilde Therese
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users
>>>
>>> RT Developer and Administrator training is coming to LA, DC and
>>> Frankfurt this spring and summer.
>>> http://bestpractical.com/services/training.html
>>>
>>> Sign up early, as class space is limited.
> 
> 
> 
>