[rt-users] AssetTracker crashes loading asset...raw horsepower solution?

[John]

On Fri, 7 Nov 2008, Curtis Bruneau wrote:

I think i may have found a bit of a solution to the AT problem.  we're 
running on 2 quad-core servers as frontends with 8gb memory each.  the 
fault im seeing in AT is due to a timeout in apache waiting for its 
backend processes to respond from huge queries...so:

   <IfModule mod_fcgid.c>
     AddHandler fcgid-script .fcgi

OutputBufferSize 1280000
IdleTimeout 600
   ProcessLifeTime 3600
   MaxProcessCount 8
   DefaultMinClassProcessCount 3
   DefaultMaxClassProcessCount 3

   </IfModule>
IPCConnectTimeout 20
IPCCommTimeout 600

has resolved the AT issue from what im seeing.  queries still take a long 
time, but thats just because RT is pulling lots of other data from mysql 
that im not certain is even pertanent to the asset/ticket at hand.  could 
the speed of the query be increased by changing maxprocesscount to a 
higher value?

im running into issues now with the population of a user rights page that 
includes 300-400 users...namely:

RT::Group::Privileged Unimplemented in HTML::Mason::Commands. 
(/opt/rt3/share/html/Elements/ShowUserConcise line 52)

any ideas?



> Date: Fri, 07 Nov 2008 12:52:04 -0500
> From: Curtis Bruneau <curtisb at vianet.ca>
> To: John <nimbius at sdf.lonestar.org>, rt-users at lists.bestpractical.com
> Subject: Re: [rt-users] AssetTracker crashes loading asset
> 
> The queries would execute fine, the problem at least in my case is it tries 
> to load the whole result set into memory causing oom-killer to go on rampage. 
> Apache will peak out, to sort of help this situation I added a mem limit to 
> the fastcgi handler which will basically die before things get really ugly. 
> I'm able to see this query in the log, it has no limit clause so it really is 
> getting everything. While doing a form of debug/trace on the apache/handler 
> I'm also able to see it trying to load the full row times search results 
> (table.*) into memory. This is all on a ticket display from a large search 
> result. So unless you are loading it in memory as opposed to say a shell 
> client output you won't run into memory problems.
>
> Best of luck with your situation, I have not been able to get any 
> confirmation until now with you. I've produced somewhat detailed bug report 
> but it appears to be somewhat isolated.. people with low tickets will never 
> run into this problem.
>
> Curtis
>
> John wrote:
>> curtis:
>> ive played around re-executing the mysql queries related to the RT crash, 
>> and to no avail.  they execute just fine.
>> 
>> could this perhaps be a syslog issue like in RT?  is there a seperate 
>> control for syslogging in AT?
>> 
>> 
>> On Fri, 7 Nov 2008, Curtis Bruneau wrote:
>> 
>>> Date: Fri, 07 Nov 2008 10:43:48 -0500
>>> From: Curtis Bruneau <curtisb at vianet.ca>
>>> To: John <nimbius at sdf.lonestar.org>, rt-users at lists.bestpractical.com
>>> Subject: Re: [rt-users] AssetTracker crashes loading asset
>>> 
>>> Sounds exactly like the issue I have, I think something is trying to get 
>>> all those records, I tried to trace it but with no luck, I think it may be 
>>> related to the back/next links on the ticket display page. It's checking 
>>> each record for something, This is ok with small results but crashes with 
>>> large sets. I really wish I could figure this one out, I get the 
>>> occasional 500 and users are made aware to keep search results smaller. In 
>>> my testing this affected 3.4.x 3.6.x and 3.8.x
>>> 
>>> John wrote:
>>>> well, just as i thought the RT slowness issue had been resolved, 
>>>> assettracker
>>>> is now dying when loading a ticket out of a large list.
>>>> 
>>>> example:  clicking "all assets" enumerates a 9000 row list fine,
>>>> but clicking on any element in the list will crunch for a while
>>>> then die with a 500 error.
>>>> 
>>>> smaller lists with say 1000-3000 rows are sluggish when loading
>>>> an asset from the list, but succeed.
>>>> 
>>>> nimbius at sdf.lonestar.org
>>>> SDF Public Access UNIX System - http://sdf.lonestar.org
>>>> _______________________________________________
>>>> http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users
>>>> 
>>>> Community help: http://wiki.bestpractical.com
>>>> Commercial support: sales at bestpractical.com
>>>> 
>>>> 
>>>> Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy 
>>>> a copy at http://rtbook.bestpractical.com
>>>> 
>>>> 
>>>> 
>>> 
>>> 
>> 
>> nimbius at sdf.lonestar.org
>> SDF Public Access UNIX System - http://sdf.lonestar.org
>

nimbius at sdf.lonestar.org
SDF Public Access UNIX System - http://sdf.lonestar.org