[rt-users] apache 100% cpu usage

Mon Mar 21 03:01:42 EDT 2011

I tried to examine your idea about big search. I did some of them (i.e. 128000 results) but httpd process only peaked (still not 100%) and in a few seconds get back to the normal state.

It seems search with large result could not be the exact source if this problem

Best Regards

From: rt-users-bounces at lists.bestpractical.com [mailto:rt-users-bounces at lists.bestpractical.com] On Behalf Of Payam Poursaied
Sent: Sunday, March 20, 2011 8:55 PM
To: 'Ruslan Zakirov'
Cc: rt-users at lists.bestpractical.com
Subject: Re: [rt-users] apache 100% cpu usage

Hi Ruslan

Thank you for your response.

There isn’t any end! Even after a week!

All modules are updated to the latest available port in FreeBSD

You gave me a good point to check, but still I am curious to know how such problems could be debugged. Do you have any idea on this issue ?

Best Regards

From: Ruslan Zakirov [mailto:ruslan.zakirov at gmail.com] 
Sent: Sunday, March 20, 2011 7:39 PM
To: Payam Poursaied
Cc: rt-users at lists.bestpractical.com
Subject: Re: [rt-users] apache 100% cpu usage

I see storable perl module there, so it's sessions handling. Look at updates for the module. Try to wait to see if it ends. If it ends then may be It's result of a big search that stored in the session. Newer versions of RT don't store big search results in sessions, only limited amount of records. Check sessions table for big records.

Regards, Ruslan. From phone.

20.03.2011 11:19 пользователь "Payam Poursaied" <me at payam124.com> написал:
> 
> Hi all
> We are using rt3.8.8 and we faced with a problem using apache 1.3.42 on FreeBSD 8.0-RELEASE.
> The problem is that httpd processes frequently reach 100% CPU usage and stay
> at this level of CPU usage. 
> This is a sample of top output. 
> 
> PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
> 77051 www 1 118 0 208M 86080K CPU4 4 87.6H 100.00% httpd
> 9403 www 1 118 0 215M 93340K CPU2 2 65.0H 100.00% httpd
> 78430 www 1 118 0 226M 100M CPU0 0 58:57 100.00% httpd
> 95332 www 1 50 0 205M 83624K sbwait 12 0:02 4.49% httpd
> 89664 www 1 46 0 210M 88144K sbwait 11 0:05 2.59% httpd
> 95331 www 1 46 0 205M 83464K sbwait 12 0:01 1.66% httpd

> 
> I think there might be a bug that caused something like endless loop but I
> am newbie to gdb and debugging and I could not find any relevant point to
> get into it.
> I tried to compile apache in debug mode and then wait to see a suspicious
> process and then attached GDB66 to that cpu intensive httpd process to find
> out what is going on. 
> But I could not go further more.
> 
> I have also consulted with apache-freebsd mailing list but there wasn't any success.
> 
> Could anyone help me to drill down this problem?
> 
> Best Regards
> Payam Poursaied
> 
> Using "next" says:
> (gdb) next
> Cannot find bounds of current function
> And here below is output of "where" command in gdb66.
> (gdb) where
> #0 0x000000080093a73b in ?? () from /lib/libc.so.7
> #1 0x000000080093d505 in ?? () from /lib/libc.so.7
> #2 0x00000008009401bc in ?? () from /lib/libc.so.7
> #3 0x0000000800942ec4 in malloc () from /lib/libc.so.7
> #4 0x0000000803079946 in Perl_safesysmalloc () from /usr/local/lib/perl5/5.10.1/mach/CORE/libperl.so
> #5 0x000000080308cee0 in Perl_av_extend () from /usr/local/lib/perl5/5.10.1/mach/CORE/libperl.so
> #6 0x000000080308dabe in Perl_av_store () from /usr/local/lib/perl5/5.10.1/mach/CORE/libperl.so
> #7 0x000000080750de53 in retrieve_byte () from /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #8 0x000000080750c66b in retrieve () from /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #9 0x00000008075113b8 in retrieve_hash () from /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #10 0x000000080750c66b in retrieve () from /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #11 0x000000080750ce29 in retrieve_ref () from /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #12 0x000000080750c66b in retrieve () from /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #13 0x00000008075113b8 in retrieve_hash () from /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #14 0x000000080750c66b in retrieve () from /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #15 0x000000080751298c in retrieve_blessed () from /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #16 0x000000080750c66b in retrieve () from /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #17 0x000000080750ce29 in retrieve_ref () from /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #18 0x000000080750c66b in retrieve () from /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #19 0x00000008075113b8 in retrieve_hash () from /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #20 0x000000080750c66b in retrieve () from /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #21 0x000000080750d55f in do_retrieve () from /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #22 0x000000080750dd28 in XS_Storable_mretrieve () from
> /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #23 0x0000000803090620 in Perl_pp_entersub () from /usr/local/lib/perl5/5.10.1/mach/CORE/libperl.so
> #24 0x000000080308eb9e in Perl_runops_standard () from /usr/local/lib/perl5/5.10.1/mach/CORE/libperl.so
> #25 0x00000008030c1ee3 in S_docatch () from /usr/local/lib/perl5/5.10.1/mach/CORE/libperl.so
> #26 0x000000080308eb9e in Perl_runops_standard () from /usr/local/lib/perl5/5.10.1/mach/CORE/libperl.so

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.bestpractical.com/pipermail/rt-users/attachments/20110321/802dfe69/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 5686 bytes
Desc: not available
URL: <http://lists.bestpractical.com/pipermail/rt-users/attachments/20110321/802dfe69/attachment.bin>