[rt-users] apache 100% cpu usage

Mathieu Longtin mathieu at closetwork.org
Sun Mar 20 07:31:37 EDT 2011 

I found out a while ago that looping emails would cause RT to go crazy like
that.

I disabled $LoopsToRTOwner and the problem disappeared. I didn't have time
to debug the issue further...

--
Mathieu Longtin
1-514-803-8977


On Sun, Mar 20, 2011 at 4:19 AM, Payam Poursaied <me at payam124.com> wrote:

>
> Hi all
> We are using rt3.8.8 and we  faced with a problem using apache 1.3.42 on
> FreeBSD 8.0-RELEASE.
> The problem is that httpd processes frequently reach 100% CPU usage and
> stay
> at this level of CPU usage.
> This is a sample of top output.
>
>  PID USERNAME   THR PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
> 77051 www          1 118    0   208M 86080K CPU4    4  87.6H 100.00% httpd
>  9403 www          1 118    0   215M 93340K CPU2    2  65.0H 100.00% httpd
> 78430 www          1 118    0   226M   100M CPU0    0  58:57 100.00% httpd
> 95332 www          1  50    0   205M 83624K sbwait 12   0:02  4.49% httpd
> 89664 www          1  46    0   210M 88144K sbwait 11   0:05  2.59% httpd
> 95331 www          1  46    0   205M 83464K sbwait 12   0:01  1.66% httpd
> 46157 www          1  47    0   258M   134M sbwait 15   3:03  1.17% httpd
> 89493 www          1  44    0   212M 90152K accept 13   0:20  0.10% httpd
> 22801 www          1  44    0   264M   137M accept 13   5:40  0.00% httpd
> 25440 www          1  44    0   275M   145M accept 11   5:35  0.00% httpd
> 30396 www          1  44    0   262M   135M accept 14   4:47  0.00% httpd
> 82456 www          1  44    0   233M   109M accept 12   1:37  0.00% httpd
> 80407 www          1  44    0   239M   110M accept 10   0:49  0.00% httpd
> 80097 www          1  44    0   265M   135M accept  1   0:35  0.00% httpd
> 88622 www          1  46    0   216M 94588K accept 15   0:16  0.00% httpd
> 88968 www          1  44    0   208M 86316K accept  6   0:03  0.00% httpd
> 89686 www          1  44    0   201M 80216K accept 11   0:01  0.00% httpd
> 95330 www          1  44    0   201M 77088K sbwait 10   0:00  0.00% httpd
>
> I think there might be a bug that caused something like endless loop but I
> am newbie to gdb and debugging and I could not find any relevant point to
> get into it.
> I tried to compile apache in debug mode and then wait to see a suspicious
> process and then attached GDB66 to that cpu intensive httpd process to find
> out what is going on.
> But I could not go further more.
>
> I have also consulted with apache-freebsd mailing list but there wasn't any
> success.
>
> Could anyone help me to drill down this problem?
>
> Best Regards
> Payam Poursaied
>
> Using "next" says:
> (gdb) next
> Cannot find bounds of current function
> And here below is output of "where" command in gdb66.
> (gdb) where
> #0  0x000000080093a73b in ?? () from /lib/libc.so.7
> #1  0x000000080093d505 in ?? () from /lib/libc.so.7
> #2  0x00000008009401bc in ?? () from /lib/libc.so.7
> #3  0x0000000800942ec4 in malloc () from /lib/libc.so.7
> #4  0x0000000803079946 in Perl_safesysmalloc () from
> /usr/local/lib/perl5/5.10.1/mach/CORE/libperl.so
> #5  0x000000080308cee0 in Perl_av_extend () from
> /usr/local/lib/perl5/5.10.1/mach/CORE/libperl.so
> #6  0x000000080308dabe in Perl_av_store () from
> /usr/local/lib/perl5/5.10.1/mach/CORE/libperl.so
> #7  0x000000080750de53 in retrieve_byte () from
> /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #8  0x000000080750c66b in retrieve () from
> /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #9  0x00000008075113b8 in retrieve_hash () from
> /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #10 0x000000080750c66b in retrieve () from
> /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #11 0x000000080750ce29 in retrieve_ref () from
> /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #12 0x000000080750c66b in retrieve () from
> /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #13 0x00000008075113b8 in retrieve_hash () from
> /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #14 0x000000080750c66b in retrieve () from
> /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #15 0x000000080751298c in retrieve_blessed () from
> /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #16 0x000000080750c66b in retrieve () from
> /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #17 0x000000080750ce29 in retrieve_ref () from
> /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #18 0x000000080750c66b in retrieve () from
> /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #19 0x00000008075113b8 in retrieve_hash () from
> /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #20 0x000000080750c66b in retrieve () from
> /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #21 0x000000080750d55f in do_retrieve () from
> /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #22 0x000000080750dd28 in XS_Storable_mretrieve () from
> /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/Storable/Storable.so
> #23 0x0000000803090620 in Perl_pp_entersub () from
> /usr/local/lib/perl5/5.10.1/mach/CORE/libperl.so
> #24 0x000000080308eb9e in Perl_runops_standard () from
> /usr/local/lib/perl5/5.10.1/mach/CORE/libperl.so
> #25 0x00000008030c1ee3 in S_docatch () from
> /usr/local/lib/perl5/5.10.1/mach/CORE/libperl.so
> #26 0x000000080308eb9e in Perl_runops_standard () from
> /usr/local/lib/perl5/5.10.1/mach/CORE/libperl.so
> #27 0x0000000803035e57 in Perl_call_sv () from
> /usr/local/lib/perl5/5.10.1/mach/CORE/libperl.so
> #28 0x00000008030d9b46 in Perl_pp_tie () from
> /usr/local/lib/perl5/5.10.1/mach/CORE/libperl.so
> #29 0x000000080308eb9e in Perl_runops_standard () from
> /usr/local/lib/perl5/5.10.1/mach/CORE/libperl.so
> #30 0x00000008030c1ee3 in S_docatch () from
> /usr/local/lib/perl5/5.10.1/mach/CORE/libperl.so
> #31 0x00000008030c343c in Perl_pp_entereval () from
> /usr/local/lib/perl5/5.10.1/mach/CORE/libperl.so
> #32 0x000000080308eb9e in Perl_runops_standard () from
> /usr/local/lib/perl5/5.10.1/mach/CORE/libperl.so
> #33 0x0000000803035e57 in Perl_call_sv () from
> /usr/local/lib/perl5/5.10.1/mach/CORE/libperl.so
> #34 0x00000008030d9b46 in Perl_pp_tie () from
> /usr/local/lib/perl5/5.10.1/mach/CORE/libperl.so
> #35 0x000000080308eb9e in Perl_runops_standard () from
> /usr/local/lib/perl5/5.10.1/mach/CORE/libperl.so
> #36 0x0000000803036191 in Perl_call_sv () from
> /usr/local/lib/perl5/5.10.1/mach/CORE/libperl.so
> #37 0x0000000802eabe36 in perl_call_handler () from
> /usr/local/libexec/apache/libperl.so
> #38 0x0000000802eac87a in perl_run_stacked_handlers () from
> /usr/local/libexec/apache/libperl.so
> #39 0x0000000802eae2b0 in perl_handler () from
> /usr/local/libexec/apache/libperl.so
> #40 0x000000000040ee42 in ap_invoke_handler (r=0x8045c9060) at
> http_config.c:476
> #41 0x0000000000427849 in process_request_internal (r=0x8045c9060) at
> http_request.c:1299
> #42 0x00000000004278a4 in ap_process_request (r=0x8045c9060) at
> http_request.c:1315
> #43 0x000000000041d423 in child_main (child_num_arg=17) at http_main.c:4885
> #44 0x000000000041d734 in make_child (s=0x800c05060, slot=17,
> now=1299387327) at http_main.c:5055
> #45 0x000000000041db6b in perform_idle_server_maintenance () at
> http_main.c:5256
> #46 0x000000000041e159 in standalone_main (argc=1, argv=0x7fffffffed70) at
> http_main.c:5520
> #47 0x000000000041e7a4 in main (argc=1, argv=0x7fffffffed70) at
> http_main.c:5773
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.bestpractical.com/pipermail/rt-users/attachments/20110320/858a5137/attachment.htm>

More information about the rt-users mailing list