[rt-users] RT/apache eats 100% CPU on login

[Kevin Murphy]

> Seems like an amazing amount of information to provide to be told a
> debugging process!

Yeah, it can save time, though.  The information could allow someone 
knowledgeable to immediately identify a well-known problem.

> Googling for answers brings up lots of links that indicate that the
> site had a disk crash.

It is annoying that that isn't fixed yet, but the pages in the google 
cache can still be useful.  Downloading and grepping the mailing list 
archives is another possibility.  My P.S. contains a script for 
downloading the archives.

> debugging process!

I'm not an expert, but:

*) As you've probably done, verify that it is httpd and not postgres 
that is hanging by using 'top' or other tool.

*) Even if the problem is in httpd, it could be informative to turn on 
postgres logs by putting this in postgresql.conf:

log_min_messages = notice
log_connections = true
log_pid = true
log_statement = true
log_duration = true
log_timestamp = true

and restarting postgres.

*) There is a discussion of hanging mod_perl apps at: 
http://perl.apache.org/docs/1.0/guide/debug.html.  Several months ago I 
went through some similar recommendations to identify the Perl code 
where the loop was happening, but I don't really recall my experiences. 
  I think I used the gdb method.  I guess you could try modifying 
./bin/webmux.pl if you want to try the "perl trace" method.

Hopefully some more knowledgeable person will hop in if my suggestions 
are off the mark.

Kevin Murphy

P.S.  Since the mailing list archives don't seem to be very searchable 
at the moment, here's a Perl script to download/mirror them locally.  
Use zgrep, or gunzip and grep.

#!/usr/bin/perl
# Script to download the RT mailing list archives
# See the POD documentation below.

use Getopt::Long qw(GetOptions);
use LWP::Simple;
use POSIX qw(strftime);

use warnings;
use strict;

my %options;
GetOptions (\%options,
             'verbose',
             'base-url=s'
) or exit 1;

# main code

my @local_time = localtime;
my $this_year = strftime "%Y", @local_time;
my $this_month = strftime "%B", @local_time;
my @months = qw( January February March April May June July August 
September October November December );

# To get the url of an archive file, append 
'/$list/$year-$month.txt.gz' to the $archive_host
my $archive_host = $options{'base-url'} || 
'http://pallas.eruditorum.org/pipermail';

my @lists = qw(rt-devel rt-users);

foreach my $list (@lists) {

    mkdir $list unless -d $list;

  YEAR: foreach my $year (2001..$this_year) {
    MONTH:   foreach my $month (@months) {
          my $file = "$list/$year-$month.txt.gz";  # name of local file
          my $url = "$archive_host/$file";
          # if the file isn't already here, download it
          my $rc = mirror( $url, $file );
          if (is_success($rc)) {
             print STDERR "Downloaded $url to $file\n" if 
$options{'verbose'};
          } elsif ($rc == RC_NOT_MODIFIED) {
             print STDERR "Did not need to download $url to $file\n" if 
$options{'verbose'};
          } else {
             die "Couldn't download $url, result code = $rc";
          }
          last YEAR if $year == $this_year and $month eq $this_month;
          sleep 1;
       } # end foreach month
    } # end foreach year
}

exit(0);

__END__


=head1 NAME

get_rt_archives - download RT mailing list archives

=head1 SYNOPSIS

get_rt_archives [options]

   Options:
   --verbose           Show progress.
   --base-url=url      Base URL of archives.
                       Default: http://pallas.eruditorum.org/pipermail

Kevin Murphy E<lt>murphy at genome.chop.eduE<gt>.

=head1 DESCRIPTION

B<get_rt_archives> mirrors the RT mailing list archive locally.  Use
the B<--verbose> if you want to see what's happening.  Use the
--base-url switch (or modify the code) if Best Practical changes the
location of the archive.  The script creates subdirectories in the 
current
directory called rt-users and rt-devel.

=head1 AUTHOR

Please report bugs or suggestions to Kevin Murphy 
E<lt>murphy at genome.chop.eduE<gt>.

=cut

On Friday, April 30, 2004, at 11:59  AM, rtuser at wayne47.com wrote:

> On Fri, Apr 30, 2004 at 11:30:21AM -0400, Kevin Murphy wrote:
>> I think the gurus on the list are more likely to be able to help if 
>> you
>> supply your version information for RT, DBIx::FormBuilder, mod_perl or
>> fastcgi, perl, database, and operating system.  A link to or the full
>> text of your httpd.conf file is probably a good idea also.  Doing
>> google searches for 'site:fsck.com keyword1 ...' is often helpful.
>> Good luck.  I have an outstanding RT/postgres issue myself which I am
>> hoping to hear back on while I dabble at it in my spare moments.
>
> Seems like an amazing amount of information to provide to be told a
> debugging process!
>
>    rt2-2.0.15_1
>    DBIx::FormBuilder - not installed
>    mod_perl-1.28
>    fastcgi - not installed
>    perl-5.6.1_15
>    postgresql-7.4.2
>    postgresql-client-7.4.2
>    FreeBSD 4.8-RELEASE-p17
>
> Googling for answers brings up lots of links that indicate that the
> site had a disk crash.
>
>> On Friday, April 30, 2004, at 10:29  AM, rtuser at wayne47.com wrote:
>>
>>> We're doing an upgrade of an existing RT/postgres environment.
>>> Things are up and stagggering at this point but obviously we are
>>> still missing something.  RT will display the login screen and
>>> properly refuses logins if invalid login/password info is given,
>>> which is good.  But, if proper info is supplied, apache eats 100%
>>> of the CPU until it's killed (and never returns an updated page).
>>>
>>> Digging through the various logs does not point out anything obvious,
>>> does anyone have any clues as to where to look?  Or suggestions on
>>> how to get additional information to debug this?
> _______________________________________________