[rt-users] rt 3.4.5 : apache load sometimes climbs to 100%

[Robert Grasso]

Hello,

I installed RT v3.4.5 on CentOS 4.4 with yum. Opening a ticket takes 1 or 2 seconds. But after
having performed a search, opening the first ticket makes Apache load the cpu to 100% for roughly 30
seconds ! Then the ticket displays normally. Further tickets opened from the same page take 1 or 2
seconds to open. If I go to the "Next page", again the Apache load when opening the first ticket
climbs to 100%, and further tickets open in 1 or 2 seconds.

I don't see any valuable info neither in the RT logs (even with Set($LogToFile      , 'debug');) nor
in the Mysql logs nor in the Apache logs. I tried to use debug tools such as mentioned on

Debug - RT Wiki
http://wiki.bestpractical.com/index.cgi?Debug

but I was not able to get any output from DBI::ProfileDumper::Apache nor from MasonX-Profiler. I
carefully read the doc though ...

I am running

httpd-2.0.52-28.ent.centos4
mod_perl-2.0.1-1.rhel4
mysql-4.1.20-1.RHEL4.1

I tried to install RT v3.6.1, but the required DBIx::SearchBuilder was so new, it looked like I had
to install a brand new Perl by hand ! Thus losing the automatic upgrades from yum ...

Can anybody give me some help ? I am getting sort of desperate !


Here are more details about my config :
---------------------------------------

I am upgrading from RT2; RT2 currently runs on a Dell server, running at 1.1 GHz, with 512 MB ram
and 512 MB swap, two RAID 1 SCSI disk, the hdparm -t test ports roughly 15 MB/sec. With such a
configuration, opening any ticket in RT2 is immediate (it takes much less than one second !).
Without much tuning.

For test purposes, I installed RT3 on a simple PC with a P4 at 1.7 GHz, 1GB ram, 1GB swap. The hard
disk is a SCSI one, the hdparm -t test may report 34 MB/sec

I imported our RT2 database into RT3 without problems.

There are many large attachments in the database. I limited them to 10 MB.

I tuned the kernel with
-----------------------

kernel.shmmax=536870912
kernel.shmmni=4096
kernel.shmall=2097152

Mysql : here is my my.cnf :
-------------------------
[root at cedrat-rt ~]# grep '^[^#]' /etc/my.cnf
[client]
port            = 3306
socket          = /var/lib/mysql/mysql.sock

[mysqld]
port            = 3306
datadir=/var/lib/mysql
socket          = /var/lib/mysql/mysql.sock
skip-locking
old_passwords=1
key_buffer = 384M
max_allowed_packet = 10M
table_cache = 512
sort_buffer_size = 2M
read_buffer_size = 2M
read_rnd_buffer_size = 8M
myisam_sort_buffer_size = 64M
thread_cache_size = 8
query_cache_size= 32M
thread_concurrency = 8
log-bin = /var/lib/mysql/backup_rt3/rt3_bin
server-id       = 1
innodb_mirrored_log_groups = 1
innodb_log_files_in_group = 3
innodb_buffer_pool_size = 384M
innodb_additional_mem_pool_size = 20M
innodb_log_file_size = 200M
innodb_log_buffer_size = 8M
innodb_flush_log_at_trx_commit = 1
innodb_log_archive = 0
innodb_file_io_threads = 4
innodb_lock_wait_timeout = 50

[mysqldump]
quick
max_allowed_packet = 16M

[mysql]
no-auto-rehash

[isamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M

[myisamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M

[mysqlhotcopy]
interactive-timeout

[mysql.server]
user=mysql
basedir=/var/lib

[mysqld_safe]
err-log=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

Apache config:
--------------
in httpd.conf, I enabled

KeepAlive On
LogLevel debug

Here is my rt.conf
------------------
[root at cedrat-rt conf.d]# grep '^[^#]' rt.conf
<VirtualHost *:80>
    ServerName rt3.cedrat.lan
    ServerAlias rt3.cedrat.com
    DocumentRoot /var/rt/html
    AddDefaultCharset UTF-8
    PerlModule Apache::DBI
    PerlRequire /usr/bin/webmux.pl
        PerlModule MasonX::Profiler
        PerlSetVar MasonPreamble "my $p = MasonX::Profiler->new($m, $r);"
    <Location />
        Order Deny,Allow
        Allow from all
        Options None
        ExpiresActive On
        ExpiresByType text/css A3600
        ExpiresByType image/png A3600
        ExpiresByType application/x-javascript A3600
        ExpiresByType image/gif A3600
        SetHandler perl-script
        PerlResponseHandler RT::Mason
        <IfDefine PerlDProf>
          <IfDefine PerlSmallProf>
            PerlFixupHandler Apache::SmallProf
          </IfDefine>
        </IfDefine>
    </Location>
    <Location /NoAuth/images>
      SetHandler default-handler
    </Location>
    # To use this you have to start apache with -DPerlStatus.
    #
    # In RHEL, add the following line to /etc/sysconfig/httpd:
    #
    # OPTIONS=-DPerlStatus
    <IfDefine PerlStatus>
      <Location /perl-status>
        SetHandler perl-script
        PerlResponseHandler Apache2::Status
        PerlSetVar StatusOptionsAll On
        PerlSetVar StatusTerse On
        PerlSetVar StatusTerseSize On
        PerlSetVar StatusTerseSizeMainSummary On
        PerlSetVar StatusLexInfo On
      </Location>
    </IfDefine>
</VirtualHost>

---
Robert GRASSO
System Engineer

CEDRAT
15, Chemin de Malacher - Inovallée - 38246 MEYLAN Cedex - FRANCE
Tel: +33 (0)4 76 90 50 45 Fax: +33 (0)4 76 90 16 09
mailto:Robert.Grasso at cedrat.com
---
Support service       : mailto:support at cedrat.com
Commercial service : mailto:cedrat at cedrat.com
Web site                  : http://www.cedrat.com