[rt-users] CPU Wait Maxed Out
Eriks Goodwin-Pfister
eriks at centritech.net
Tue Jun 1 09:48:24 EDT 2010
But then my question becomes, "why did this happen all of a sudden?" We have not changed user loads, usage levels, etc. This is a dedicated box being used by about 5 active users and 20 intermittent (once a week) users. Is 1 Gb of memory not enought for RT with such a light loading?
And does anyone have any idea what those odd mailq entries are? I have to go in and delete several hundred of them every day or two.
Thanks,
Eriks
----- Original Message -----
From: "Richard Skelton" <Richard.Skelton at infineon.com>
To: "Eriks Goodwin-Pfister" <eriks at centritech.net>
Sent: Tuesday, June 1, 2010 9:07:58 AM
Subject: Re: [rt-users] CPU Wait Maxed Out
Hi Eriks,
Yes as I thought you are short on real memory.
Look at the swap.
Here is a busy 16 core Linux system of mine with load of free memory:-
procs -----------memory---------- ---swap-- -----io---- --system--
----cpu---- r b swpd free buff cache si so bi bo in cs us sy
id wa
9 0 204 13177160 157388 25908864 0 0 0 46 3774 4411
52 1 47 0
7 0 204 12952512 157388 25915572 0 0 0 48 3405 3409
52 1 46 0
10 0 204 12847832 157388 25918668 0 0 0 23 3043 3351
51 2 47 0
9 0 204 12678600 157388 25921764 0 0 0 39 1776 2432
50 5 45 0
9 0 204 12779600 157388 25924860 0 0 0 26 2480 6489
49 6 46 0
6 0 204 13102136 157388 25910412 0 0 0 106 3861 17726
38 7 55 0
5 0 204 13522232 157388 25903188 0 0 0 43 3290 23044
27 7 66 0
5 0 204 13375776 157388 25882548 0 0 0 46 1946 1996
21 5 75 0
5 0 204 13189392 157388 25883580 0 0 0 22 1161 905
20 5 75 0
5 0 204 12992112 157388 25884612 0 0 0 21 1194 1002
21 5 75 0
Eriks Goodwin-Pfister wrote:
> [root at support ~]# vmstat 5
> procs -----------memory---------- ---swap-- -----io---- --system--
> -----cpu------
> r b swpd free buff cache si so bi bo in cs us sy id wa st
> 0 2 983748 48248 3472 14964 556 259 605 448 1 59 20 2 33 45 0
> 1 2 983572 17100 2824 22092 2367 501 2367 2045 1276 866 45 9 0 47 0
> 0 2 983036 18208 2936 22112 1085 73 1101 611 1359 375 27 5 0 68 0
> 0 2 983036 11340 2944 22488 1306 0 1306 265 1199 442 22 2 0 77 0
> 0 3 983748 11296 2596 22656 790 650 793 861 1252 373 39 6 0 55 0
> 0 3 983340 11144 2640 21908 887 234 914 487 1346 445 13 4 0 83 0
> 1 2 983036 11208 2536 22268 659 49 659 249 1142 284 46 2 0 52 0
> 0 3 983616 11428 2236 22188 911 581 970 1010 1392 512 19 4 0 76 0
> 0 2 983836 12452 2236 22548 640 739 640 962 1224 335 19 3 0 78 0
> 0 2 983136 11516 2184 22596 824 428 824 566 1241 357 5 2 0 92 0
> 3 1 983036 11216 1928 21544 802 482 802 508 1218 278 18 2 0 80 0
> 0 4 984124 12052 1944 22168 630 1196 630 1413 1283 270 23 3 0 74 0
> 1 1 983828 14684 2080 21628 506 548 546 842 1347 354 22 4 0 74 0
> 1 2 983228 11896 2096 22312 973 440 973 718 1204 378 25 3 0 72 0
> 0 2 983036 15360 2136 22060 430 294 430 586 1177 252 39 3 0 59 0
> 0 2 983036 13252 2212 22516 628 0 628 190 1271 372 39 3 0 57 0
> 0 1 983036 23836 2296 21768 597 17 597 622 1216 331 53 4 0 43 0
> 0 2 983036 11968 2360 23068 765 0 814 167 1215 376 42 3 0 55 0
>
>
>
> ----- Original Message -----
> From: "Richard Skelton" <Richard.Skelton at infineon.com>
> To: "Eriks Goodwin-Pfister" <eriks at centritech.net>
> Sent: Tuesday, June 1, 2010 9:00:49 AM
> Subject: Re: [rt-users] CPU Wait Maxed Out
>
> Hi Eriks,
> You have run out of memory.
> What does vmstat 5 show?
>
> Eriks Goodwin-Pfister wrote:
>
>> That's the kicker, nothing looks "abnormal" in the top listing or
>> processes to me. Perhaps you see something I don't?
>>
>> I have included:
>> 1) results of "ps -auxww"
>> 2) results of "top -b -i -n 20 >> ./top_procs"
>> 3) odd entries in "mailq"
>>
>> Thanks in advance for any more advice...
>>
>> --Eriks
>>
>> ======================================
>>
>> [root at support ~]# ps -auxww
>> Warning: bad syntax, perhaps a bogus '-'? See
>> /usr/share/doc/procps-3.2.7/FAQ USER PID %CPU %MEM VSZ RSS TTY STAT
>> START TIME COMMAND
>> root 1 0.0 0.0 2072 512 ? Ss May29 0:01 init
>> [3] root 2 0.0 0.0 0 0 ? S< May29 0:00
>> [migration/0] root 3 0.0 0.0 0 0 ? SN May29 0:00
>> [ksoftirqd/0] root 4 0.0 0.0 0 0 ? S< May29 0:00
>> [watchdog/0] root 5 0.0 0.0 0 0 ? S< May29 0:00
>> [events/0] root 6 0.0 0.0 0 0 ? S< May29 0:00 [khelper]
>> root 7 0.0 0.0 0 0 ? S< May29 0:00 [kthread]
>> root 10 0.0 0.0 0 0 ? S< May29 0:16
>> [kblockd/0] root 11 0.0 0.0 0 0 ? S< May29 0:00 [kacpid]
>> root 93 0.0 0.0 0 0 ? S< May29 0:00
>> [cqueue/0] root 96 0.0 0.0 0 0 ? S< May29 0:00 [khubd]
>> root 98 0.0 0.0 0 0 ? S< May29 0:00 [kseriod]
>> root 159 0.0 0.0 0 0 ? S May29 0:00
>> [khungtaskd] root 162 0.1 0.0 0 0 ? S< May29 6:52 [kswapd0]
>> root 163 0.0 0.0 0 0 ? S< May29 0:00 [aio/0]
>> root 315 0.0 0.0 0 0 ? S< May29 0:00
>> [kpsmoused] root 337 0.0 0.0 0 0 ? S< May29 0:00 [ata/0]
>> root 338 0.0 0.0 0 0 ? S< May29 0:00 [ata_aux]
>> root 341 0.0 0.0 0 0 ? S< May29 0:00
>> [scsi_eh_0] root 342 0.0 0.0 0 0 ? S< May29 0:00
>> [scsi_eh_1] root 347 0.0 0.0 0 0 ? S< May29 0:00
>> [kstriped] root 356 0.0 0.0 0 0 ? S< May29 0:00 [ksnapd]
>> root 367 0.0 0.0 0 0 ? D< May29 0:38
>> [kjournald] root 392 0.0 0.0 0 0 ? S< May29 0:00 [kauditd]
>> root 425 0.0 0.0 2280 364 ? S<s May29 0:00
>> /sbin/udevd -d
>> apache 630 4.9 4.7 213088 46192 ? D 07:45 3:16
>> /usr/sbin/httpd root 1139 0.0 0.0 0 0 ? S< May29 0:00
>> [kmpathd/0] root 1140 0.0 0.0 0 0 ? S< May29 0:00
>> [kmpath_handlerd] root 1162 0.0 0.0 0 0 ? S< May29 0:00
>> [kjournald] root 1682 0.0 0.0 12520 596 ? S<sl May29 0:00 auditd
>> root 1684 0.0 0.0 13100 600 ? S<sl May29 0:00
>> /sbin/audispd root 1714 0.0 0.0 1728 492 ? Ss May29 0:04
>> syslogd -m 0
>> root 1717 0.0 0.0 1680 312 ? Ss May29 0:00 klogd -x
>> root 1753 0.0 0.0 0 0 ? S< May29 0:29
>> [kondemand/0] rpc 1788 0.0 0.0 1816 376 ? Ss May29 0:00 portmap
>> root 1818 0.0 0.0 0 0 ? S< May29 0:00
>> [rpciod/0] rpcuser 1824 0.0 0.0 1868 480 ? Ss May29 0:00 rpc.statd
>> root 1856 0.0 0.0 5820 228 ? Ss May29 0:00
>> rpc.idmapd dbus 1879 0.0 0.0 2748 764 ? Ss May29 0:00
>> dbus-daemon --system
>> root 1892 0.0 0.0 2172 396 ? Ss May29 0:00
>> /usr/sbin/hcid root 1898 0.0 0.0 1748 372 ? Ss May29 0:00
>> /usr/sbin/sdpd apache 1901 3.4 6.7 204920 65568 ? S 08:16 1:12
>> /usr/sbin/httpd apache 1902 1.1 3.0 137472 29992 ? S 08:16 0:23
>> /usr/sbin/httpd apache 1903 2.1 2.2 208788 22224 ? S 08:16 0:45
>> /usr/sbin/httpd root 1919 0.0 0.0 0 0 ? S< May29 0:00
>> [krfcommd] root 1941 0.0 0.0 12868 548 ? Ssl May29 0:00 pcscd
>> root 1955 0.0 0.0 1676 376 ? Ss May29 0:00
>> /usr/sbin/acpid 68 1972 0.0 0.1 5628 992 ? Ss May29 0:00 hald
>> root 1973 0.0 0.0 3164 496 ? S May29 0:00
>> hald-runner 68 1980 0.0 0.0 2020 472 ? S May29 0:00
>> hald-addon-acpi: listening on acpid socket /var/run/acpid.socket
>> root 1988 0.0 0.0 1976 356 ? S May29 0:04
>> hald-addon-storage: polling /dev/hdc
>> root 2008 0.0 0.0 1916 348 ? Ss May29 0:00
>> /usr/bin/hidd --server
>> root 2032 0.0 0.0 27260 720 ? Ssl May29 0:00 automount
>> root 2056 0.0 0.0 7080 672 ? Ss May29 0:00
>> /usr/sbin/sshd root 2069 0.0 0.0 10128 676 ? Ss May29 0:00 cupsd
>> root 2109 0.0 0.0 4536 504 ? S May29 0:00
>> /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql
>> --socket=/var/lib/mysql/mysql.sock --log-error=/var/log/mysqld.log
>> --pid-file=/var/run/mysqld/mysqld.pid --user=mysql
>> mysql 2159 0.0 0.3 149024 3716 ? Sl May29 0:18
>> /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql
>> --user=mysql --pid-file=/var/run/mysqld/mysqld.pid
>> --skip-external-locking --socket=/var/lib/mysql/mysql.sock
>> root 2196 0.0 0.0 9232 948 ? Ss May29 0:07
>> sendmail: accepting connections
>> smmsp 2204 0.0 0.0 8148 688 ? Ss May29 0:00
>> sendmail: Queue runner at 01:00:00 for /var/spool/clientmqueue
>> root 2218 0.0 0.0 1908 244 ? Ss May29 0:00 gpm
>> -m /dev/input/mice -t exps2
>> root 2286 0.0 0.0 5292 552 ? Ss May29 0:00 crond
>> xfs 2309 0.0 0.0 3168 524 ? Ss May29 0:00 xfs
>> -droppriv -daemon
>> root 2334 0.0 0.0 2268 336 ? Ss May29 0:00
>> /usr/sbin/atd avahi 2360 0.0 0.0 2600 560 ? Ss May29 0:00
>> avahi-daemon: running [support.local]
>> avahi 2361 0.0 0.0 2600 116 ? Ss May29 0:00
>> avahi-daemon: chroot helper
>> root 2861 0.0 0.0 3516 348 ? S May29 0:00
>> /usr/sbin/smartd -q never
>> root 2868 0.0 0.0 1664 316 tty1 Ss+ May29 0:00
>> /sbin/mingetty tty1
>> root 2869 0.0 0.0 1664 316 tty2 Ss+ May29 0:00
>> /sbin/mingetty tty2
>> root 2872 0.0 0.0 1664 316 tty3 Ss+ May29 0:00
>> /sbin/mingetty tty3
>> root 2875 0.0 0.0 1664 316 tty4 Ss+ May29 0:00
>> /sbin/mingetty tty4
>> root 2877 0.0 0.0 1664 316 tty5 Ss+ May29 0:00
>> /sbin/mingetty tty5
>> root 2878 0.0 0.0 1664 316 tty6 Ss+ May29 0:00
>> /sbin/mingetty tty6
>> root 2926 0.0 0.2 26108 2768 ? SN May29 0:02
>> /usr/bin/python -tt /usr/sbin/yum-updatesd
>> root 2928 0.0 0.0 2564 628 ? SN May29 0:00
>> /usr/libexec/gam_server root 3012 0.0 0.2 9512 2636 ? S 08:45 0:00
>> sendmail: ./o51CGaSM001908 from queue
>> root 3023 0.0 0.2 9932 2844 ? Ss 08:45 0:00 sshd:
>> root at pts/0 root 3026 0.0 0.1 4660 1436 pts/0 Ss 08:45 0:00 -bash
>> root 3093 0.0 0.2 9432 2196 ? S 08:48 0:00
>> sendmail: ./o51CmDqe003090 from queue
>> mail 3094 0.1 0.9 10808 8880 ? Ss 08:48 0:00
>> /usr/bin/perl -w /etc/smrsh/rt-mailgate --queue umx: client helpdesk
>> --action correspond --url http://support.unimatrix.net
>> mail 3095 0.2 0.9 10808 8844 ? Ss 08:48 0:00
>> /usr/bin/perl -w /etc/smrsh/rt-mailgate --queue umx: client helpdesk
>> --action correspond --url http://support.unimatrix.net
>> root 3097 0.0 0.2 9440 2212 ? S 08:48 0:00
>> sendmail: ./o51CmICW003096 from queue
>> mail 3098 0.2 0.9 10808 8844 ? Ss 08:48 0:00
>> /usr/bin/perl -w /etc/smrsh/rt-mailgate --queue umx: client helpdesk
>> --action correspond --url http://support.unimatrix.net
>> root 3105 0.0 0.0 0 0 ? S 08:49 0:00 [pdflush]
>> root 3107 0.0 0.2 9432 2136 ? S 08:49 0:00
>> sendmail: ./o51CnAWd003106 from queue
>> mail 3108 0.3 0.9 10808 8844 ? Ss 08:49 0:00
>> /usr/bin/perl -w /etc/smrsh/rt-mailgate --queue umx: client helpdesk
>> --action correspond --url http://support.unimatrix.net
>> root 3111 0.0 0.0 0 0 ? S 08:50 0:00 [pdflush]
>> root 3113 0.2 0.4 9568 4332 ? DN 08:50 0:00
>> /usr/bin/python -tt /usr/libexec/yum-updatesd-helper --check --dbus
>> root 3114 0.0 0.0 4260 944 pts/0 R+ 08:51 0:00 ps -auxww
>> root 22634 0.0 0.0 4868 484 ? S 02:12 0:00
>> /usr/sbin/nss_pcache off /etc/httpd/alias
>> root 22636 0.0 1.0 118420 9916 ? Ssl 02:12 0:04
>> /usr/sbin/httpd apache 22641 4.7 7.3 211988 71428 ? D 02:12 19:01
>> /usr/sbin/httpd apache 22642 4.7 10.4 227944 101056 ? D 02:12 19:08
>> /usr/sbin/httpd apache 22643 4.9 7.0 212152 68752 ? D 02:12 19:38
>> /usr/sbin/httpd apache 22644 4.5 7.8 207620 75932 ? S 02:12 18:10
>> /usr/sbin/httpd apache 22645 4.3 7.7 218668 74904 ? D 02:12 17:15
>> /usr/sbin/httpd apache 22646 4.7 3.7 214824 36704 ? S 02:12 19:03
>> /usr/sbin/httpd apache 22647 4.9 7.9 210512 76936 ? S 02:12 19:43
>> /usr/sbin/httpd apache 22648 4.3 10.0 215600 97432 ? S 02:12 17:29
>> /usr/sbin/httpd apache 26071 7.8 11.4 216356 111292 ? S 04:45 19:17
>> /usr/sbin/httpd
>>
>> ======================================
>>
>> top - 08:48:04 up 2 days, 21:03, 1 user, load average: 6.04, 3.78,
>> 2.64
>> Tasks: 100 total, 2 running, 98 sleeping, 0 stopped, 0 zombie
>> Cpu(s): 19.5%us, 1.1%sy, 0.2%ni, 33.1%id, 45.0%wa, 0.3%hi,
>> 0.7%si, 0.0%st
>> Mem: 970084k total, 960168k used, 9916k free, 2844k buffers
>> Swap: 1966072k total, 983072k used, 983000k free, 30420k cached
>>
>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>> 1901 apache 17 0 201m 49m 3076 D 1.2 5.2 1:09.12 httpd
>> 22642 apache 16 0 222m 59m 3116 D 1.2 6.3 19:04.86 httpd
>> 22644 apache 18 0 204m 74m 3116 D 1.2 7.9 18:08.90 httpd
>> 162 root 10 -5 0 0 0 D 0.0 0.0 6:51.52 kswapd0
>> 367 root 10 -5 0 0 0 D 0.0 0.0 0:38.12 kjournald
>> 1988 root 15 0 1976 356 340 R 0.0 0.0 0:04.13
>> hald-addon-stor
>> 3010 root 15 0 0 0 0 D 0.0 0.0 0:00.00 pdflush
>> 3078 root 15 0 0 0 0 D 0.0 0.0 0:00.00 pdflush
>> 3080 root 15 0 0 0 0 D 0.0 0.0 0:00.00 pdflush
>> 3081 apache 17 0 8176 3060 2092 D 0.0 0.3 0:00.05 sendmail
>> 3082 root 15 0 2200 896 696 R 0.0 0.1 0:00.00 top
>>
>>
>> top - 08:48:07 up 2 days, 21:04, 1 user, load average: 6.28, 3.86,
>> 2.67
>> Tasks: 101 total, 2 running, 99 sleeping, 0 stopped, 0 zombie
>> Cpu(s): 29.9%us, 1.9%sy, 0.0%ni, 0.0%id, 66.4%wa, 0.3%hi,
>> 1.6%si, 0.0%st
>> Mem: 970084k total, 960012k used, 10072k free, 2848k buffers
>> Swap: 1966072k total, 983092k used, 982980k free, 30928k cached
>>
>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>> 22644 apache 18 0 204m 74m 3116 R 30.6 7.9 18:09.99 httpd
>> 1901 apache 17 0 201m 50m 3076 D 2.2 5.3 1:09.20 httpd
>> 3084 root 16 0 9276 2300 1564 D 1.4 0.2 0:00.05 sendmail
>> 22642 apache 16 0 222m 61m 3116 D 0.6 6.5 19:04.88 httpd
>> 162 root 10 -5 0 0 0 D 0.3 0.0 6:51.53 kswapd0
>> 3082 root 15 0 2208 992 784 R 0.3 0.1 0:00.01 top
>> 367 root 10 -5 0 0 0 D 0.0 0.0 0:38.12 kjournald
>> 3010 root 15 0 0 0 0 D 0.0 0.0 0:00.00 pdflush
>> 3078 root 15 0 0 0 0 D 0.0 0.0 0:00.00 pdflush
>> 3080 root 15 0 0 0 0 D 0.0 0.0 0:00.00 pdflush
>> top - 08:48:10 up 2 days, 21:04, 1 user, load average: 6.66, 3.98,
>> 2.71
>> Tasks: 102 total, 2 running, 100 sleeping, 0 stopped, 0 zombie
>> Cpu(s): 23.7%us, 2.3%sy, 0.0%ni, 0.0%id, 72.3%wa, 0.3%hi,
>> 1.3%si, 0.0%st
>> Mem: 970084k total, 959132k used, 10952k free, 2872k buffers
>> Swap: 1966072k total, 983076k used, 982996k free, 31244k cached
>>
>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>> 22644 apache 18 0 204m 75m 3116 D 20.6 8.0 18:10.61 httpd
>> 1901 apache 17 0 201m 52m 3076 D 2.7 5.5 1:09.28 httpd
>> 22642 apache 16 0 222m 63m 3116 D 0.7 6.7 19:04.90 httpd
>> 367 root 10 -5 0 0 0 D 0.0 0.0 0:38.12 kjournald
>> 3010 root 15 0 0 0 0 R 0.0 0.0 0:00.00 pdflush
>> 3078 root 15 0 0 0 0 D 0.0 0.0 0:00.00 pdflush
>> 3080 root 15 0 0 0 0 D 0.0 0.0 0:00.00 pdflush
>> 3082 root 15 0 2348 996 784 R 0.0 0.1 0:00.01 top
>> 3084 root 16 0 9276 2364 1616 D 0.0 0.2 0:00.05 sendmail
>>
>>
>> top - 08:48:13 up 2 days, 21:04, 1 user, load average: 6.66, 3.98,
>> 2.71
>> Tasks: 101 total, 1 running, 100 sleeping, 0 stopped, 0 zombie
>> Cpu(s): 6.6%us, 2.3%sy, 0.0%ni, 0.0%id, 88.7%wa, 1.0%hi,
>> 1.3%si, 0.0%st
>> Mem: 970084k total, 955480k used, 14604k free, 2920k buffers
>> Swap: 1966072k total, 983036k used, 983036k free, 31000k cached
>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>> 1901 apache 17 0 201m 53m 3076 D 2.7 5.7 1:09.36 httpd
>> 22642 apache 16 0 222m 65m 3116 D 1.0 6.9 19:04.93 httpd
>> 3085 apache 17 0 8176 3080 2116 D 0.7 0.3 0:00.05 sendmail
>> 3082 root 15 0 2316 996 784 R 0.3 0.1 0:00.02 top
>> 367 root 10 -5 0 0 0 D 0.0 0.0 0:38.12 kjournald
>> 22647 apache 18 0 207m 76m 3124 D 0.0 8.1 19:43.79 httpd
>>
>>
>> top - 08:48:16 up 2 days, 21:04, 1 user, load average: 6.52, 4.00,
>> 2.73
>> Tasks: 99 total, 3 running, 96 sleeping, 0 stopped, 0 zombie
>> Cpu(s): 21.6%us, 5.0%sy, 0.0%ni, 0.0%id, 71.1%wa, 1.0%hi,
>> 1.3%si, 0.0%st
>> Mem: 970084k total, 949180k used, 20904k free, 3044k buffers
>> Swap: 1966072k total, 982932k used, 983140k free, 29604k cached
>>
>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>> 3094 mail 19 0 7352 5544 1704 R 6.3 0.6 0:00.19 rt-mailgate
>> 1901 apache 17 0 201m 56m 3076 D 6.0 5.9 1:09.54 httpd
>> 22642 apache 16 0 222m 70m 3116 D 3.3 7.5 19:05.03 httpd
>> 3082 root 15 0 2316 996 784 R 0.3 0.1 0:00.03 top
>> 3016 mail 16 0 10808 8664 1940 R 0.0 0.9 0:00.30 rt-mailgate
>>
>>
>> top - 08:48:19 up 2 days, 21:04, 1 user, load average: 6.48, 4.03,
>> 2.74
>> Tasks: 99 total, 3 running, 96 sleeping, 0 stopped, 0 zombie
>> Cpu(s): 21.7%us, 3.0%sy, 0.0%ni, 0.0%id, 74.0%wa, 0.3%hi,
>> 1.0%si, 0.0%st
>> Mem: 970084k total, 956980k used, 13104k free, 3084k buffers
>> Swap: 1966072k total, 983140k used, 982932k free, 30380k cached
>>
>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>> 1901 apache 17 0 201m 57m 3076 D 2.7 6.1 1:09.62 httpd
>> 22642 apache 16 0 222m 73m 3116 D 2.3 7.8 19:05.10 httpd
>> 162 root 11 -5 0 0 0 D 0.3 0.0 6:51.59 kswapd0
>> 22641 apache 18 0 208m 10m 3116 D 0.3 1.1 19:00.45 httpd
>> 3082 root 15 0 2316 996 784 R 0.0 0.1 0:00.03 top
>> 3096 root 15 0 9300 2512 1712 R 0.0 0.3 0:00.00 sendmail
>> 22645 apache 16 0 211m 10m 3124 R 0.0 1.1 17:13.98 httpd
>>
>>
>> top - 08:48:22 up 2 days, 21:04, 1 user, load average: 6.48, 4.03,
>> 2.74
>> Tasks: 99 total, 2 running, 97 sleeping, 0 stopped, 0 zombie
>> Cpu(s): 3.0%us, 1.1%sy, 0.0%ni, 0.0%id, 93.8%wa, 0.5%hi,
>> 1.6%si, 0.0%st
>> Mem: 970084k total, 960256k used, 9828k free, 3092k buffers
>> Swap: 1966072k total, 983036k used, 983036k free, 30920k cached
>>
>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>> 3096 root 16 0 9300 2512 1712 D 2.3 0.3 0:00.07 sendmail
>> 22642 apache 16 0 222m 75m 3116 D 1.7 8.0 19:05.15 httpd
>> 1901 apache 17 0 201m 58m 3076 D 1.0 6.2 1:09.65 httpd
>> 162 root 10 -5 0 0 0 D 0.3 0.0 6:51.60 kswapd0
>> 3082 root 15 0 2316 996 784 R 0.3 0.1 0:00.04 top
>> 22641 apache 18 0 208m 11m 3116 D 0.3 1.2 19:00.46 httpd
>> 22645 apache 16 0 211m 10m 3124 D 0.3 1.1 17:13.99 httpd
>> 367 root 10 -5 0 0 0 D 0.0 0.0 0:38.13 kjournald
>> 3086 root 15 0 0 0 0 R 0.0 0.0 0:00.00 pdflush
>>
>>
>> top - 08:48:26 up 2 days, 21:04, 1 user, load average: 6.68, 4.11,
>> 2.78
>> Tasks: 99 total, 2 running, 97 sleeping, 0 stopped, 0 zombie
>> Cpu(s): 4.3%us, 1.3%sy, 0.0%ni, 0.0%id, 93.3%wa, 0.3%hi,
>> 0.7%si, 0.0%st
>> Mem: 970084k total, 960216k used, 9868k free, 3092k buffers
>> Swap: 1966072k total, 983072k used, 983000k free, 30996k cached
>>
>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>> 1901 apache 17 0 201m 59m 3076 D 2.4 6.3 1:09.74 httpd
>> 22642 apache 16 0 222m 77m 3116 D 1.6 8.2 19:05.21 httpd
>> 162 root 10 -5 0 0 0 D 0.3 0.0 6:51.61 kswapd0
>> 3082 root 15 0 2316 996 784 R 0.3 0.1 0:00.05 top
>> 22645 apache 16 0 211m 11m 3124 D 0.3 1.2 17:14.00 httpd
>> 3080 root 15 0 0 0 0 D 0.0 0.0 0:00.00 pdflush
>> 3086 root 15 0 0 0 0 D 0.0 0.0 0:00.00 pdflush
>> 3097 root 18 0 9300 1760 928 D 0.0 0.2 0:00.00 sendmail
>> 22641 apache 18 0 208m 11m 3116 R 0.0 1.2 19:00.46 httpd
>>
>>
>> top - 08:48:29 up 2 days, 21:04, 1 user, load average: 6.68, 4.11,
>> 2.78
>> Tasks: 99 total, 1 running, 98 sleeping, 0 stopped, 0 zombie
>> Cpu(s): 4.0%us, 1.0%sy, 0.0%ni, 0.0%id, 93.7%wa, 0.3%hi,
>> 1.0%si, 0.0%st
>> Mem: 970084k total, 960192k used, 9892k free, 3092k buffers
>> Swap: 1966072k total, 983160k used, 982912k free, 30960k cached
>>
>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>> 1901 apache 17 0 201m 60m 3076 D 2.7 6.4 1:09.82 httpd
>> 22642 apache 16 0 222m 78m 3116 D 1.0 8.3 19:05.24 httpd
>> 162 root 10 -5 0 0 0 D 0.7 0.0 6:51.63 kswapd0
>> 22645 apache 16 0 211m 11m 3124 D 0.3 1.2 17:14.01 httpd
>> 367 root 10 -5 0 0 0 D 0.0 0.0 0:38.13 kjournald
>> 3080 root 15 0 0 0 0 D 0.0 0.0 0:00.00 pdflush
>> 3082 root 15 0 2316 996 784 R 0.0 0.1 0:00.05 top
>> 3086 root 15 0 0 0 0 D 0.0 0.0 0:00.00 pdflush
>> 3097 root 18 0 9300 1764 932 D 0.0 0.2 0:00.00 sendmail
>> 22641 apache 18 0 208m 12m 3116 D 0.0 1.3 19:00.46 httpd
>>
>>
>> top - 08:48:32 up 2 days, 21:04, 1 user, load average: 6.87, 4.19,
>> 2.81
>> Tasks: 100 total, 1 running, 99 sleeping, 0 stopped, 0 zombie
>> Cpu(s): 12.3%us, 2.7%sy, 0.0%ni, 0.0%id, 83.0%wa, 0.3%hi,
>> 1.7%si, 0.0%st
>> Mem: 970084k total, 960100k used, 9984k free, 3028k buffers
>> Swap: 1966072k total, 983072k used, 983000k free, 31044k cached
>>
>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>> 3098 mail 18 0 7352 5504 1656 D 6.7 0.6 0:00.20 rt-mailgate
>> 1901 apache 18 0 201m 61m 3076 D 4.3 6.5 1:09.95 httpd
>> 22642 apache 16 0 222m 80m 3116 D 1.7 8.5 19:05.29 httpd
>> 162 root 10 -5 0 0 0 D 0.7 0.0 6:51.65 kswapd0
>> 3082 root 15 0 2316 996 784 R 0.3 0.1 0:00.06 top
>> 22641 apache 18 0 208m 12m 3116 D 0.3 1.3 19:00.47 httpd
>> 3080 root 15 0 0 0 0 D 0.0 0.0 0:00.00 pdflush
>> 3086 root 15 0 0 0 0 D 0.0 0.0 0:00.00 pdflush
>> 22645 apache 16 0 211m 11m 3124 D 0.0 1.2 17:14.01 httpd
>>
>>
>> top - 08:48:35 up 2 days, 21:04, 1 user, load average: 7.12, 4.29,
>> 2.85
>> Tasks: 100 total, 2 running, 98 sleeping, 0 stopped, 0 zombie
>> Cpu(s): 9.9%us, 1.7%sy, 0.0%ni, 0.0%id, 86.8%wa, 0.3%hi,
>> 1.3%si, 0.0%st
>> Mem: 970084k total, 960128k used, 9956k free, 3028k buffers
>> Swap: 1966072k total, 983036k used, 983036k free, 30988k cached
>>
>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>> 1901 apache 18 0 201m 62m 3076 D 4.0 6.6 1:10.07 httpd
>> 22642 apache 16 0 222m 81m 3116 D 1.3 8.6 19:05.33 httpd
>> 162 root 10 -5 0 0 0 D 0.3 0.0 6:51.66 kswapd0
>> 22641 apache 18 0 208m 12m 3116 D 0.3 1.4 19:00.48 httpd
>> 22645 apache 16 0 211m 12m 3124 D 0.3 1.3 17:14.02 httpd
>> 367 root 10 -5 0 0 0 D 0.0 0.0 0:38.13 kjournald
>> 3080 root 15 0 0 0 0 D 0.0 0.0 0:00.00 pdflush
>> 3082 root 15 0 2316 996 784 R 0.0 0.1 0:00.06 top
>> 3086 root 15 0 0 0 0 R 0.0 0.0 0:00.00 pdflush
>> 22643 apache 16 0 207m 12m 3116 D 0.0 1.3 19:36.90 httpd
>>
>>
>> top - 08:48:38 up 2 days, 21:04, 1 user, load average: 7.12, 4.29,
>> 2.85
>> Tasks: 100 total, 1 running, 99 sleeping, 0 stopped, 0 zombie
>> Cpu(s): 7.3%us, 1.3%sy, 0.0%ni, 0.0%id, 90.0%wa, 0.3%hi,
>> 1.0%si, 0.0%st
>> Mem: 970084k total, 959856k used, 10228k free, 3024k buffers
>> Swap: 1966072k total, 985244k used, 980828k free, 31032k cached
>>
>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>> 1901 apache 18 0 201m 62m 3076 D 6.3 6.6 1:10.26 httpd
>> 22642 apache 16 0 222m 82m 3116 D 1.7 8.7 19:05.38 httpd
>> 162 root 10 -5 0 0 0 D 0.7 0.0 6:51.68 kswapd0
>> 3082 root 15 0 2316 996 784 R 0.3 0.1 0:00.07 top
>> 22641 apache 18 0 208m 13m 3116 D 0.3 1.4 19:00.49 httpd
>> 3080 root 15 0 0 0 0 D 0.0 0.0 0:00.00 pdflush
>> 3086 root 15 0 0 0 0 D 0.0 0.0 0:00.00 pdflush
>> 22643 apache 16 0 207m 12m 3116 D 0.0 1.3 19:36.90 httpd
>> 22645 apache 16 0 211m 12m 3124 D 0.0 1.3 17:14.02 httpd
>>
>>
>> top - 08:48:41 up 2 days, 21:04, 1 user, load average: 7.27, 4.37,
>> 2.88
>> Tasks: 100 total, 1 running, 99 sleeping, 0 stopped, 0 zombie
>> Cpu(s): 10.3%us, 1.3%sy, 0.0%ni, 0.0%id, 87.0%wa, 0.3%hi,
>> 1.0%si, 0.0%st
>> Mem: 970084k total, 959516k used, 10568k free, 3016k buffers
>> Swap: 1966072k total, 983232k used, 982840k free, 31028k cached
>>
>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>> 1901 apache 18 0 201m 62m 3076 D 9.0 6.6 1:10.53 httpd
>> 22642 apache 17 0 222m 83m 3116 D 1.7 8.8 19:05.43 httpd
>> 162 root 10 -5 0 0 0 D 0.3 0.0 6:51.69 kswapd0
>> 3082 root 15 0 2316 996 784 R 0.3 0.1 0:00.08 top
>> 22643 apache 16 0 207m 12m 3116 D 0.3 1.4 19:36.91 httpd
>> 22645 apache 16 0 211m 13m 3124 D 0.3 1.4 17:14.03 httpd
>> 367 root 10 -5 0 0 0 D 0.0 0.0 0:38.13 kjournald
>> 3080 root 15 0 0 0 0 D 0.0 0.0 0:00.00 pdflush
>> 3086 root 15 0 0 0 0 D 0.0 0.0 0:00.00 pdflush
>> 22641 apache 18 0 208m 13m 3116 D 0.0 1.4 19:00.49 httpd
>>
>>
>> top - 08:48:44 up 2 days, 21:04, 1 user, load average: 7.27, 4.37,
>> 2.88
>> Tasks: 100 total, 1 running, 99 sleeping, 0 stopped, 0 zombie
>> Cpu(s): 14.0%us, 1.3%sy, 0.0%ni, 0.0%id, 83.4%wa, 0.3%hi,
>> 1.0%si, 0.0%st
>> Mem: 970084k total, 959932k used, 10152k free, 3012k buffers
>> Swap: 1966072k total, 983124k used, 982948k free, 31024k cached
>>
>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>> 1901 apache 18 0 201m 63m 3076 D 13.2 6.7 1:10.93 httpd
>> 22642 apache 17 0 222m 84m 3116 D 1.0 8.9 19:05.46 httpd
>> 162 root 10 -5 0 0 0 D 0.3 0.0 6:51.70 kswapd0
>> 22641 apache 16 0 208m 14m 3116 D 0.3 1.5 19:00.50 httpd
>> 3080 root 15 0 0 0 0 D 0.0 0.0 0:00.00 pdflush
>> 3082 root 15 0 2316 996 784 R 0.0 0.1 0:00.08 top
>> 3086 root 15 0 0 0 0 D 0.0 0.0 0:00.00 pdflush
>> 22643 apache 16 0 207m 13m 3116 D 0.0 1.4 19:36.91 httpd
>>
>>
>> top - 08:48:47 up 2 days, 21:04, 1 user, load average: 7.33, 4.43,
>> 2.91
>> Tasks: 100 total, 1 running, 99 sleeping, 0 stopped, 0 zombie
>> Cpu(s): 41.9%us, 1.7%sy, 0.0%ni, 0.0%id, 54.5%wa, 0.7%hi,
>> 1.3%si, 0.0%st
>> Mem: 970084k total, 960020k used, 10064k free, 2992k buffers
>> Swap: 1966072k total, 983036k used, 983036k free, 31048k cached
>>
>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>> 1901 apache 18 0 201m 63m 3076 D 41.9 6.7 1:12.19 httpd
>> 22642 apache 17 0 222m 84m 3116 D 1.0 9.0 19:05.49 httpd
>> 22643 apache 16 0 207m 13m 3116 D 0.7 1.4 19:36.93 httpd
>> 3082 root 15 0 2316 996 784 R 0.3 0.1 0:00.09 top
>> 162 root 10 -5 0 0 0 D 0.0 0.0 6:51.70 kswapd0
>> 3080 root 15 0 0 0 0 D 0.0 0.0 0:00.00 pdflush
>> 3086 root 15 0 0 0 0 D 0.0 0.0 0:00.00 pdflush
>> 22641 apache 16 0 208m 14m 3116 D 0.0 1.5 19:00.50 httpd
>> 22645 apache 16 0 211m 14m 3124 D 0.0 1.5 17:14.04 httpd
>>
>> top - 08:48:50 up 2 days, 21:04, 1 user, load average: 7.46, 4.50,
>> 2.94
>> Tasks: 101 total, 3 running, 98 sleeping, 0 stopped, 0 zombie
>> Cpu(s): 3.2%us, 1.7%sy, 0.0%ni, 0.0%id, 93.1%wa, 0.3%hi,
>> 1.7%si, 0.0%st
>> Mem: 970084k total, 960184k used, 9900k free, 2992k buffers
>> Swap: 1966072k total, 984032k used, 982040k free, 31180k cached
>>
>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>> 1901 apache 18 0 201m 64m 3076 D 2.0 6.8 1:12.25 httpd
>> 22642 apache 17 0 222m 85m 3116 R 1.0 9.0 19:05.52 httpd
>> 162 root 10 -5 0 0 0 D 0.7 0.0 6:51.72 kswapd0
>> 22645 apache 16 0 211m 14m 3124 D 0.7 1.6 17:14.06 httpd
>> 22641 apache 16 0 208m 15m 3116 D 0.3 1.6 19:00.51 httpd
>> 367 root 11 -5 0 0 0 D 0.0 0.0 0:38.13 kjournald
>> 3080 root 15 0 0 0 0 R 0.0 0.0 0:00.00 pdflush
>> 3082 root 15 0 2316 996 784 R 0.0 0.1 0:00.09 top
>> 3086 root 15 0 0 0 0 D 0.0 0.0 0:00.00 pdflush
>> 22643 apache 16 0 207m 13m 3116 D 0.0 1.4 19:36.93 httpd
>>
>>
>> top - 08:48:53 up 2 days, 21:04, 1 user, load average: 7.46, 4.50,
>> 2.94
>> Tasks: 101 total, 1 running, 100 sleeping, 0 stopped, 0 zombie
>> Cpu(s): 5.3%us, 1.7%sy, 0.0%ni, 0.0%id, 91.7%wa, 0.7%hi,
>> 0.7%si, 0.0%st
>> Mem: 970084k total, 960156k used, 9928k free, 2992k buffers
>> Swap: 1966072k total, 983068k used, 983004k free, 31464k cached
>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>> 1901 apache 18 0 201m 64m 3076 D 3.5 6.9 1:12.37 httpd
>> 22642 apache 17 0 222m 85m 3116 D 1.4 9.1 19:05.57 httpd
>> 162 root 10 -5 0 0 0 D 0.3 0.0 6:51.73 kswapd0
>> 3082 root 15 0 2316 996 784 R 0.3 0.1 0:00.10 top
>> 22641 apache 16 0 208m 15m 3116 D 0.3 1.7 19:00.52 httpd
>> 22643 apache 16 0 207m 14m 3116 D 0.3 1.5 19:36.94 httpd
>> 3080 root 15 0 0 0 0 D 0.0 0.0 0:00.00 pdflush
>> 3086 root 15 0 0 0 0 D 0.0 0.0 0:00.00 pdflush
>> 22645 apache 16 0 211m 15m 3124 D 0.0 1.6 17:14.06 httpd
>>
>>
>> top - 08:48:56 up 2 days, 21:04, 1 user, load average: 7.50, 4.56,
>> 2.97
>> Tasks: 101 total, 3 running, 98 sleeping, 0 stopped, 0 zombie
>> Cpu(s): 1.2%us, 0.9%sy, 0.0%ni, 0.0%id, 96.5%wa, 0.6%hi,
>> 0.9%si, 0.0%st
>> Mem: 970084k total, 960380k used, 9704k free, 2936k buffers
>> Swap: 1966072k total, 983044k used, 983028k free, 31392k cached
>>
>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>> 22642 apache 17 0 222m 86m 3116 D 1.0 9.1 19:05.60 httpd
>> 162 root 10 -5 0 0 0 D 0.3 0.0 6:51.74 kswapd0
>> 3082 root 15 0 2316 996 784 R 0.3 0.1 0:00.11 top
>> 22643 apache 16 0 207m 14m 3116 D 0.3 1.5 19:36.95 httpd
>> 22645 apache 16 0 211m 15m 3124 D 0.3 1.6 17:14.07 httpd
>> 367 root 10 -5 0 0 0 D 0.0 0.0 0:38.13 kjournald
>> 3080 root 15 0 0 0 0 R 0.0 0.0 0:00.00 pdflush
>> 3086 root 15 0 0 0 0 R 0.0 0.0 0:00.00
>> pdflush
>> 3099 apache 17 0 8172 3052 2092 D 0.0 0.3 0:00.04 sendmail
>> 22641 apache 16 0 208m 16m 3116 D 0.0 1.7 19:00.52 httpd
>>
>>
>> top - 08:49:00 up 2 days, 21:04, 1 user, load average: 7.62, 4.63,
>> 3.00
>> Tasks: 101 total, 2 running, 99 sleeping, 0 stopped, 0 zombie
>> Cpu(s): 1.7%us, 1.0%sy, 0.0%ni, 0.0%id, 95.3%wa, 0.3%hi,
>> 1.7%si, 0.0%st
>> Mem: 970084k total, 959064k used, 11020k free, 2936k buffers
>> Swap: 1966072k total, 983532k used, 982540k free, 31148k cached
>>
>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>> 22642 apache 17 0 222m 86m 3116 D 1.4 9.2 19:05.65 httpd
>> 22643 apache 16 0 207m 15m 3116 D 0.3 1.6 19:36.96 httpd
>> 22645 apache 16 0 211m 16m 3124 D 0.3 1.7 17:14.08 httpd
>> 3080 root 15 0 0 0 0 D 0.0 0.0 0:00.00 pdflush
>> 3082 root 15 0 2316 996 784 R 0.0 0.1 0:00.11 top
>> 3086 root 15 0 0 0 0 D 0.0 0.0 0:00.00 pdflush
>> 3099 apache 17 0 8172 3076 2116 R 0.0 0.3 0:00.04 sendmail
>> 22641 apache 16 0 208m 16m 3116 D 0.0 1.8 19:00.52 httpd
>>
>>
>> top - 08:49:03 up 2 days, 21:04, 1 user, load average: 7.62, 4.63,
>> 3.00
>> Tasks: 103 total, 2 running, 101 sleeping, 0 stopped, 0 zombie
>> Cpu(s): 3.7%us, 1.7%sy, 0.0%ni, 0.0%id, 93.3%wa, 0.3%hi,
>> 1.0%si, 0.0%st
>> Mem: 970084k total, 959560k used, 10524k free, 2928k buffers
>> Swap: 1966072k total, 983440k used, 982632k free, 30496k cached
>>
>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>> 22642 apache 17 0 222m 87m 3116 D 2.0 9.2 19:05.71 httpd
>> 3102 root 16 0 9276 2300 1564 D 1.3 0.2 0:00.04 sendmail
>> 162 root 10 -5 0 0 0 D 0.7 0.0 6:51.77 kswapd0
>> 3082 root 15 0 2316 996 784 R 0.3 0.1 0:00.12 top
>> 22641 apache 16 0 208m 17m 3116 D 0.3 1.8 19:00.53 httpd
>> 22643 apache 16 0 207m 15m 3116 D 0.3 1.7 19:36.97 httpd
>> 367 root 10 -5 0 0 0 D 0.0 0.0 0:38.13 kjournald
>> 22645 apache 16 0 211m 16m 3124 R 0.0 1.7 17:14.08 httpd
>>
>> ==========================================
>>
>> [root at support ~]# mailq
>> /var/spool/mqueue (6 requests)
>> -----Q-ID----- --Size-- -----Q-Time-----
>> ------------Sender/Recipient----------- o51Cxkw2003214 8112 Tue Jun 1
>> 08:59 <jay at centritech.net>
>> 7BIT (Deferred: prog mailer (/usr/sbin/smrsh) exited with
>> EX_TEMPF)
>> "|/etc/smrsh/rt-mailgate
>> --queue 'ccs:
>> o51CuKTF003192 633211 Tue Jun 1 08:56 <>
>> 7BIT ("|/etc/smrsh/rt-mailgate --queue 'umx: client
>> helpdesk' --ac)
>> "|/etc/smrsh/rt-mailgate
>> --queue 'umx:
>> o51CGaSM001908 1170766 Tue Jun 1 08:16 <>
>> 7BIT ("|/etc/smrsh/rt-mailgate --queue 'umx: client
>> helpdesk' --ac)
>> "|/etc/smrsh/rt-mailgate
>> --queue 'umx:
>> o51CuIdl003186 1355798 Tue Jun 1 08:56 <>
>> 7BIT ("|/etc/smrsh/rt-mailgate --queue 'umx: client
>> helpdesk' --ac)
>> "|/etc/smrsh/rt-mailgate
>> --queue 'umx:
>> o51Ckond003072 3612709 Tue Jun 1 08:46 <>
>> 7BIT ("|/etc/smrsh/rt-mailgate --queue 'umx: client
>> helpdesk' --ac)
>> "|/etc/smrsh/rt-mailgate
>> --queue 'umx:
>> o51CutnV003202 3618441 Tue Jun 1 08:56 <>
>> 7BIT ("|/etc/smrsh/rt-mailgate --queue 'umx: client
>> helpdesk' --ac)
>> "|/etc/smrsh/rt-mailgate
>> --queue 'umx:
>> Total requests: 6
>> [root at support ~]#
>>
>>
>
> --
>
>
>
> Cheers
>
> Richard Skelton
> Richard.Skelton at infineon.com Infineon Technologies UK Ltd
> Infineon House
> Great Western Court
> Hunts Ground Road
> Stoke Gifford
> Bristol BS34 8HP
> Tel +44(0)117 9528808
>
--
Cheers
Richard Skelton
Richard.Skelton at infineon.com Infineon Technologies UK Ltd
Infineon House
Great Western Court
Hunts Ground Road
Stoke Gifford
Bristol BS34 8HP
Tel +44(0)117 9528808
More information about the rt-users
mailing list