[rt-users] Memory leak in RT Crontool?

[Ham MI-ID, Torsten Brumm]

Hi RT Users,

Since yesterday i have a very strange behavior on my RT. I have a cron
script starting rt-crontool every 5 minutes.

/opt/rt3ast/bin/rt-crontool 
--search RT::Search::FromSQL 
--search-arg "Status='stalled' AND Queue='GAI_BreakBulk' AND (
'CF.{CIEL-STATUS}' = '1300' OR 'CF.{CIEL-STATUS}' = '1400' )" 
--action RT::Action::AutoNew 
--action-arg new

Inside this Queue are round about 130 tickets. 

After few hours the system slows down until it crashes with this error
inside the messages:

Nov  9 11:24:19 lxfrabort01 kernel: printk: 2731 messages suppressed.
Nov  9 11:24:19 lxfrabort01 kernel: rt-crontool invoked oom-killer:
gfp_mask=0x201d2, order=0, oomkilladj=0
Nov  9 11:24:19 lxfrabort01 kernel:  [<c0452c32>]
out_of_memory+0x3b/0x179
Nov  9 11:24:19 lxfrabort01 kernel:  [<c0454065>]
__alloc_pages+0x1fe/0x27e
Nov  9 11:24:19 lxfrabort01 kernel:  [<c04552af>]
__do_page_cache_readahead+0xc4/0x1c6
Nov  9 11:24:19 lxfrabort01 kernel:  [<c044f991>] sync_page+0x0/0x3b
Nov  9 11:24:19 lxfrabort01 kernel:  [<c044c6f5>]
__delayacct_blkio_end+0x32/0x35
Nov  9 11:24:19 lxfrabort01 kernel:  [<c05fb55c>]
__wait_on_bit_lock+0x4b/0x52
Nov  9 11:24:19 lxfrabort01 kernel:  [<f8926b96>]
dm_any_congested+0x2f/0x35 [dm_mod]
Nov  9 11:24:19 lxfrabort01 kernel:  [<c0452268>]
filemap_nopage+0x151/0x315
Nov  9 11:24:19 lxfrabort01 kernel:  [<c045a8c2>]
__handle_mm_fault+0x172/0x87b
Nov  9 11:24:19 lxfrabort01 kernel:  [<c05fad89>] schedule+0x90d/0x9ba
Nov  9 11:24:19 lxfrabort01 kernel:  [<c042d63f>]
do_sigaction+0x117/0x156
Nov  9 11:25:25 lxfrabort01 kernel:  [<c05fd55f>]
do_page_fault+0x20a/0x4b8
Nov  9 11:25:25 lxfrabort01 kernel:  [<c05fd355>]
do_page_fault+0x0/0x4b8
Nov  9 11:25:25 lxfrabort01 kernel:  [<c0404a71>] error_code+0x39/0x40
Nov  9 11:25:25 lxfrabort01 kernel:  =======================
Nov  9 11:25:25 lxfrabort01 kernel: Mem-info:
Nov  9 11:25:25 lxfrabort01 kernel: DMA per-cpu:
Nov  9 11:25:25 lxfrabort01 kernel: cpu 0 hot: high 0, batch 1 used:0
Nov  9 11:25:25 lxfrabort01 kernel: cpu 0 cold: high 0, batch 1 used:0
Nov  9 11:25:25 lxfrabort01 kernel: DMA32 per-cpu: empty
Nov  9 11:25:25 lxfrabort01 kernel: Normal per-cpu:
Nov  9 11:25:25 lxfrabort01 kernel: cpu 0 hot: high 186, batch 31
used:27
Nov  9 11:25:25 lxfrabort01 kernel: cpu 0 cold: high 62, batch 15
used:44
Nov  9 11:25:25 lxfrabort01 kernel: HighMem per-cpu:
Nov  9 11:25:25 lxfrabort01 kernel: cpu 0 hot: high 186, batch 31
used:14
Nov  9 11:25:25 lxfrabort01 kernel: cpu 0 cold: high 62, batch 15
used:11
Nov  9 11:25:25 lxfrabort01 kernel: Free pages:      111660kB (1596kB
HighMem)
Nov  9 11:25:25 lxfrabort01 kernel: Active:470936 inactive:469165
dirty:0 writeback:0 unstable:0 free:27915 slab:1793 mapped:91
pagetables:2289
Nov  9 11:25:25 lxfrabort01 kernel: DMA free:12340kB min:68kB low:84kB
high:100kB active:0kB inactive:0kB present:16384kB pages_scanned:0
all_unreclaimable? yes
Nov  9 11:25:25 lxfrabort01 kernel: lowmem_reserve[]: 0 0 880 3824
Nov  9 11:25:25 lxfrabort01 kernel: DMA32 free:0kB min:0kB low:0kB
high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0
all_unreclaimable? no
Nov  9 11:25:25 lxfrabort01 kernel: lowmem_reserve[]: 0 0 880 3824
Nov  9 11:25:25 lxfrabort01 kernel: Normal free:97724kB min:3756kB
low:4692kB high:5632kB active:382784kB inactive:377384kB
present:901120kB pages_scanned:1268501 all_unreclaimable? yes
Nov  9 11:25:25 lxfrabort01 kernel: lowmem_reserve[]: 0 0 0 23552
Nov  9 11:25:25 lxfrabort01 kernel: HighMem free:1596kB min:512kB
low:3656kB high:6800kB active:1500960kB inactive:1499276kB
present:3014656kB pages_scanned:4695953 all_unreclaimable? yes
Nov  9 11:25:25 lxfrabort01 kernel: lowmem_reserve[]: 0 0 0 0
Nov  9 11:25:25 lxfrabort01 kernel: DMA: 3*4kB 5*8kB 4*16kB 2*32kB
4*64kB 3*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 2*4096kB = 12340kB
Nov  9 11:25:25 lxfrabort01 kernel: DMA32: empty
Nov  9 11:25:25 lxfrabort01 kernel: Normal: 1*4kB 1*8kB 1*16kB 1*32kB
0*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 23*4096kB = 97724kB
Nov  9 11:25:25 lxfrabort01 kernel: HighMem: 271*4kB 16*8kB 0*16kB
0*32kB 0*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB =
1596kB
Nov  9 11:25:25 lxfrabort01 kernel: Swap cache: add 3707117, delete
3707077, find 546011/727786, race 0+20
Nov  9 11:25:25 lxfrabort01 kernel: Free swap  = 0kB
Nov  9 11:25:25 lxfrabort01 kernel: Total swap = 2031608kB
Nov  9 11:25:25 lxfrabort01 kernel: Free swap:            0kB
Nov  9 11:25:25 lxfrabort01 kernel: 983040 pages of RAM
Nov  9 11:25:25 lxfrabort01 kernel: 753664 pages of HIGHMEM
Nov  9 11:25:25 lxfrabort01 kernel: 9002 reserved pages
Nov  9 11:25:25 lxfrabort01 kernel: 1520 pages shared
Nov  9 11:25:25 lxfrabort01 kernel: 40 pages swap cached
Nov  9 11:25:25 lxfrabort01 kernel: 0 pages dirty
Nov  9 11:25:25 lxfrabort01 kernel: 0 pages writeback
Nov  9 11:25:25 lxfrabort01 kernel: 91 pages mapped
Nov  9 11:25:25 lxfrabort01 kernel: 1793 pages slab
Nov  9 11:25:25 lxfrabort01 kernel: 2289 pages pagetables

What i googled so far is, that the oom_killer is a process that checks
for process that causes massive ram usage and tries to kill them -
without success.

This is a vm installation with centos 5. 1 Processor at 3 GHz with 4 GB
RAM and 4 GB Swap.

Are there any know issues with RT-CRONTOOL and memory problems?

Torsten