[rt-users] Can't fork at Mailer.pm

[Tim Cutts]

On 8 Oct 2012, at 10:03, Martin Drasar <drasar at ics.muni.cz> wrote:

> thank you for the tips, I was not aware about doubling the virtual
> memory requirements. I will look into it.

It's only for a split second, and the memory isn't used.  UNIX doesn't just start an application in one go, it uses two steps:

1)  fork

This duplicates the current process, and both processes continue executing.
UNIX does this with so-called copy-on-write, so the pages aren't actually duplicated until they're modified, but nevertheless, the potential exists for both copies to be completely required, so if you have vm.overcommit_memory set to 2, you need all that virtual memory to be available just in case.

2)  exec

the newly forked process usually immediately executes the exec() system call, which replaces the current process' virtual memory image with the desired program.  At this point, the virtual memory requirements go back down again (assuming the new process is something small, which in  the case of sendmail, it is).

Apologies if there are errors and oversimplifications in the above, I'm not exactly a total UNIX beardy type.

We've seen this bite us on some of our HPC clusters at work - there, we do have vm.overcommit_memory set to 2, because we want programs allocating too much memory to die immediately, without risk of the kernel's out of memory killer zapping the wrong thing.  A common problem on such machines is very large computational jobs trying to start sub-processes - as soon as they do, their virtual memory requirement doubles and the fork() fails with an out of memory error.

It just seemed to me that it might be the case with your system.  Try the output of:

cat /proc/sys/vm/overcommit_memory

if it's 2, then this is probably your problem.
Tim



-- 
 The Wellcome Trust Sanger Institute is operated by Genome Research 
 Limited, a charity registered in England with number 1021457 and a 
 company registered in England with number 2742969, whose registered 
 office is 215 Euston Road, London, NW1 2BE.