[Net-IMAP-Server] having issues with managing memory for large accounts

[Alex Vandiver]

On Tue, 2011-05-24 at 14:54 -0700, Sachin Sebastian wrote:
> I'm writing a imap server and have built it looking into the example 
> provided. But I'm having issues where I'm running out of memory for 
> large accounts. I'm adding messages by passing it to 
> Net::IMAP::Server::Message while creating the object. So all the 
> messages are getting loaded to memory. Is there a way I can pass in a 
> file handle and Net::IMAP::Server::Message read it when my imap client 
> request for the message? Or is there any other work around?

Attempting to store all messages in memory will absolutely lead to
memory management problems.  The easiest fix is to subclass
Net::IMAP::Server::Message, and override its 'mime' method to read off
disk on-demand.  For example, in Hiveminder's IMAP server simply stores
the backing task ID in the message class, and overrides the 'mime'
method to perform a database query for the content.
  An additional memory-saving trick is to lazily populate the mailbox on
->poll, not on create, and purge messages from the mailbox when there
are no connections observing it.  I can post some of the code we use to
do this later if you're interested.
 - Alex