Author Topic: Attachment Excessive Memory Use Error  (Read 5697 times)

Offline sharms

  • Newbie
  • *
  • Posts: 2
Attachment Excessive Memory Use Error
« on: June 20, 2007, 12:16:59 AM »
Hi guys,

I am suffering from a memory related error.

  "Fatal error: Allowed memory size of blah bytes exhausted (tried to allocate blah bytes)"

I updated upload_max_size, post_max_size and memory_limit inside php.ini to 128MB (Dont do this unless you have the resources to do so as some of its on a per script basis) and also made relevant changes to my .htaccess files. Just for kicks I also increased max_execution_time to 2400.

This only went part way to resolving my initial issue but also revealed a more serious issue. For example when I upload a 10mb file or attempt to download a 10mb file for that matter I was finding that it was seriously over using memory. After some investigation I found the following:

When trying to send an email the whole email and all attachments are being passed into the function quotedata() in program/lib/Net/SMTP.php the offending line is where roundcube tries to do some funky stuff with the $data object

 $data = preg_replace(array('/(?
Commenting this line out resolves all my memory related issues or so it would seem and sending files between mac and windows causes no issues all files appear fine.

Is there any real reason you are doing this string operation on the body let alone the attachments?
messing around with that much memory just does not seem right to me.

Simon
 



Offline sharms

  • Newbie
  • *
  • Posts: 2
Re: Attachment Excessive Memory Use Error
« Reply #1 on: June 24, 2007, 10:39:27 PM »
Ok so this is actually needed. Well sort of.

Without this line there is some odd stuff in the headers of emails sent.
So the real problem is that we need to not be passing attachments into this function.

Any suggestions or comments..

yes I am talking to you devs :)

Offline JohnDoh

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 2,245
Re: Attachment Excessive Memory Use Error
« Reply #2 on: September 24, 2007, 05:07:27 AM »
/bump

Does anyone have any ideas what (if anything) can be done about this? When I try to send approx 7mb of attachments it exceeds the 64mb memory limit. This strikes me as being a little excessive.

Thanks
Roundcube Plugins: Contextmenu, SpamAssassin Prefs, and more…

Offline Scine

  • Newbie
  • *
  • Posts: 8
Re: Attachment Excessive Memory Use Error
« Reply #3 on: October 12, 2007, 05:31:58 PM »
I had a very similar issue, but I am running Linux, so it may not be exact, but here's what I did to solve my issue:

http://roundcubeforum.net/forum/index.php?topic=760.0

Hopefully that helps! Sorry if it doesn't.


Offline JohnDoh

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 2,245
Re: Attachment Excessive Memory Use Error
« Reply #4 on: October 20, 2007, 05:06:54 AM »
Hi, Thanks Scine, I am using Debian as well. Unfortuantly upping the PHP memory limit on the server isnt really a solution, only a work around untill some one comes along and tries to attach bigger (or more) files.

From what I seen other webmail clients dont have this problem so there must be a way round it, unfortuantly at the moment i cant find it :(
Roundcube Plugins: Contextmenu, SpamAssassin Prefs, and more…

Offline tom555

  • Jr. Member
  • **
  • Posts: 19
Re: Attachment Excessive Memory Use Error
« Reply #5 on: October 31, 2007, 08:33:48 AM »
I have exactly the same problem.

Offline zdali

  • Newbie
  • *
  • Posts: 3
Re: Attachment Excessive Memory Use Error
« Reply #6 on: November 19, 2007, 07:28:15 PM »
Like the TRAC bug points out, the culprit is PEAR Net_SMTP class.

Well, after a little poking around i have few thoughts...

#1 replace PEAR classes with ZendFramework mail classes which still have same underlying issue but at least have predictable memory use. From my testing ZendFramework will use 2x(sumof(filesize)) + 700K overhead.

#2 custom class using PHP streams, possibly API compatible with PEAR. using temp files to construct the message and send it on...

dali



Offline chinaman

  • Newbie
  • *
  • Posts: 9
Re: Attachment Excessive Memory Use Error
« Reply #7 on: December 15, 2007, 07:31:35 AM »
Hi,

how did u get along with this issue?
I am using v0.1-rc2 an like it very much, but this problem has brought me a lot of hours searching around and trying to find a solution to send large attachments using roundcube.

Best regards
Thomas

Offline corradofiore

  • Newbie
  • *
  • Posts: 4
Re: Attachment Excessive Memory Use Error - a proposal
« Reply #8 on: January 26, 2008, 06:30:13 AM »
Hi All,

this is a common problem among webmail clients: the body of the message can easily become a huge string that needs roughly 2x its size to be processed. That happens because the PHP interpreter shall parse the entire string to convert newlines characters so they're RFC compliant before doing the SMTP connection.

If you have Sendmail running on the same machine as Roundcube, you can also follow another approach: instead of a SMTP connection, open a direct socket connection to Sendmail (assuming you're working on a Linux/UNIX box). Not only it's much faster, you also don't need to replace newlines because Sendmail will do it for you (thus honoring the Unix mantra "be liberal in what you accept, be strict in what you send"):

http://www.webservertalk.com/message1285765.html

The last webmail system I used (Telaen.org) let you choose between a SMTP connection and a straight Sendmail connection in its configuration file. I ran some tests, and the results were quite impressive: the same message that made SMTP run out of memory was being delivered fast and without errors when I used the Sendmail connection.

By the way, Telaen uses the well-known and well-documented PHP Mailer class:

http://sourceforge.net/projects/phpmailer

...therefore I assume that also Roundcube could.

IMHO it's a matter of development strategy: this workaround would work for a large share of users, but it's not applicable by everyone. For that reasons, developers could be reluctant in add such a feature. I would be interested in hearing their thoughts.

Cheers,
Corrado Fiore

Offline till

  • Jr. Member
  • **
  • Posts: 49
Re: Attachment Excessive Memory Use Error
« Reply #9 on: February 16, 2008, 07:24:44 AM »
http://trac.roundcube.net/wiki/Howto_ReportIssues

(Yes, I am talking to you "bugreporters".)

Offline JohnDoh

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 2,245
Re: Attachment Excessive Memory Use Error
« Reply #10 on: February 16, 2008, 08:25:48 AM »
Hi Till,

This is the ticket in trac http://trac.roundcube.net/ticket/1484660.

I think all the informaiton is there, you attach a big file then hit send and you will get the error. If not then I should be idle'ing in #roundcube, give me a poke and I will help as much as I can.
Roundcube Plugins: Contextmenu, SpamAssassin Prefs, and more…

Offline till

  • Jr. Member
  • **
  • Posts: 49
Re: Attachment Excessive Memory Use Error
« Reply #11 on: February 16, 2008, 10:16:42 AM »
Quote from: JohnDoh
Hi Till,

This is the ticket in trac http://trac.roundcube.net/ticket/1484660.

I think all the informaiton is there, you attach a big file then hit send and you will get the error. If not then I should be idle'ing in #roundcube, give me a poke and I will help as much as I can.

I'd be interested who "zdali" is. I think I never received a patch from him. ;)

If you can hunt him down, we might be able to fix this soon!

Offline till

  • Jr. Member
  • **
  • Posts: 49
Re: Attachment Excessive Memory Use Error
« Reply #12 on: February 20, 2008, 12:25:39 PM »
Updated the ticket, more a configuration issue.