Roundcube Community Forum

News and Announcements => General Discussion => Topic started by: pkrc on March 12, 2019, 07:39:40 AM

Title: High CPU usage by IMAP process (and for long time)
Post by: pkrc on March 12, 2019, 07:39:40 AM
High CPU usage by IMAP process (and for long time)
during Roundcube connection
==================================================

Hello,

1. During connection from Roundcube (RC) to our IMAP server (and mail server),
there is problem with very high CPU usage by IMAP process:

- IMAP process uses almost 100% of CPU (as it is shown by 'top' command)
during first stage of login (between 1 and 2 minutes). 
After that I can work with RC (I can see all of my imap mailboxes
defined in $HOME/.subscriptions file), my INBOX, read emails from mailboxes,
send email and so on -- so it looks OK

- but this IMAP process still works on IMAP server and it still uses 100% of CPU
for long time

I can verify by 'lsof' command and by SELinux messages, that it tries to open
(probably) all of my subdirectories in my home directory.

2. The next step of the problem is, that when I try to use 'Settings - Folders' in RC
(for example, to make any changes with my subscriptions),
than the second instance of IMAP process is started on our IMAP server,
and it also uses almost 100% of CPU. 

So in summary, two IMAP processes are running (for my account)
on our IMAP server and they both are using almost 100% of CPU,
as reported by 'top'.

My connection to RC then sometimes stops working, I get error message:
(This page isn't working, .. is currently unable to handle this request,
HTTP Error 500, Reload).

I can also found in '/var/log/roundcube/error.log' information such as:

PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 100635344 bytes) in /usr/share/roundcubemail/program/lib/Roundcube/rcube_imap_generic.php on line 3738

PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 8192 bytes) in /usr/share/roundcubemail/program/lib/Roundcube/rcube_imap_generic.php on line 292

3. Even after I reconnect via RC to IMAP server, and than make properly 'Logout'
from my account, that IMAP process (or two processes) are still running.

These IMAP processes are shown in 'dovecot adm who' command and in 'top' results.
I have to kill these IMAP processes manually using 'kill -HUP' because
'kill -TERM' or 'dovadm kick' do not work.

4. There are no such problems as below:

(p1) high CPU usage for long time by IMAP processes

(p2) IMAP process still running even after 'logout'

when I use IMAP connection from Thunderbird or connection
from SquirrelMail -- using the same account and the same IMAP server.

So I assume, that the reason of the above problems
may be in connection method used by Roundcube
to work with IMAP (dovecot) server. 

How can we solve such problem?

Our configuration:
Roundcube Webmail 1.3.8
PHP 7.3.2
IMAP Server: dovecot 2.2.10

Regards,
pkrc

Title: Re: High CPU usage by IMAP process (and for long time)
Post by: alec on March 12, 2019, 07:59:32 AM
While there's obviously an error somewhere, you should know that even if you close the browser some http requests might still not abort the connection and be e.g. stuck in "an infinite loop" state. This is how it is. You might try enabling imap_debug in Roundcube to see if that gives you some hints. Such PHP memory errors often happen when there's some infinite loop/condition.

ps. you use recent Roundcube and PHP, but dovecot version is very old.
Title: Re: High CPU usage by IMAP process (and for long time)
Post by: pkrc on March 12, 2019, 09:24:04 AM
Thank you for your response.

The dovecot version is old, but it works OK (so far),
for all types of our IMAP clients.

The only problem is now with Roundcube connection,
which generates long time and CPU consuming IMAP process
of scanning (reading, getattr) full content of home directory.

I think this IMAP connection should work as follows:

- during login, it should check the subscribed IMAP folders only
(for example using $HOME/.subscriptions file)

- when using 'Settings - Folders' it should list the main level
of home directory and wait for the user operation to
select subdirectory and then list its contents,
or select mailbox folder and then subscribe it

Why does this IMAP process scan recursively full content of home directory?
 
I think, Thunderbid (using IMAP) or SquirrelMail do not request such scanning.

Regards,
pkrc