Roundcube Community Forum

News and Announcements => General Discussion => Topic started by: adb101 on January 24, 2017, 12:58:47 PM

Title: File upload failed.
Post by: adb101 on January 24, 2017, 12:58:47 PM
I'm hoping someone can help. Ever since I installed version 1.1.3 I've had issues attaching files to emails. Attaching a file results in "File upload failed.". I can see the apache user writing a file out to the "temp" directory, but it is always zero size. I have loosened the permissions to 777 and, clearly, the web server can write to the area but, for some reason, doesn't write any content. I have upgraded to version 1.2.3 and I'm still getting the issue.

From the php.ini side of things I've got the following settings:

; Whether to allow HTTP file uploads.
; http://php.net/file-uploads
file_uploads = On

; Temporary directory for HTTP uploaded files (will use system default if not
; specified).
; http://php.net/upload-tmp-dir
upload_tmp_dir = /tmp

; Maximum allowed size for uploaded files.
; http://php.net/upload-max-filesize
upload_max_filesize = 50M

; Maximum number of files that can be uploaded via a single request
max_file_uploads = 20


Files in roundcube temp look like:

[root@xore temp]# ls -l
total 0
-rw------- 1 apache apache 0 Jan 24 11:37 rcmAttmntUOgZnX
-rw------- 1 apache apache 0 Jan 24 17:47 rcmAttmntfYm0s2
-rw------- 1 apache apache 0 Jan 24 11:35 rcmAttmntmKjVRN
-rw------- 1 apache apache 0 Jan 24 11:41 rcmAttmntxtvs1S


SELinux is disabled.

As an aside, I have Horde installed on the same server and that does not have any issues with attachments.

I have put the Roundcube logging to max but nothing is being written to the logs directory about the file upload failing.

If anyone could suggest how I can get any additional debug or what to try next I'd me most grateful as it's driving me nuts!
Title: Re: File upload failed.
Post by: SKaero on January 24, 2017, 01:13:34 PM
What is your post_max_size?
Title: Re: File upload failed.
Post by: adb101 on January 24, 2017, 03:18:40 PM
post_max_size is not set in php.ini.

Other apps, such as Horde, are posting attachments fine with the same php.ini settings...
Title: Re: File upload failed.
Post by: SKaero on January 24, 2017, 03:37:05 PM
That could be the problem, post_max_size needs to larger then upload_max_filesize size since files are in the post data. Horde most likely has it's own php.ini, .htaccess or is overwriting the setting at run time.
Title: Re: File upload failed.
Post by: adb101 on January 25, 2017, 03:52:18 AM
Yes, good point. I have checked those and made them the same, but still getting the issue (any file attachment, any size).

I have narrowed the issue down to this code in ./program/steps/mail/attachments.inc:

      else {  // upload failed
            if ($err == UPLOAD_ERR_INI_SIZE || $err == UPLOAD_ERR_FORM_SIZE) {
                $size = $RCMAIL->show_bytes(rcube_utils::max_upload_size());
                $msg  = $RCMAIL->gettext(array('name' => 'filesizeerror', 'vars' => array('size' => $size)));
            }
            else if ($attachment['error']) {
                $msg = $attachment['error'];
            }
            else {
                $msg = $RCMAIL->gettext('fileuploaderror');
            }

            if ($attachment['error'] || $err != UPLOAD_ERR_NO_FILE) {
                $OUTPUT->command('display_message', $msg, 'error');    <---THIS LINE IS GENERATING THE ERROR MSG ON SCREEN
                $OUTPUT->command('remove_from_attachment_list', $uploadid);
            }


I have put some debug in and $err is coming back as "0". Unfortunately I've got extremely limited PHP experience so if anyone can suggest some additional debug to add to this section to actually work out what the issue is I'd be grateful.

Thanks!
Title: Re: File upload failed.
Post by: SKaero on January 25, 2017, 05:17:56 AM
This sounds like a permissions problem, what is the exact error message in the error log?
Title: Re: File upload failed.
Post by: adb101 on January 25, 2017, 06:00:01 AM
Unfortunately the error log is not writing anything when the error occurs. I have the following debug settings for roundcube set in config_inc.php:

// ----------------------------------
// LOGGING/DEBUGGING
// ----------------------------------
// system error reporting, sum of: 1 = log; 4 = show
$config['debug_level'] = 1;

  // Log SQL queries to <log_dir>/sql or to syslog
  $config['sql_debug'] = true;

  // Log IMAP conversation to <log_dir>/imap or to syslog
  $config['imap_debug'] = true;

  // Log LDAP conversation to <log_dir>/ldap or to syslog
  $config['ldap_debug'] = true;

  // Log SMTP conversation to <log_dir>/smtp or to syslog
  $config['smtp_debug'] = true;


Nothing being written to the error log  :(
Title: Re: File upload failed.
Post by: SKaero on January 25, 2017, 06:05:29 AM
It definitely sounds like a permissions problem, if Roundcube isn't writing anything to the log then it most likely doesn't have access. If Roundcube doesn't have access to write to the temp folder then it wont be able to add attachments.
Title: Re: File upload failed.
Post by: adb101 on January 25, 2017, 06:54:35 AM
But it is writing to the other logs in that directory?

[root@xore logs]# ls -l
total 163636
-rw-rw-rw- 1 root root      4707 Jan 23 10:30 errors    <-- Contains just user login error messages
-rw-rw-rw- 1 root root 161504231 Jan 25 11:48 imap
-rw-rw-rw- 1 root root     30812 Jan 24 09:11 sendmail
-rw-rw-rw- 1 root root   5993827 Jan 25 11:48 sql


It is able to write to the "temp" directory as the files appear, but are zero length:

[root@xore temp]# ls -lt | head
total 0
-rw------- 1 apache apache 0 Jan 25 11:05 rcmAttmntpDOyC5
-rw------- 1 apache apache 0 Jan 25 11:04 rcmAttmntPVOp07
-rw------- 1 apache apache 0 Jan 25 11:03 rcmAttmntfRwYtt
-rw------- 1 apache apache 0 Jan 25 11:02 rcmAttmntC8DNeO
-rw------- 1 apache apache 0 Jan 25 08:49 rcmAttmntw6ini7
-rw------- 1 apache apache 0 Jan 25 08:45 rcmAttmntrCVke1
-rw------- 1 apache apache 0 Jan 25 08:37 rcmAttmntVRsid7
-rw------- 1 apache apache 0 Jan 25 08:34 rcmAttmntS6wxNt
-rw------- 1 apache apache 0 Jan 25 08:33 rcmAttmntelSjdR


The filesystem the roundcube installation directory resides in has no special restrictions:

[root@xore temp]# mount
/dev/mapper/VolGroup00-LvRoot on / type ext4 (rw,user_xattr,acl)


Unless roundcube is trying to open some sort of special pipe or other construct to write files in "temp" which the OS is blocking..? Without any useful error message I'm a bit stuck...

Thanks.
Title: Re: File upload failed.
Post by: rm13 on January 27, 2017, 12:09:32 AM
Possibly the htaccess file. These php values may be set there.

php_value   upload_max_filesize   
php_value   post_max_size         
php_value   memory_limit         

Do your apache logs have any php errors that may shed some light?
Title: Re: File upload failed.
Post by: adb101 on February 02, 2017, 04:58:00 AM
Unfortunately there seems to be a complete lack of logging for this error, hence by difficultly in tracking down the issue!

I've set the following values in the .htaccess file in the RC directory:

php_flag    display_errors  On
php_flag    log_errors      On
php_value    error_log    logs/errors

php_value   upload_max_filesize   50M
php_value   post_max_size         50M
php_value   memory_limit          128M


As I mentioned before, the attachment file gets "touched" in the directory but there is no data inside it. This indicates that the process has write access.

Horde, which I have installed on the same server, can upload file attachments but I prefer the RC interface ;) The htaccess settings are the same for RC and Horde (in respect to the above values).

Thanks
Title: Re: File upload failed.
Post by: JohnDoh on February 02, 2017, 07:12:12 AM
are you running any roundcube plugins? if so make sure you try with them all disabled.
Title: Re: File upload failed.
Post by: m_rc_b on June 28, 2017, 12:50:06 PM
Just in case anyone else has the "File upload failed" problem, my own experiences might help a percentage of cases.

It is not just the PHP file (.../attachments.inc) that uses the (English language) message "File upload failed", it is also used in app.min.js, which then lead me down the path I followed. Using Firefox debugger, I could see when and where the message was occurring and then found that it was that a JavaScript failure to access an iframe document (see https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Errors/Property_access_denied?utm_source=mozilla&utm_medium=firefox-console-errors&utm_campaign=default). After further investigation, I see that in my web server's configuration file for SSL, it was FORCING an HTTP header (X-FRAME-OPTIONS) always to DENY - even adding in "sameorigin" in to the RoundCube config options only caused the header to become invalid: DENY, sameorigin.  So, I ended up amending the web server's config file, restart the web server, reload the RoundCube page and problem solved.

Hope this helps.
Title: Re: File upload failed.
Post by: ljbomir on July 11, 2017, 12:27:42 PM
@adb101, Did you manage resolving your problem ?

I've been struggling to find out any clue on the same.

X-FRAME-OPTIONS is correctly set to "sameorigin" in the header. I have tried also disabling it without any results.

The biggest pain is that I can not get a single error message in /var/log/roundcube/, neither in /var/log/apache2/domain-name/, neither in /var/log/apache2/ !

I can only upload some small ascii files which are being uploaded in /var/lib/roundcube/temp folder.
For all the rest it fails.

Any help on enabling logging for this specific upload action will be much appreciated.

Thanks
Lyubo
Title: Re: File upload failed.
Post by: SKaero on July 11, 2017, 01:26:45 PM
If you create a file with the following:
Code: [Select]
<?php
phpinfo
();

What is returned?
Title: Re: File upload failed.
Post by: adb101 on July 31, 2017, 09:40:13 AM
Nope - not had any luck getting attachments to work. Just tried version 1.3.0 and still get the same error (so still having to log into Horde to add attachments to emails!).  :(

Obviously I get loads back from the phpinfo() call - what sections in particular are of interest?
Title: Re: File upload failed.
Post by: SKaero on July 31, 2017, 10:24:56 AM
upload_max_filesize and max_post_size
Title: Re: File upload failed.
Post by: adb101 on August 01, 2017, 05:06:04 AM
upload_max_filesize   20M   20M
post_max_size                   25M   25M

Just to note: file uploads/attachments work fine through Horde running on the same server (same apache/php etc)
Title: Re: File upload failed.
Post by: SKaero on August 01, 2017, 09:50:47 AM
Whats in the Roundcube error log?
Title: Re: File upload failed.
Post by: adb101 on August 02, 2017, 11:01:11 AM
I have "$config['debug_level'] = 1;" set but no "error" log file being generated in "logs" directory.

I have set the debug for the imap etc and they are generating logs in the "logs" directory, but don't contain any errors.

The only error is the one generated by the UI (see attached). Nothing in /var/log either.
Title: Re: File upload failed.
Post by: SKaero on August 02, 2017, 11:31:52 AM
Do you have any logs that relate to the upload in your main web server logs?
Title: Re: File upload failed.
Post by: adb101 on August 04, 2017, 11:39:46 AM
No, nothing in OS or Apache logs. Ideally I'd like to know what the function is trying to do, but my PHP skillz are not sufficiently in-depth to work it out. Obviously RC can write to the directory (as it creates a zero length file), it's just the writing of the data to that file that seems to fail. Clearly it's a configuration issue with the server but until I can work out what the file upload function is failing on exactly I'm a bit stuck as to what's going on (could strace it I guess...)  :-\
Title: Re: File upload failed.
Post by: phil2k on March 15, 2018, 10:44:15 PM
Hi,

I had the same issue  :o !
And after loosing some long hours doing some debuging on every side of the problem (browser side: JS, Networking, etc., server side: web-server, php-config, memcache config, Filesystem: owners, permissions, disk-space, etc. ::) ) :( , I found out that somehow the exec_hook "attachment_upload" is called twice (don't know why  :o because I'm not a dev-guy for roundcube, neither PHP OOP  :-[ ) :
 1. first call somehow alters(empty) the value of args['path'] (loosing the original uploaded file), than
 2. at the second exec_hook calls the function upload() (from plugins/filesystem_attachments/filesystem_attachments.php), having this argument altered(emptied) already because as comments from above definition of the function exec_hook() says:
Quote
The (probably) altered hook arguments
, this already happened ( that "probably" should be emphasised ;D ) :

./program/lib/Roundcube/rcube_plugin_api.php :
 
Code: [Select]
   /**
     * Triggers a plugin hook.
     * This is called from the application and executes all registered handlers
     *
     * @param string $hook Hook name
     * @param array $args Named arguments (key->value pairs)
     *
     * @return array The (probably) altered hook arguments
     */
    public function exec_hook($hook, $args = array())
    {
        if (!is_array($args)) {
            $args = array('arg' => $args);
        }

        // TODO: avoid recursion by checking in_array($hook, $this->exec_stack) ?

        $args += array('abort' => false);
        array_push($this->exec_stack, $hook);

        // Use for loop here, so handlers added in the hook will be executed too
        for ($i = 0; $i < count($this->handlers[$hook]); $i++) {
            if ($hook=="attachment_upload") error_log("debug: exec_hook(".$hook."): i=$i, args = ".print_r($args, true));
            $ret = call_user_func($this->handlers[$hook][$i], $args);
            if ($hook=="attachment_upload") error_log("debug: exec_hook(".$hook."): i=$i, ret = ".print_r($ret, true));
            if ($ret && is_array($ret)) {
                $args = $ret + $args;
            }

            if ($args['break']) {
                break;
            }
        }

        array_pop($this->exec_stack);
        return $args;
    }

The temporary fix that it seems to work for the moment for me, was to add a backup value stored as key "uploaded_full_path" (beside existing "path") in the constructed args array from program/steps/mail/attachments.inc at line 123, like below:
Code: [Select]
// handle file(s) upload
if (is_array($_FILES['_attachments']['tmp_name'])) {
    $multiple = count($_FILES['_attachments']['tmp_name']) > 1;

    foreach ($_FILES['_attachments']['tmp_name'] as $i => $filepath) {
        // Process uploaded attachment if there is no error
        $err = $_FILES['_attachments']['error'][$i];

        if (!$err) {
            $attachment = $RCMAIL->plugins->exec_hook('attachment_upload', array(
                'uploaded_full_path' => $filepath,
                'path' => $filepath,
                'size' => $_FILES['_attachments']['size'][$i],
                'name' => $_FILES['_attachments']['name'][$i],
                'mimetype' => rcube_mime::file_content_type($filepath, $_FILES['_attachments']['name'][$i], $_FILES['_attachments']['type'][$i]),
                'group' => $COMPOSE_ID,
            ));
        }

than to change the used value by move_uploaded_file() in upload() function in plugins/filesystem_attachments/filesystem_attachments.php, from $args['path'] to $args['uploaded_full_path'], like below:
Code: [Select]
    /**
     * Save a newly uploaded attachment
     */
    function upload($args)
    {
        $args['status'] = false;
        $group  = $args['group'];
        $rcmail = rcube::get_instance();

        // use common temp dir for file uploads
        $temp_dir = $rcmail->config->get('temp_dir');
        $tmpfname = tempnam($temp_dir, 'rcmAttmnt');

        if (move_uploaded_file($args['uploaded_full_path'], $tmpfname) && file_exists($tmpfname)) {
            $args['id']     = $this->file_id();
            $args['path']   = $tmpfname;
            $args['status'] = true;
            @chmod($tmpfname, 0600);  // set correct permissions (#1488996)

            // Note the file for later cleanup
            $_SESSION['plugins']['filesystem_attachments'][$group][$args['id']] = $tmpfname;
        }

        return $args;
    }

Proving logs (after I've added some debugs):
Code: [Select]
[16-Mar-2018 04:06:46 Europe/Bucharest] debug: Array
(
    [_attachments] => Array
        (
            [name] => Array
                (
                    [0] => IMG-20180315-WA0026.jpg
                )

            [type] => Array
                (
                    [0] => image/jpeg
                )

            [tmp_name] => Array
                (
                    [0] => /var/lib/roundcube/uploads/php9MztBt
                )

            [error] => Array
                (
                    [0] => 0
                )

            [size] => Array
                (
                    [0] => 81506
                )

        )

)

[16-Mar-2018 04:06:46 Europe/Bucharest] debug: i=0 filepath=/var/lib/roundcube/uploads/php9MztBt err=0
[16-Mar-2018 04:06:46 Europe/Bucharest] debug: exec_hook(attachment_upload): i=0, args = Array
(
    [uploaded_full_path] => /var/lib/roundcube/uploads/php9MztBt
    [path] => /var/lib/roundcube/uploads/php9MztBt
    [size] => 81506
    [name] => IMG-20180315-WA0026.jpg
    [mimetype] => image/jpeg
    [group] => 1606078455aab1a19cacae
    [abort] =>
)

[16-Mar-2018 04:06:46 Europe/Bucharest] debug: exec_hook(attachment_upload): i=0, ret = Array
(
    [uploaded_full_path] => /var/lib/roundcube/uploads/php9MztBt
    [path] =>
    [size] => 81506
    [name] => IMG-20180315-WA0026.jpg
    [mimetype] => image/jpeg
    [group] => 1606078455aab1a19cacae
    [abort] =>
    [status] => 1
    [id] => 1606078455aab1a19cacae71e3f4515469c99be6682965159bc06a
)
[16-Mar-2018 04:06:46 Europe/Bucharest] debug: exec_hook(attachment_upload): i=1, args = Array
(
    [uploaded_full_path] => /var/lib/roundcube/uploads/php9MztBt
    [path] =>
    [size] => 81506
    [name] => IMG-20180315-WA0026.jpg
    [mimetype] => image/jpeg
    [group] => 1606078455aab1a19cacae
    [abort] =>
    [status] => 1
    [id] => 1606078455aab1a19cacae71e3f4515469c99be6682965159bc06a
)

[16-Mar-2018 04:06:46 Europe/Bucharest] debug: upload(args= Array
(
    [uploaded_full_path] => /var/lib/roundcube/uploads/php9MztBt
    [path] =>
    [size] => 81506
    [name] => IMG-20180315-WA0026.jpg
    [mimetype] => image/jpeg
    [group] => 1606078455aab1a19cacae
    [abort] =>
    [status] =>
    [id] => 1606078455aab1a19cacae71e3f4515469c99be6682965159bc06a
)
 )
[16-Mar-2018 04:06:46 Europe/Bucharest] debug: exec_hook(attachment_upload): i=1, ret = Array
(
    [uploaded_full_path] => /var/lib/roundcube/uploads/php9MztBt
    [path] => /var/lib/roundcube/temp/rcmAttmnt9y92Px
    [size] => 81506
    [name] => IMG-20180315-WA0026.jpg
    [mimetype] => image/jpeg
    [group] => 1606078455aab1a19cacae
    [abort] =>
    [status] => 1
    [id] => 1181521166006064803400
)


P.S.: My Roundcube version is the latest from Ubuntu LTS 16.04: roundcube-1.2~beta+dfsg.1-0ubuntu1.
Title: Re: File upload failed.
Post by: SKaero on March 16, 2018, 02:29:47 AM
You running a beta version of Roundcube from 2015 I highly recommend that you upgrade because your version is very out of date.
Title: Re: File upload failed.
Post by: phil2k on March 17, 2018, 06:18:29 AM
Hi SKaero,

Thank you for your answer  8)

Unfortunately the version "1.2~beta+dfsg.1-0ubuntu1" is the latest version offered by the last stable version of Ubuntu LTS sever (16.04)  :(
I know, I have the option to either install the last roundcube version from the sources, or try another new Ubuntu version (non LTS) like 17.10.1 (artful) that comes with rouncube version 1.3.0+dfsg.1-1  ::)
But that server is a production server with hundreds of mailboxes, so I guess I'll have to try the first approach  :-\

P.S.: Good lesson over howto not losing time doing debugging, but rather finding newer versions in the "wild"  :-[
Title: Re: File upload failed.
Post by: nikitasius on April 08, 2018, 05:42:56 PM
Hi SKaero,

Thank you for your answer  8)

Unfortunately the version "1.2~beta+dfsg.1-0ubuntu1" is the latest version offered by the last stable version of Ubuntu LTS sever (16.04)  :(
I know, I have the option to either install the last roundcube version from the sources, or try another new Ubuntu version (non LTS) like 17.10.1 (artful) that comes with rouncube version 1.3.0+dfsg.1-1  ::)
But that server is a production server with hundreds of mailboxes, so I guess I'll have to try the first approach  :-\

P.S.: Good lesson over howto not losing time doing debugging, but rather finding newer versions in the "wild"  :-[

I had same problem on my server with attachements.  My solution very simple:

In a config file keep/add in active plugins filesystem_attachments and remove database_attachments (if you had it).
They can't work together and attachements needs at least a filesystem_attachments.
And all will work like a charm :)

I tried all, even setting basedir, and fullfilling upload* php variables. But finally disabling database_attachements let my roundcube work and it doesn't create empty files.

Tested on 1.3.3, 1.3.4 and on actual 1.3.5.