Roundcube Community Forum

Miscellaneous => Roundcube Discussion => Topic started by: hipsterdoofus on December 15, 2009, 03:41:11 PM

Title: Filtering
Post by: hipsterdoofus on December 15, 2009, 03:41:11 PM
Hello all

I'm a fairly new user here - I believe that we are very near being able to switch our organization to roundcube.  I do have an issue - our provider runs spam assassin at the server level.  In order to divert items to a spam folder, we simply set up a filter in our client based on a ":  X-Spam-Level" term in the header.  So it looks like I do not need the spam assassin prefs plugin (since our provider already does this) - is there a way to allow users to configure this type of filtering themselves?
Title: Filtering
Post by: rosali on December 16, 2009, 01:06:26 AM
Roundcube does not support filtering on client side. You would have to write a plugin for that. Spam assassin prefs plugin is an interface to Spamassassin. It does not meet your needs.
Title: Filtering
Post by: hipsterdoofus on December 16, 2009, 08:52:37 AM
Quote from: rosali;23987
Roundcube does not support filtering on client side. You would have to write a plugin for that. Spam assassin prefs plugin is an interface to Spamassassin. It does not meet your needs.


Will it ever support this?  Other than the ability to have groups and global groups it is the only feature that it is missing.  Seems a shame that there is a plugin for some more complex things but nothing as simple as some basic filtering :(
Title: Filtering
Post by: SKaero on December 16, 2009, 09:46:59 AM
RoundCube can't support client side filtering, ever time a client login it wound have to sort all of the new emails into folders. It would make the server use up to many resources. It has to be done at the server lever as the emails come it, however I am confused as to what you are trying to do. You have Spam Assassin witch does the filtering do you just want your users to be able to pick the spam level they consider to be spam?

P.S. Contact groups are planed for Roundcube 0.4
Title: Filtering
Post by: hipsterdoofus on December 16, 2009, 09:53:12 AM
Quote from: skaero;24003
RoundCube can't support client side filtering, ever time a client login it wound have to sort all of the new emails into folders. It would make the server use up to many resources. It has to be done at the server lever as the emails come it, however I am confused as to what you are trying to do. You have Spam Assassin witch does the filtering do you just want your users to be able to pick the spam level they consider to be spam?

P.S. Contact groups are planed for Roundcube 0.4


Ok - well my understanding is this - currently on our mail server, they run spam assassin - I believe what it does is assign a ranking to a message and puts that in the header. It does not prevent users from receiving that mail.  When they receive the mail, based on a filter they have set up, it will put mail into a spam folder - and yes, they configure what spam level goes into the spam folder.

I'm very open to another option, but we don't own our mailserver, our provider does, so I realize we are limited.
Title: Filtering
Post by: SKaero on December 16, 2009, 09:57:12 AM
To me it sounds like the Spam Assassin plugin would do what you want. It allows users pick the spam level they feel is spam. I could be wrong (I am not using it) but it would hurt to try it and see if it does what you want it to.
Title: Filtering
Post by: hipsterdoofus on December 16, 2009, 10:02:42 AM
Quote from: skaero;24005
To me it sounds like the Spam Assassin plugin would do what you want. It allows users pick the spam level they feel is spam. I could be wrong (I am not using it) but it would hurt to try it and see if it does what you want it to.


If it is JohnDoh's spam assassin user prefs plugin - I'm not sure how we would do it since it requires knowledge of the database information for spam assassin.
Title: Filtering
Post by: rosali on December 16, 2009, 10:29:36 AM
As skaero said this has to be done by server side filtering. For dovecot there is the managesieve plugin and for hMailServer there is something in the pipeline coded by me (nearly finished).

Spamassassin just tags messages. JohnDoh's spam assassin user prefs plugin just configures Spamassassin on a per user level. But Spamassassin will not filter messages. It adds headers based on the users prefs to the message. Nothing more or less.

The next step is to get server side filtering as a part of the IMAP server. As said something like the built in hMailserver filters or Dovecot's sieve module. This is the part where the message is scanned for specific headers before the message is moved by the imap server to the user's INBOX or another mailbox like Junk.
Title: Filtering
Post by: hipsterdoofus on December 16, 2009, 10:41:19 AM
Quote from: rosali;24007
As skaero said this has to be done by server side filtering. For dovecot there is the managesieve plugin and for hMailServer there is something in the pipeline coded by me (nearly finished).

Spamassassin just tags messages. JohnDoh's spam assassin user prefs plugin just configures Spamassassin on a per user level. But Spamassassin will not filter messages. It adds headers based on the users prefs to the message. Nothing more or less.

The next step is to get server side filtering as a part of the IMAP server. As said something like the built in hMailserver filters or Dovecot's sieve module. This is the part where the message is scanned for specific headers before the message is moved by the imap server to the user's INBOX or another mailbox like Junk.


Ok - I'm still not sure I understand why it can't be done - please forgive my ignorance - but you can manually receive a message in your inbox and move it to a folder - isn't filtering just automating that process based on a criteria?  It seems that your filter settings would simply check your inbox at the same time it checks for new mail and filter anything new accordingly.
Title: Filtering
Post by: SKaero on December 16, 2009, 10:58:04 AM
If 4 users get 100 emails over a weekend and all login at 8:00AM on Monday RoundCube has to look at and move each of the emails into folders before the user can get to there inbox. Not only does the user have to wait for RoundCube to get done but this would also put a big load on your server as its asked to sort 400 email at once.
How it needs to work is the mail server gets the email gives it to the filter and the filter checks the users preferences and then moves it to the right folder so when the user login to RoundCube the email is already in the folder.
Also if a user checked there email using Thunderbird or a iPhone, it wound get the mail from the server and not go thought RoundCube at all so no filtering would happen.
You said that you have a filter on your current mail client how does that work? Could you port the code to RoundCube as a plugin?
Title: Filtering
Post by: hipsterdoofus on December 16, 2009, 11:08:40 AM
Quote from: skaero;24009

You said that you have a filter on your current mail client how does that work? Could you port the code to RoundCube as a plugin?


Ha - unfortunately I am no coder :(  Our current client is a desktop client and essentially users can set up rules to check the spam level assigned by spam assassin  - and move the messages based on their preferences.  It is an individual configuration - for example, I have no filtering set up - but some users do have it set up - it just checks the header for the level that spam assassin assigned to it.
Title: Filtering
Post by: rosali on December 16, 2009, 11:10:08 AM
Skaero, I think he is referring to filters of desktop clients like Outlook (Express). But in this point we can't compare the capabilities of a desktop client with those of a webmail client where everything is done server side. In a reasonable way this could be only done by a hugh javascript overhead on client side's browser. So, I agree with you that Roundcube should not focus on that.
Title: Filtering
Post by: hipsterdoofus on December 16, 2009, 11:21:42 AM
Quote from: rosali;24011
Skaero, I think he is referring to filters of desktop clients like Outlook (Express). But in this point we can't compare the capabilities of a desktop client with those of a webmail client where everything is done server side. In a reasonable way this could be only done by a hugh javascript overhead on client side's browser. So, I agree with you that Roundcube should not focus on that.


I understand that the point of roundcube is to be pretty basic - but I would say any desktop client would be capable of filtering - if roundcube is to be a replacement for a desktop client, I would think it would need some capability of filtering as well?
Title: Filtering
Post by: SKaero on December 16, 2009, 11:32:12 AM
I was thinking of web mail not a desktop mail client. If you want to have filters with any web mail client it has to be done by the mail sever I don't see any way that RoundCube or any other web mail could do filtering without taking up to many resources.
Title: Filtering
Post by: hipsterdoofus on December 16, 2009, 11:36:45 AM
Quote from: skaero;24014
I was thinking of web mail not a desktop mail client. If you want to have filters with any web mail client it has to be done by the mail sever I don't see any way that RoundCube or any other web mail could do filtering without taking up to many resources.



Yeah I mean I guess I see what you mean - unless it did the filtering whether or not the person was logged onto roundcube at the time...spread it out as the messages came in.
Title: Filtering
Post by: SKaero on December 16, 2009, 11:49:18 AM
Right witch is what the server would/should do as they come in. RoundCube is only active when some is using it so it couldn't be done using RoundCube. If you had control or access of the mail server it could be done.
Title: Filtering
Post by: hipsterdoofus on December 16, 2009, 12:09:45 PM
Quote from: skaero;24017
Right witch is what the server would/should do as they come in. RoundCube is only active when some is using it so it couldn't be done using RoundCube. If you had control or access of the mail server it could be done.


Unfortunately I have no control over it.  I do have a question though - you talk about a heavy load from doing this on the webmail client - why is this a problem?  What makes a heavy load on roundcube (which I have installed on a server) worse than a heavy load on the mail server?
Title: Filtering
Post by: SKaero on December 16, 2009, 12:23:04 PM
RoundCube only runs when a user is accessing it, a mail server is always running. When a email that would be filtered comes in to a mail server in automatically makes the file in the right directory. Where as if RoundCube did the filtering it would be checking and moving more then one email. The main problem is the quantity of email the RoundCube would be processing at once, you could wright a script that would check and sort email ever hour but it wouldn't be as seamless as having the mail server do it.
Title: Filtering
Post by: hipsterdoofus on December 16, 2009, 12:30:48 PM
Quote from: skaero;24020
RoundCube only runs when a user is accessing it, a mail server is always running. When a email that would be filtered comes in to a mail server in automatically makes the file in the right directory. Where as if RoundCube did the filtering it would be checking and moving more then one email. The main problem is the quantity of email the RoundCube would be processing at once, you could wright a script that would check and sort email ever hour but it wouldn't be as seamless as having the mail server do it.


Ok - so I don't know much about spam assassin, but here is a question for you (or someone who knows more) - would it be possible to run my own spam assassin server that runs against my provider's mail server?
Title: Filtering
Post by: rosali on December 17, 2009, 08:09:06 AM
No, it isn't. The process is:

- imap server receives mail
- imap server connects to spamd service (spamassassin)
- imap server delivers message to recipient
- client connects to imap server

You have to run your own mail server.
Title: Filtering
Post by: hipsterdoofus on December 17, 2009, 09:51:43 AM
Well thanks for the info - so essentially in order for this to work, RoundCube would have to run all the time (or have some script that runs) and does filtering occasionally.  Still wish it were possible, even if I had to run some timed script.  RoundCube is be best solution for our needs that I've run across besides this.
Title: Filtering
Post by: SKaero on December 17, 2009, 10:59:57 AM
Well you could have a timed script do it, you would need to know the username and password for the user so you could connect to the mail server. You would then set the script to run every half hour or ever hour using a cron job. The script would login to the mail server and check for any new email that had come in after the last time it was checked, if it finds any it would check the spam level and if it was at a level that was considered to be spam it would move the email. You would also need a RoundCube plugin that would allow the user to turn it on and off and set the spam level as well as run the filter script.

This is amusing that email marked as spam would always go into on folder i.e. spam. It would also need to be setup for the number of users you have.
Title: Filtering
Post by: jtrick on December 18, 2009, 01:38:36 AM
To have a RoundCube interface allowing simple editing of filter rules used by the server seems like a pretty fundamental need.  Does anyone know of a plugin or project that is currently focusing on this?  I would be happy to help make this functionality available.  Thanks.
Title: Filtering
Post by: rosali on December 18, 2009, 01:49:48 AM
Well, the problem is, that the way of filtering varies from server to server. So I don't see a way for a general solution.

There is a managesieve plugin for Dovecot. And there will be a plugin for hMailserver soon which allows to filter by 'X-hMailServer-Reason-Score' header field.
Title: Filtering
Post by: jtrick on December 18, 2009, 02:01:12 AM
Ok, I can see what you mean; thanks for the reply.  I found http://www.tehinterweb.co.uk/roundcube/ which has several plugins and was just about to go through some of the code to get a better sense of how they would fit together.  Is there a central resource for RoundCube plugins?
Title: Filtering
Post by: jtrick on December 18, 2009, 02:26:14 AM
Um, sorry to ask the question before checking with more depth...  I clicked your signature link and came across the following list of plugins: Plugin_Repository ? Roundcube Webmail (http://trac.roundcube.net/wiki/Plugin_Repository).  I am new to RoundCube as of about an hour ago, so I hope you can forgive my ignorance here.  Familiarizing myself with everything as I go.