Author Topic: markasjunk2 sa-learn problem  (Read 18463 times)

Offline crusher

  • Jr. Member
  • **
  • Posts: 24
    • http://crusher.untergrund.net
markasjunk2 sa-learn problem
« on: December 12, 2009, 05:50:43 AM »
Hello!

I already posted my problem in the german subforum.
But it seems nobody can help me, so I want to try it here.

I included markasjunk2 and configured it for my needs. Marking as Spam/Ham is working well. Also the messages got moved into Spam/Inbox.
But sa-learn doesn't seem to work out of RC.

Here's my config.inc.php:

$rcmail_config['markasjunk2_learning_driver'] = 'cmd_learn';

// Mark messages as read when reporting them as spam
$rcmail_config['markasjunk2_read_spam'] = true;

// Mark messages as unread when reporting them as ham
$rcmail_config['markasjunk2_unread_ham'] = false;

// When reporting a message as ham, if the orginial message is attached (like with SpamAssassin reports),
// detach the orginial message and move that to the INBOX, deleteing the spam report
$rcmail_config['markasjunk2_detach_ham'] = true;

// Add flag to messages marked as spam (flag will be removed when marking as ham)
// If you do not want to use message flags set this to null
$rcmail_config['markasjunk2_spam_flag'] = 'Junk';

// Add flag to messages marked as ham (flag will be removed when marking as spam)
// If you do not want to use message flags set this to null
$rcmail_config['markasjunk2_ham_flag'] = null;

// Write output from spam/ham commands to the log for debug
$rcmail_config['markasjunk2_debug'] = false;

// cmd_learn Driver options
// ------------------------
// The command used to learn that a message is spam
// The command can contain the following macros that will be expanded as follows:
//      %u is replaced with the username (from the session info)
//      %l is replaced with the local part (before the @) of the username (from the session info)
//      %d is replaced with the domain part (after the @) of the username (from the session info)
//      %f is replaced with the path to the message file (THIS SHOULD ALWAYS BE PRESENT)
// If you do not want run the command set this to null
$rcmail_config['markasjunk2_spam_cmd'] = '/usr/bin/sa-learn --spam --prefs-file=/var/qmail/mailnames/%d/%l/.spamassassin/user_prefs %f';

// The command used to learn that a message is ham
// The command can contain the following macros that will be expanded as follows:
//      %u is replaced with the username (from the session info)
//      %l is replaced with the local part (before the @) of the username (from the session info)
//      %d is replaced with the domain part (after the @) of the username (from the session info)
//      %f is replaced with the path to the message file (THIS SHOULD ALWAYS BE PRESENT)
// If you do not want run the command set this to null
$rcmail_config['markasjunk2_ham_cmd'] = '/usr/bin/sa-learn --ham --prefs-file=/var/qmail/mailnames/%d/%l/.spamassassin/user_prefs %f';  


Manually learning via the shell with the same command is working correctly.
I checked it with "sa-learn --dump magic". The exact amount of learned spam/hams is added to nspams/nhams.
It seems markasjunk2 doesn't execute sa-learn or there is a permission problem.
I don't know where to search for the problem as manually learning is working.

Any hint somebody?

Offline JohnDoh

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 2,044
markasjunk2 sa-learn problem
« Reply #1 on: December 12, 2009, 07:23:44 AM »
try setting $rcmail_config['markasjunk2_debug'] to true in your config then take a look in the logs folder, the command that was run and the result should be logged. may be that will help.
Roundcube Plugins: Contextmenu, SpamAssassin Prefs, and more…

Offline crusher

  • Jr. Member
  • **
  • Posts: 24
    • http://crusher.untergrund.net
markasjunk2 sa-learn problem
« Reply #2 on: December 12, 2009, 10:02:13 AM »
this is in the log

[12-Dec-2009 15:54:27 +0100]: /usr/bin/sa-learn --spam --prefs-file=/var/qmail/mailnames/mydomain/user/.spamassassin/user_prefs /usr/share/roundcube/temp/rcmSALearnlihEHn
[12-Dec-2009 15:54:27 +0100]: array (
  
=> 'Learned tokens from 1 message(s) (1 message(s) examined)',


Seems learned correctly.
But examining via "sa-learn --dump magic" the messages are not added to nspams/nhams :(

Offline JohnDoh

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 2,044
markasjunk2 sa-learn problem
« Reply #3 on: December 13, 2009, 04:44:49 AM »
Hmmm that looks right to me but then I don’t know much about sa-learn. Perhaps there is some one who is more familial with it who can help.
Roundcube Plugins: Contextmenu, SpamAssassin Prefs, and more…

Offline crusher

  • Jr. Member
  • **
  • Posts: 24
    • http://crusher.untergrund.net
markasjunk2 sa-learn problem
« Reply #4 on: December 17, 2009, 11:46:09 AM »
I did further testing to narrow down the problem.
Some more infos to my system: I'm on OpenSuse 11.1 + Plesk 9.2.3 + Qmail + SA setted up to use individual Bayes-DB for each user.
(DB is in  /var/qmail/mailnames/domain/user/.spamassassin)

I altered config.inc.php to let markasjunk2 execute following command to see if the executing out of markasjunk2 is working correctly itself:

$rcmail_config['markasjunk2_spam_cmd'] = 'echo "Test" > /var/qmail/mailnames/%d/%l/.spamassassin/debug.txt';

No debug.txt was written.
OK, probably a permission problem I thought and changed the command into the following one knowing that RC is allowed to write there:

$rcmail_config['markasjunk2_spam_cmd'] = 'echo "Test" > /usr/share/roundcube/logs/debug.txt';

...et voila! debug.txt is there.

Next step I did chown wwwrun:www the .spamassassin dir for my test account and gave 777 rights to it to assure it's writeable for apache.
But still he doesn't write debug.txt :(

So, problem seems to be that the .spamassassin dir isn't writeable for apache. Do you have any idea what I have to change?
What confuses me is that he can write in the logs dir but not in .spamassassin even with the same owner/group and 777 rights.

//Edit:
Just tried to add wwwrun to the group "popuser" which has access rights to the .spamassassin dir. That doesn't help neither :(
Any idea somebody? I think this is a basic Linux problem now. Unfortunalety I don't have that much Linux knowledge...

//Edit 2:
I added following line in /etc/sudeors with visudo:

#wwwrun ALL=NOPASSWD: /usr/bin/sa-learn
#wwwrun ALL=(root) NOPASSWD: /usr/bin/sa-learn
wwwrun ALL=(ALLNOPASSWD: /usr/bin/sa-learn


I also tried the outcommented ones as I'm not sure about the right syntax.
It all doesn't work.

/var/qmail/mailnames/mydomain/user/.spamassassin looks like this:

drwxr
-xr-x  2 popuser popuser 4096 18. Dez 23:10 .spamassassin/


the files within:

drwxr
-xr-x 2 popuser popuser  4096 18. Dez 23:10 ./
drwx------ 5 popuser popuser  4096 18. Dez 12:43 ../
-
rw------- 1 popuser popuser 12288 18. Dez 12:37 auto-whitelist
-rw------- 1 popuser popuser 12288 18. Dez 22:21 bayes_seen
-rw------- 1 popuser popuser 12288 18. Dez 22:21 bayes_toks
-rw------- 1 popuser popuser    44 18. Dez 22:11 user_prefs


groups wwwrun:

wwwrun 
www popuser psaserv


What could still be the problem??
« Last Edit: December 18, 2009, 05:25:23 PM by crusher »

Offline shashilx

  • Newbie
  • *
  • Posts: 2
markasjunk2 sa-learn problem
« Reply #5 on: March 20, 2010, 09:04:15 PM »
Also have permission problems.. :( Using Archlinux as my Linux distro. After some mind storm I've tried insert this command in config

Code: [Select]
$rcmail_config['markasjunk2_spam_cmd'] = 'ls -la /srv/http/mail/temp > /srv/http/mail/temp/list';


And then see what permissions to on-the-fly-temp-file

Code: [Select]
drwxr-xr-x  2 http http  160 Mar 21 02:48 .
drwxr-xr-x 10 root root  472 Mar 16 00:34 ..
-rw-r--r--  1 root root   31 Sep 25  2005 .htaccess
-rw-r--r--  1 http http    0 Mar 21 02:48 list
-rw-------  1 http http 4984 Mar 21 02:48 rcmSALearnRTDiij


And now I think that command

Code: [Select]
/usr/bin/perlbin/vendor/sa-learn --spam --username=shashilx /srv/http/mail/temp/rcmSALearnRTDiij

just don't have permissions to access on-the-fly-temp-file.
Correct me if I'm wrong?

p.s. my debug output
Code: [Select]
roundcube: [21-Mar-2010 02:19:48 +0200]: /usr/bin/perlbin/vendor/sa-learn --spam --username=shashilx /srv/http/mail/temp/rcmSALearnVEMlVR
roundcube: [21-Mar-2010 02:19:48 +0200]: array (
0: => 'Learned tokens from 0 message(s) (1 message(s) examined)',
):

Offline shashilx

  • Newbie
  • *
  • Posts: 2
markasjunk2 sa-learn problem
« Reply #6 on: March 20, 2010, 10:12:51 PM »
Also tried to change command to
Code: [Select]
$rcmail_config['markasjunk2_spam_cmd'] = '/usr/bin/perlbin/vendor/sa-learn --spam --username=%u %f 2>&1';


and got strange errors :(

Code: [Select]
Mar 21 04:07:21 shashilx roundcube: [21-Mar-2010 04:07:21 +0200]: /usr/bin/perlbin/vendor/sa-learn --spam --username=shashilx /srv/http/mail/temp/rcmSALearnZzpuJX 2>&1
Mar 21 04:07:21 shashilx roundcube: [21-Mar-2010 04:07:21 +0200]: array (
Mar 21 04:07:21 shashilx 0: => 'config: path "/var/lib/spamassassin/3.003000" is inaccessible: Permission denied',
Mar 21 04:07:21 shashilx 1: => 'config: path "/var/lib/spamassassin/3.003000/languages" is inaccessible: Permission denied',
Mar 21 04:07:21 shashilx 2: => 'bayes: expire_old_tokens: locker: safe_lock: cannot create tmp lockfile //.spamassassin/bayes.lock.shashilx.22842 for //.spamassassin/bayes.lock: No such file or directory',
Mar 21 04:07:21 shashilx 3: => 'plugin: eval failed: bayes: (in learn) locker: safe_lock: cannot create tmp lockfile //.spamassassin/bayes.lock.shashilx.22842 for //.spamassassin/bayes.lock: No such file or directory',
Mar 21 04:07:21 shashilx 4: => 'ERROR: the Bayes learn function returned an error, please re-run with -D for more information at /usr/bin/perlbin/vendor/sa-learn line 493.',
Mar 21 04:07:21 shashilx 5: => 'Learned tokens from 0 message(s) (1 message(s) examined)',


How can I fix this so spamassassin will learn spam/notspam?

Offline Martian

  • Jr. Member
  • **
  • Posts: 35
markasjunk2 sa-learn problem
« Reply #7 on: March 22, 2010, 05:22:53 PM »
You say that you added the wwwrun user to the popuser group but per your post none of the files in .spamassassin have group read or write permissions!!!  Try a chmod g+r on the .spamassassin folder and see if that helps.

Martian

Offline Martian

  • Jr. Member
  • **
  • Posts: 35
markasjunk2 sa-learn problem
« Reply #8 on: March 25, 2010, 12:47:03 AM »
This thread inspired me to look into the markasjunk2 plugin.

First, I apologize for my previous post as after looking into this plugin further I obviously had no idea what I was talking about...:confused:

I'm also running Arch Linux and can confirm shashilx's observations.

For some reason though, I can't figure out how to get the logging for markasjunk2 to work.  I've tried adding "$rcmail_config['markasjunk2_debug'] = TRUE;" to both the config.inc.php in the markasjunk2 folder as well as the main.inc.php file in the config folder but in either case there was nothing new in the logs folder.

Did anyone get any further on this? or could someone as least set me straight on how to get the logging to work (I'm sure I'm doing something stupid) so I can play with it some more.

Thanks,

Martian

Offline JohnDoh

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 2,044
markasjunk2 sa-learn problem
« Reply #9 on: March 25, 2010, 04:31:17 AM »
you had it right, just set $rcmail_config['markasjunk2_debug'] in the plugin's config file and the information is written to markasjunk2 log file (which assuming you have a default setup is in /logs/). Make sure the user the web server runs as has permission to write to the logs folder (this should have been done as part of the rc installation)
Roundcube Plugins: Contextmenu, SpamAssassin Prefs, and more…

Offline Martian

  • Jr. Member
  • **
  • Posts: 35
markasjunk2 sa-learn problem
« Reply #10 on: March 25, 2010, 10:03:51 AM »
Thanks for the reply JohnDoh!

I don't know what I'm doing wrong.  I put:
Code: [Select]
$rcmail_config['markasjunk2_debug'] = true;
right after the:
Code: [Select]
$rcmail_config['markasjunk2_learning_driver'] = cmd_learn;
line in my config.inc.php file.

Code: [Select]
$rcmail_config['markasjunk2_spam_cmd'] = 'ls -la /srv/http/roundcube/temp > /srv/http/roundcube/temp/list';
writes the "list" file to /srv/http/roundcube/temp/ so I know it's working.


My permissions on logs and temp are:
Code: [Select]
Arch roundcube $ ls -la | grep log
drwxr-xr-x  2 http http  4096 Jan 31 22:14 logs
Arch roundcube $ ls -la | grep temp
drwxr-xr-x  2 http http  4096 Mar 25 09:50 temp

http is my apache user (Arch Linux default) which can write the "list" file to temp which has the same permissions as logs.

but no markasjunk2 log file:
Code: [Select]
Arch logs $ ls -la
total 140
drwxr-xr-x  2 http http   4096 Jan 31 22:14 .
drwxr-xr-x 11 root root   4096 Jan 31 21:41 ..
-rw-r--r--  1 http http 126142 Mar 25 09:47 errors
-rw-r--r--  1 root root     31 Sep 25  2005 .htaccess


I even completely removed the markasjunk2 plugin folder and restarted from scratch with the same results.  So frustrating...

Martian

Offline JohnDoh

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 2,044
markasjunk2 sa-learn problem
« Reply #11 on: March 25, 2010, 12:09:48 PM »
when you say you put $rcmail_config['markasjunk2_debug'] = true; right after the markasjunk2_learning_driver, did you remove the line that is in there further down which has it set to false (by default)?
Roundcube Plugins: Contextmenu, SpamAssassin Prefs, and more…

Offline Martian

  • Jr. Member
  • **
  • Posts: 35
markasjunk2 sa-learn problem
« Reply #12 on: March 25, 2010, 01:00:24 PM »
Quote from: JohnDoh;26326
when you say you put $rcmail_config['markasjunk2_debug'] = true; right after the markasjunk2_learning_driver, did you remove the line that is in there further down which has it set to false (by default)?

DOH!

I knew it was some stupid little thing I was doing wrong but :eek: - I didn't think it was that stupid...

Thanks for pointing our the error of my ways.  I've had a good laugh at my own expense and recommend you do the same :)

Thanks again!

Martian
« Last Edit: March 26, 2010, 04:44:44 AM by Julius Caesar »

Offline CaptSaltyJack

  • Jr. Member
  • **
  • Posts: 11
Re: markasjunk2 sa-learn problem
« Reply #13 on: April 20, 2013, 11:43:09 PM »
Well?? Did anyone ever solve this problem? I'm having the same issue. I can mark junk all day from RoundCube, and the log says it learned it, but the nspam count never goes up.. and if I use sa-learn in the shell, then it works.

Offline aeonfinis

  • Newbie
  • *
  • Posts: 1
Re: markasjunk2 sa-learn problem
« Reply #14 on: June 28, 2013, 06:03:06 PM »
I'm having the same problem and I think the --username option of sa-learn does not work the way it should.
If you 'su - www-data' and do a 'sa-learn --dump magic' you will probably be supprised to learn that that user is being updated, not the user you are logged into as on roundcube.
The webserver is running as www-data (on debian) and sa-learn either ignores the --username flag or it doesn't use it to change the home path.