Author Topic: Spam/Ham Report AddOn  (Read 16382 times)

Offline rosali

  • Hero Member
  • *****
  • Posts: 2,533
Spam/Ham Report AddOn
« on: February 02, 2008, 12:59:33 AM »
Here is an AddOn which allows users to report mis-scored messages.

Reported messages will be copied to a spam/ham folder (also to remote folders via ftp).
After revision by the administrator just let run spamassassin sa-learn to train the bayes database of spamassassin.

For further instruction see the code ...

-Roland

Code: [Select]
<?php

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// RoundCube v0.1 - RC2
// 
// AddOn:  Report Spam/Ham - program/steps/mail/report_spam_ham.inc
// Purpose: Copies messages to specific folders $rcmail_config['report_spam_folder'] / $rcmail_config['report_ham_folder']
// or via FTP to remote folders $rcmail_config['report_spam_ftp_folder'] / $rcmail_config['report_ham_ftp_folder']
// Reported messages can be used to train a bayes database (f.e. spamassassin sa-learn)
//
// (c) [url]www.roland-liebl.de[/url]
//
// NO WARRANTY - USE AS IS AT YOUR OWN RISK
//
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//
// config/main.inc.php
//
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
/*
//
// show &quot;move to&quot; selector on message list
$rcmail_config['move_to_select'] = TRUE;

//spam/ham report
$rcmail_config['report_spam_ham'] = TRUE;
$rcmail_config['report_spam_folder'] = &quot;c:/sa/spam/&quot;;
$rcmail_config['report_ham_folder'] = &quot;c:/sa/ham/&quot;;
$rcmail_config['report_spam_ham_ftp'] = TRUE; // if true, folders above will be ignored
$rcmail_config['ftp_server'] = &quot;localhost&quot;;
$rcmail_config['ftp_port'] = 21;
$rcmail_config['ftp_passive_mode'] = TRUE;
$rcmail_config['ftp_user'] = &quot;roundcube&quot;;
$rcmail_config['ftp_pass'] = &quot;pass&quot;;
$rcmail_config['report_spam_ftp_folder'] = &quot;spam/&quot;;
$rcmail_config['report_ham_ftp_folder'] = &quot;ham/&quot;;
*/
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//
// program/include/main.inc
//
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
/*
 // add no-selection option
 if ($type=='select' && $attrib['noselection'])
  $out .= sprintf('<option value=&quot;0&quot;>%s</option>'.&quot;\n&quot;,
          rcube_label('action') . &quot;...&quot;);

 // find me: report spam/ham, message flags, forward as attachment
 if ($type=='select')
  $out .= sprintf('<option value=&quot;-7&quot;>%s</option>'.&quot;\n&quot;,
          rcube_label('forward_as_attachment'));
 if($type == &quot;select&quot; && isset($CONFIG['report_spam_ham']) && $CONFIG['report_spam_ham'] === TRUE){
$out .= &quot;<optgroup label=\&quot;&quot; . rcube_label(&quot;spam_actions&quot;) . &quot;\&quot;>\r\n&quot;;
$out .= &quot;<option value=-1>-- &quot; . rcube_label(&quot;report_Spam&quot;) . &quot; --</option>\r\n&quot;;
$out .= &quot;<option value=-2>-- &quot; . rcube_label(&quot;report_Legit&quot;) . &quot; --</option>\r\n&quot;;
$out .= &quot;</optgroup>\r\n&quot;;

 }

 // NOTICE: This is part of another AddOn message_flags
 if($type == &quot;select&quot; && isset($CONFIG['message_flags']) && $CONFIG['message_flags'] === TRUE){
$out .= &quot;<optgroup label=\&quot;&quot; . rcube_label(&quot;message_flags&quot;) . &quot;\&quot;>\r\n&quot;;
$out .= &quot;<option value=-3>-- &quot; . rcube_label(&quot;mark_as_read&quot;) . &quot; --</option>\r\n&quot;;
$out .= &quot;<option value=-4>-- &quot; . rcube_label(&quot;mark_as_unread&quot;) . &quot; --</option>\r\n&quot;;
$out .= &quot;<option value=-5>-- &quot; . rcube_label(&quot;mark_as_flagged&quot;) . &quot; --</option>\r\n&quot;;
$out .= &quot;<option value=-6>-- &quot; . rcube_label(&quot;mark_as_unflagged&quot;) . &quot; --</option>\r\n&quot;;
$out .= &quot;</optgroup>\r\n&quot;;

 }
 if(
$type == &quot;select&quot; && 
(isset($CONFIG['report_spam_ham']) && $CONFIG['report_spam_ham'] === TRUE ||
 isset($CONFIG['message_flags']) && $CONFIG['message_flags'] === TRUE)
 ){
$out .= &quot;<optgroup label=\&quot;&quot; . rcube_label(&quot;message_location&quot;) . &quot;\&quot;>\r\n&quot;;
$out .= &quot;<option value=0>&quot; . rcube_label($attrib['noselection']) . &quot;</option>\r\n&quot;;
 } 

 // get mailbox list
 $mbox_name = $IMAP->get_mailbox_name();
*/
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//
// index.php
//
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
/*
 if ($_action=='get')
  include('program/steps/mail/get.inc');

 // find me: report spam/ham
 if ($_action=='moveto' && (trim($_POST['_target_mbox']) == -1 || trim($_POST['_target_mbox']) == -2))
  include('program/steps/mail/report_spam_ham.inc');
*/
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//
// skins/default/templates/mail.html
//
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
/*
<roundcube:if condition=&quot;config:move_to_select == true&quot; />
<roundcube:object name=&quot;mailboxlist&quot; type=&quot;select&quot; noSelection=&quot;moveto&quot; maxlength=&quot;25&quot; onchange=&quot;rcmail.command('moveto', this.options[this.selectedIndex].value)&quot; class=&quot;mboxlist&quot; />
<roundcube:endif />
*/
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//
// CODE starts here ...
//
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

if ($uid get_input_value('_uid'RCUBE_INPUT_POST)){

 
$a_uid explode(&quot;,&quot;,$uid);
$err_type '';

  
if(get_input_value('_target_mbox'RCUBE_INPUT_POST) == -1){
$folder $CONFIG['report_spam_folder'];
$ftp_folder $CONFIG['report_spam_ftp_folder'];
  
}
  
else if(get_input_value('_target_mbox'RCUBE_INPUT_POST) == -2){
$folder $CONFIG['report_ham_folder'];
$ftp_folder $CONFIG['report_ham_ftp_folder'];
  
}

 
if(isset($CONFIG['report_spam_ham_ftp']) && $CONFIG['report_spam_ham_ftp'] === TRUE){
$conn_id ftp_connect($CONFIG['ftp_server'], $CONFIG['ftp_port']);
$folder $CONFIG['temp_dir'];
if(isset($CONFIG['ftp_passive_mode']) && $CONFIG['ftp_passive_mode'] === TRUE){
ftp_pasv ($conn_id TRUE);
}
$login_result ftp_login($conn_id$CONFIG['ftp_user'], $CONFIG['ftp_pass']); 
 
}

 
foreach($a_uid as $key => $uid){
  
if(isset($folder) && file_exists($folder)){
$file str_makerand (1616falsefalsefalse) . &quot;.eml&quot;;
while(file_exists($folder $file)){
$file str_makerand (1616falsefalsefalse) . &quot;.eml&quot;;
}
 
$message $IMAP->get_raw_body($uid);
if(!@file_put_contents($folder $file ,$message)){
//failure
$err_type 'error';
}
else{
if(isset($CONFIG['report_spam_ham_ftp']) && $CONFIG['report_spam_ham_ftp'] === TRUE){

if((!$conn_id) || (!$login_result)){
//failure
$err_type 'error';
}
else{
//success
$ftp_file $file;
while(ftp_size($conn_id$ftp_folder.$ftp_file) > -1){
$ftp_file str_makerand (1616falsefalsefalse) . &quot;.eml&quot;;
}

$upload ftp_put($conn_id$ftp_folder.$ftp_file$folder.$fileFTP_BINARY); 

if(!$upload){
//failure
$err_type 'error';
}
else{
//success
}

@unlink($folder.$ftp_file);
}
}
}
 
}
}

if($conn_id){
ftp_close($conn_id);
}
}

if(
$err_type == &quot;&quot;){
$message rcube_label('message(s)_successfully_reported');
$err_type 'confirmation';
}
else{
$message rcube_label('there_was_an_error');
}

$OUTPUT->show_message($message$err_type);

if(
get_input_value('_target_mbox'RCUBE_INPUT_POST) == -1){
$_POST['_target_mbox'] = $CONFIG['junk_mbox'];
}
else if(
get_input_value('_target_mbox'RCUBE_INPUT_POST) == -2){
$_POST['_target_mbox'] = &quot;INBOX&quot;;
}
else{
$_POST['_target_mbox'] = $_POST['_mbox'];
}

?>

Regards,
Rosali
__________________
MyRoundcube Project (commercial)

Offline seansan

  • Jr. Member
  • **
  • Posts: 84
Re: Spam/Ham Report AddOn
« Reply #1 on: February 02, 2008, 05:49:05 PM »
Please log your fix to bugtracker : http://trac.roundcube.net/newticket

Offline jasonar

  • Jr. Member
  • **
  • Posts: 18
Re: Spam/Ham Report AddOn
« Reply #2 on: February 06, 2008, 04:03:24 AM »
I like this alot but I'm having trouble following your install file... what is the "code starts here..." supposed to be, and what am I supposed to do with the rest above that...? I'm lost,help please.

Offline luxerama

  • Newbie
  • *
  • Posts: 8
Re: Spam/Ham Report AddOn
« Reply #3 on: February 06, 2008, 05:51:00 AM »
Create a new file: program/steps/mail/report_spam_ham.inc Paste everything in there. Then modify all files that are mentioned with the code segments below the file names.

Offline rosali

  • Hero Member
  • *****
  • Posts: 2,533
Re: Spam/Ham Report AddOn
« Reply #4 on: February 06, 2008, 11:55:18 AM »
YEP, follow luxerama's advice.

FYI, I will release a my RoundCube with a lot of additional AddOn's soon.

-Roland
Regards,
Rosali
__________________
MyRoundcube Project (commercial)

Offline seansan

  • Jr. Member
  • **
  • Posts: 84
Re: Spam/Ham Report AddOn
« Reply #5 on: February 06, 2008, 02:21:19 PM »

Wouldnt it be great if you moved a message that it was automatically sent to either spam or ham based on move to spam or out of spam folder?

Is there a way in PHP to try if sa-learn command exists in eval or system mode?

program would be.
- test if config option is YES
- test if spamassasin installed
- test if sa-learn is available
- depending on move in or out marks as spam or ham
- pipe the email to sa-learn


Offline rosali

  • Hero Member
  • *****
  • Posts: 2,533
Re: Spam/Ham Report AddOn
« Reply #6 on: February 06, 2008, 03:15:13 PM »
Yes and No ... I like it to review things which are reported before adding it automatically.

I think your request could be done easily by a cron job or (WIN) scheduled task.

I come from Squirrelmail along with ArGoSoft backend and am in process to switch to RC/hMailServer. In my current setup I do it with a scheduled job.

-Roland
Regards,
Rosali
__________________
MyRoundcube Project (commercial)

Offline seansan

  • Jr. Member
  • **
  • Posts: 84
Re: Spam/Ham Report AddOn - need help in development!!
« Reply #7 on: February 07, 2008, 08:18:36 AM »

What about helping us develop the code below? that can later be added as plugin if they a hook is defined for it in the main program? Code is ment to be placed in the move_del.inc file (or later by hook) online 31, but I think that all the code is here to develop it stand-alone.

Please see all the todo;s below. Who can help?

Code: [Select]

# add to move_del.inv line 31

 // ok: ['learn_spam'] is new config variable
 // ok: For testing purposes add it to your main config file as
 // $rcmail_config['learn_spam'] = TRUE;
 if (isset($_CONFIG['junk_mbox']) && isset($_CONFIG['learn_spam']) &&
   $_CONFIG['learn_spam'] === true)
 {

  // todo: Do I need to check other vars like SESSION to confirm current mbox?
  $source = get_input_value('_mbox', RCUBE_INPUT_POST)

  // todo: Do I need to take _mbox and $target mbox (already fetched in move_del) through checks?
  // todo: Do I need to take _mbox and $target and formaat otherwise so they are comparable with config?
  $b_type = NULL;
  if ($target == $_CONFIG['junk_mbox'])
   $s_type = 'spam';
  else if ($source == $_CONFIG['junk_mbox'])
   $s_type = 'ham';

  $ret = system(&quot;which sa-learn&quot;, $retval);

  // todo: Anymore checks to add?
  if ($retval == 0 && file_exists($ret) && is_executable($ret) && !is_null($b_type))
   {



    // convert the list of uids to array
    $a_uids = is_string($uids) ? explode(',', $uids) : (is_array($uids) ? $uids : NULL);

    foreach ($a_uids as $uid)
    {

    // todo: Does this get the full e-mail, inlcuding headers?
    // todo: The IMAP function is referenced, how to call correctly?
    $src = $IMAP->get_raw_body($uid);

    // todo: Can we fork this process? (Seeing we are looping)
    // todo: How safe is this command and do we need to use escapeshellarg
    // todo: Is this the way to feed the full message source to sa-learn?
    $success = exec(escapeshellcmd(sprintf(&quot;sa-learn --%s %s&quot;, $s_type,$src)));

    // todo: Do we need to log the feedback if log is yes?
    }

   } else {

    // todo: Do we need to log the feedback if command not found and config set to YES?
   }
  }


Offline rosali

  • Hero Member
  • *****
  • Posts: 2,533
Re: Spam/Ham Report AddOn
« Reply #8 on: February 07, 2008, 03:49:12 PM »
If I understand your code structure right, you want to trigger mails to be learned as spam/ham if they are moved into spam folder ...

IMO you will mess your bayes DB.

In addition you plan to trigger to launch sa-learn every time a message is moved to junk folder ... ooouuucccchhhh ....

Is there a reason why you don't want to follow my approach just to copy the stuff into a specific folder to process it by some other stuff than the webmail suite?

-Roland
Regards,
Rosali
__________________
MyRoundcube Project (commercial)

Offline robbieg

  • Newbie
  • *
  • Posts: 5
Spam/Ham Report AddOn
« Reply #9 on: April 28, 2008, 03:38:57 PM »
im unable to get this working.

i get a call to Fatal error: Call to a member function get_mailbox_name() on a non-object in /usr/local/web/roundcube/program/include/main.inc on line 1993

when i remove get_mailbox_name() from the file..  it does nothing except show the move to bar on every page.

Offline rosali

  • Hero Member
  • *****
  • Posts: 2,533
Spam/Ham Report AddOn
« Reply #10 on: May 02, 2008, 05:21:56 AM »
Check this thread

http://www.roundcubeforum.net/showthread.php?t=2469

The plugin was coded for Release Canditate 2 ... it does not work with the stable releases.

-Roland
Regards,
Rosali
__________________
MyRoundcube Project (commercial)

Offline robbieg

  • Newbie
  • *
  • Posts: 5
Spam/Ham Report AddOn
« Reply #11 on: May 02, 2008, 09:38:47 AM »
that only works for hm mail server...  which is a free mail server for windows...

Offline rosali

  • Hero Member
  • *****
  • Posts: 2,533
Spam/Ham Report AddOn
« Reply #12 on: May 03, 2008, 02:30:03 AM »
No, there are some plugins which work only along with hmailserver. hmailserver specific things are disabled if you use another MTA. So it works with all servers as default RoundCube does.

-Roland
Regards,
Rosali
__________________
MyRoundcube Project (commercial)

Offline Ythan

  • Newbie
  • *
  • Posts: 4
    • http://www.shroomery.org/
Spam/Ham Report AddOn
« Reply #13 on: August 06, 2008, 04:05:40 PM »
Sorry to bump such an old thread. I just wanted to say I found this very useful, and also once you add this modification it's fairly simple to add Spam and Ham buttons to make the functionality more prominent.

skins/default/templates/mail.html

Find this:

Code: [Select]
<roundcube:button command=&quot;print&quot; imageSel=&quot;/images/buttons/print_sel.png&quot; imageAct=&quot;/images/buttons/print_act.png&quot; imagePas=&quot;/images/buttons/print_pas.png&quot; width=&quot;32&quot; height=&quot;32&quot; title=&quot;printmessage&quot; />

Immediately after, add this:

Code: [Select]



program/js/app.js

Find this:

Code: [Select]
     case 'checkmail':
        this.check_for_recent();
        break;

       
Immediately after, add this:

Code: [Select]
case 'spam':
this.move_messages('-1');
break;

case 'ham':
this.move_messages('-2');
break;

            
Find this:

Code: [Select]
   else
      {
      this.enable_command('show', 'reply', 'reply-all', 'forward', 'print', selected);
      this.enable_command('delete', 'moveto', 'mark', (list.selection.length > 0 ? true : false));
      }

     
Immediately after, add this:

Code: [Select]
   if (this.env.mailbox == this.env.junk_mailbox){
    this.enable_command('spam', false);
    this.enable_command('ham', (list.selection.length > 0 ? true : false));
    } else {
    this.enable_command('spam', (list.selection.length > 0 ? true : false));
    this.enable_command('ham', false);
    }

   
Here are the images I'm using for now:

http://mail.shroomery.org/skins/default/images/buttons/spam_sel.png
http://mail.shroomery.org/skins/default/images/buttons/spam_act.png
http://mail.shroomery.org/skins/default/images/buttons/spam_pas.png
http://mail.shroomery.org/skins/default/images/buttons/ham_sel.png
http://mail.shroomery.org/skins/default/images/buttons/ham_act.png
http://mail.shroomery.org/skins/default/images/buttons/ham_pas.png

Thanks again rosali for this great mod! :)

Offline javiaw

  • Newbie
  • *
  • Posts: 7
Spam/Ham Report AddOn
« Reply #14 on: September 16, 2008, 04:27:15 AM »
Hi Ythan,

I found some problems about this:

Code: [Select]
case 'spam':
this.move_messages('-1');
break;

case 'ham':
this.move_messages('-2');
break;

somehow it's strange to pass numbers to the function, I'm not sure if it will work for everyone, it should depend on how folders are configurated. At least in my case did not work, so I changed " '-1' " for "this.env.junk_mailbox". In the 'ham' case I haven't found the correct argument to pass.

Javi