Author Topic: Optimization of IMAP full text searches (FTS) ?  (Read 4611 times)

Offline johanp

  • Newbie
  • *
  • Posts: 3
Optimization of IMAP full text searches (FTS) ?
« on: November 25, 2011, 07:10:23 PM »
I've noticed one behavior of Roundcube in relation to IMAP searches that I belive could relatively easy gain a performance boost.

Some IMAP servers (e.g. Dovecot and Cyrus) can be told to maintain a separate full-text index to speed up IMAP TEXT searches.

However these optimizations are only done when issuing the SEARCH TEXT IMAP command.

In the search dialogue of roundcube which by default have the "Subject" and "From" fields selected and do a search this will not trigger a FTS, which is fine. However if I keep these selected and also select "entire message" as a search field one would expect the imap "search text" command to be used.However that does not seem to be the case - (but I might be wrong here, but I don't believe so :_) )

On the other hand if I only have the "entire message" field select then the indication is that the imap "search text" command will be used.

I have not verified this by looking at the logs of the exact IMAP commands issued by roundcube but instead indirectly by the behavior of the IMAP server logs (and indexes). In this case Dovecot.

It would really significantly increase the search performance if the imap search text command was used whenever the "Entire message" field was selected as the search area since that would have the IMAP server use its FTS index.

Update:
These are the exact IMAP commands sent by roundcube

CASE 1: Seach with only "Subject" and "Header"
A0002 SELECT INBOX
A0003 SEARCH CHARSET UTF-8 HEADER SUBJECT {5+}
test-text-to-search
A0004 LOGOUT

CASE 2: Seach with "Subject", "Header" and entire message
A0002 SELECT INBOX
A0003 SEARCH CHARSET UTF-8 OR HEADER SUBJECT {5+}
test-text-to-search TEXT {5+}
test-text-to-search
A0004 LOGOUT

CASE 3: Seach only entire message
A0002 SELECT INBOX
A0003 SEARCH CHARSET UTF-8 TEXT {4+}
test-text-to-search
A0004 FETCH 25,12,3 (UID RFC822.SIZE FLAGS INTERNALDATE BODY.PEEK[HEADER.FIELDS (DATE FROM TO SUBJECT CONTENT-TYPE CC REPLY
-TO LIST-POST DISPOSITION-NOTIFICATION-TO)])
A0005 LOGOUT


Looking at the above and CASE 2 makes me wonder. It seems like the full text search is done albeit as an additional search. It seem like the semantics of having "SEARCH TEXT" directly and combining it with a HEADER search is different on the IMAP server. More to follow ...

/Johan
« Last Edit: November 26, 2011, 04:33:06 AM by johanp »

Offline alec

  • Hero Member
  • *****
  • Posts: 1,365