Roundcube Community Forum

Release Support => Requests => Topic started by: calande on November 15, 2007, 02:23:12 PM

Title: A few ideas to turn RC faster
Post by: calande on November 15, 2007, 02:23:12 PM
I have studied the way Gmail works, it's really interesting to see how they did to limit the amount and frequency of traffic between the client and server.

I think RC should limit this amount and frequency of traffic as well to be really fast, almost like a desktop application. What is needed is an actual standalone Javascript application that manages everything itself on the client side. For instance to display a list of messages, have a function that creates a list of messages based on a JS array full of messages. Then every minute, the JS application polls the server to know if there's something new, and the server returns a signal that either says there's nothing new, or returns an organised list of data separated by a special character, for instance if there's a new message in the inbox, the returned string could be:

Code: [Select]
"inbox||0||Hello||Hello John, thank you for your confirmation||25KiB||17h14&inbox||1||Meeting tomorrow||Dear John, We have a meeting tomorrow||5KiB||17h18"
The JS application would have and keep in cache the list of messages in arrays. When you have the data in JS arrays, you don't have to retrieve it from the server again, it's on your computer. And if the data is changed on the server at some point, the JS application knows it within less than 60 seconds, and updates the JS array (ie: remove a contact from the address book array). Same for the contact list, you have the whole list of contacts in the JS array. When you hit "Compose", "Reply" or "Forward", JS creates the form without having to poll the server. For each new event you save 2 seconds :)

This is for the client-side Javascript application. Now, for the server side, I think the most critical point and I think the reason why webmails are so slow is that when you have hundreds of mebibytes of emails and attachments on your server, the IMAP server has a real hard time because messages aren't indexed. If you want speed and if you want the server to hand over the information in a blink of an eye, we need two things: There's a database-driven project called dbmail.org, it's very fast and works with message indexing in mind. Doing a research amoung 100,000 emails is done with an SQL select and returns the results in a fraction of a second, like on a forum. You could combine several criteria to make an advanced search with no effort, ie:
Quote
Search for "report" + "tchernobyl" in "subject" field of messages stored with "Reports" label older than 6 months
Your mailbox may have 5GiB or mails, your actual SQL database uses only 60MiB of text, the rest is made of attachments stored on the file system, it doesn't make your database slow and your server is happy :)
Receiving all emails and storing them to a database is not complicated at all, just use email piping: http://wiki.dreamhost.com/Email_to_Script - I tried it on my account, it's instantaneous and efficient. Just process every incoming email using PHP, separate text from attachments, store text into the MySQL DB, store the list of attachments to the HDD and that's it. Create a cron job to optimize your database everyday at 3:00am, when you open your mail at 7h30am it's super fast. I'm sure you can be as fast as Gmail. Hope these suggestions will help the project to stand out from the crowd. Good luck ;)
Title: Re: A few ideas to turn RC faster
Post by: calande on November 17, 2007, 02:14:48 PM
No feedback? :)
Title: Re: A few ideas to turn RC faster
Post by: leesmark on December 10, 2007, 12:17:08 AM
Can you give some information "how-to" for newbies like me? :) By the way, i like your ideas a lot.
Title: Re: A few ideas to turn RC faster
Post by: calande on December 10, 2007, 06:18:40 AM
Ah, no, this is for developers, for future releases ;)
Title: Re: A few ideas to turn RC faster
Post by: jeremyr on December 13, 2007, 01:12:44 AM
Sounds like it has potential. This would completely change the way RC works. I'd be interested in helping where i can if it truly makes it run faster.
Title: Re: A few ideas to turn RC faster
Post by: calande on December 13, 2007, 04:57:12 AM
Yeah, when I use the latest Gmail version, I can see it runs like a local application. RoundCube can do it to! :)

I don't know if developers look at this forum section, I'd like some feedback to know what they think about the idea ;)