Author Topic: A few ideas to turn RC faster  (Read 2909 times)

Offline calande

  • Jr. Member
  • **
  • Posts: 64
A few ideas to turn RC faster
« on: November 15, 2007, 02:23:12 PM »
I have studied the way Gmail works, it's really interesting to see how they did to limit the amount and frequency of traffic between the client and server.

I think RC should limit this amount and frequency of traffic as well to be really fast, almost like a desktop application. What is needed is an actual standalone Javascript application that manages everything itself on the client side. For instance to display a list of messages, have a function that creates a list of messages based on a JS array full of messages. Then every minute, the JS application polls the server to know if there's something new, and the server returns a signal that either says there's nothing new, or returns an organised list of data separated by a special character, for instance if there's a new message in the inbox, the returned string could be:

Code: [Select]
"inbox||0||Hello||Hello John, thank you for your confirmation||25KiB||17h14&inbox||1||Meeting tomorrow||Dear John, We have a meeting tomorrow||5KiB||17h18"
The JS application would have and keep in cache the list of messages in arrays. When you have the data in JS arrays, you don't have to retrieve it from the server again, it's on your computer. And if the data is changed on the server at some point, the JS application knows it within less than 60 seconds, and updates the JS array (ie: remove a contact from the address book array). Same for the contact list, you have the whole list of contacts in the JS array. When you hit "Compose", "Reply" or "Forward", JS creates the form without having to poll the server. For each new event you save 2 seconds :)

This is for the client-side Javascript application. Now, for the server side, I think the most critical point and I think the reason why webmails are so slow is that when you have hundreds of mebibytes of emails and attachments on your server, the IMAP server has a real hard time because messages aren't indexed. If you want speed and if you want the server to hand over the information in a blink of an eye, we need two things:
  • Store text email content in a database
  • Store attachments on the file system in directories, ie: /attachments/bob_at_example_dot_com/2007/11/15/project.pdf
There's a database-driven project called dbmail.org, it's very fast and works with message indexing in mind. Doing a research amoung 100,000 emails is done with an SQL select and returns the results in a fraction of a second, like on a forum. You could combine several criteria to make an advanced search with no effort, ie:
Quote
Search for "report" + "tchernobyl" in "subject" field of messages stored with "Reports" label older than 6 months
Your mailbox may have 5GiB or mails, your actual SQL database uses only 60MiB of text, the rest is made of attachments stored on the file system, it doesn't make your database slow and your server is happy :)
Receiving all emails and storing them to a database is not complicated at all, just use email piping: http://wiki.dreamhost.com/Email_to_Script - I tried it on my account, it's instantaneous and efficient. Just process every incoming email using PHP, separate text from attachments, store text into the MySQL DB, store the list of attachments to the HDD and that's it. Create a cron job to optimize your database everyday at 3:00am, when you open your mail at 7h30am it's super fast. I'm sure you can be as fast as Gmail. Hope these suggestions will help the project to stand out from the crowd. Good luck ;)

Offline calande

  • Jr. Member
  • **
  • Posts: 64
Re: A few ideas to turn RC faster
« Reply #1 on: November 17, 2007, 02:14:48 PM »
No feedback? :)

Offline leesmark

  • Newbie
  • *
  • Posts: 1
Re: A few ideas to turn RC faster
« Reply #2 on: December 10, 2007, 12:17:08 AM »
Can you give some information "how-to" for newbies like me? :) By the way, i like your ideas a lot.

Offline calande

  • Jr. Member
  • **
  • Posts: 64
Re: A few ideas to turn RC faster
« Reply #3 on: December 10, 2007, 06:18:40 AM »
Ah, no, this is for developers, for future releases ;)

Offline jeremyr

  • Newbie
  • *
  • Posts: 6
Re: A few ideas to turn RC faster
« Reply #4 on: December 13, 2007, 01:12:44 AM »
Sounds like it has potential. This would completely change the way RC works. I'd be interested in helping where i can if it truly makes it run faster.

Offline calande

  • Jr. Member
  • **
  • Posts: 64
Re: A few ideas to turn RC faster
« Reply #5 on: December 13, 2007, 04:57:12 AM »
Yeah, when I use the latest Gmail version, I can see it runs like a local application. RoundCube can do it to! :)

I don't know if developers look at this forum section, I'd like some feedback to know what they think about the idea ;)