Author Topic: Transferring from a Shared Hosting Environment to VPS Hosting Environment  (Read 1780 times)

Offline mikeklon

  • Newbie
  • *
  • Posts: 8
Hello, thank you for any suggestions.

Our current RoundCube distribution is running in a Shared Hosting Environment.  We'll be transitioning to a VPS Environment at another hosting company in the near future.

Is there a procedure I can follow to move everyone's email service keeping all emails and folders intact?

Thank you for any guidance.

Mike K.

Offline mikeklon

  • Newbie
  • *
  • Posts: 8
Re: Transferring from a Shared Hosting Environment to VPS Hosting Environment
« Reply #1 on: December 22, 2023, 01:36:07 PM »
Some additional information:

The Shared Hosting is under cPanel and the VPS system is under Direct Access.  I have super user capability in the VPS system.

Offline JohnDoh

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 2,856
Re: Transferring from a Shared Hosting Environment to VPS Hosting Environment
« Reply #2 on: December 23, 2023, 02:09:18 AM »
Roundcube is an IMAP client. It does not store any emails or folder information, that is held by the IMAP server. Users' preferences, contacts etc are stored in the database. That can be dumped and restored like any other db, nothing special is required.
Roundcube Plugins: Contextmenu, SpamAssassin Prefs, and more…

Offline mikeklon

  • Newbie
  • *
  • Posts: 8
Re: Transferring from a Shared Hosting Environment to VPS Hosting Environment
« Reply #3 on: December 24, 2023, 11:39:25 AM »
Thank you for the reply. However . . .

I should have said this initially, but I do NOT have root access on the Shared Hosting environment. 

As such, I don't see any  IMAP tables in MyPHPAdmin (assuming RoundCube is using MySQL).

I am so very new to RoundCube.  Thank you again for any guidance.

Mike K.

Offline mikeklon

  • Newbie
  • *
  • Posts: 8
Re: Transferring from a Shared Hosting Environment to VPS Hosting Environment
« Reply #4 on: December 24, 2023, 12:23:37 PM »
Some additional info:

On the existing Shared Hosting environment, a directory called "mail" exists under the main account directory.  Within this mail directory are a myriad of dovecot.... files, several hidden files, (Trash, Sent, Junk, Drafts, Archive, hidden links for each user's email directory, a directory named for the destination domain name of the web site), and directories  cur, new, and tmp.  It appears that all the contents of these files (and any files in sub-directories) are hashed values. 

I'm guessing that just copying this directory to the new site would not serve to move everything from host A to Host B.

Is there some documentation/tutorial for backing up all email for all email users and then restoring the backup ?

Thank you again for helping a newbie.

Offline SKaero

  • Administrator
  • Hero Member
  • *****
  • Posts: 5,882
    • SKaero - Custom Roundcube development
Re: Transferring from a Shared Hosting Environment to VPS Hosting Environment
« Reply #5 on: December 24, 2023, 02:42:06 PM »
You would likely be better off asking your question on a Direct Admin forum since the question your asking has more to do with both how cPanel and Direct Admin are configured rather than how Roundcube is setup since Roundcube is just webmail software and how it was setup on each system can vary.

That said I will put some general notes that maybe of use, note that all of these are how a typical cPanel server is setup but its could be configured differently. In your cPanel account the mail folder should be a typical Maildir format and hold all accounts emails/folders. The Roundcube database is stored in etc/<cPanel username>.rcube.db and is a sqlite database, that holds all the users contacts and setting.

As for how to migrate that into Direct Admin you should ask people who are more familiar with Direct Admin.

Offline PlymouthExpat

  • Newbie
  • *
  • Posts: 7
Re: Transferring from a Shared Hosting Environment to VPS Hosting Environment
« Reply #6 on: December 26, 2023, 07:34:41 AM »
As has already been mentioned, Roundcube is a web-based facility that provides access to IMAP/SMTP facilities, whether on the same server or elsewhere; moving the Roundcube installation folder to another server will require DNS changes to point to the new location (for web access) but nothing else as long as the database and IMAP/SMTP server locations do not change. I have no doubt that you have access to the database on the shared facilities (I used to run just such a system prior to the adoption of a VPS), so will be able to dump the records and upload same to a new database on the VPS. If you need to change the location of the IMAP/SMTP servers that I assume are on the shared facilities at present, then you’ll have to seek advice in another forum as that is not a Roundcube issue. My own procedure was to set up Postfix and Dovecot/Sieve on the VPS first and ensure it was fully functional prior to any other action.
« Last Edit: December 26, 2023, 11:45:08 PM by SKaero »

Offline mikeklon

  • Newbie
  • *
  • Posts: 8
Re: Transferring from a Shared Hosting Environment to VPS Hosting Environment
« Reply #7 on: December 26, 2023, 06:12:07 PM »
You would likely be better off asking your question on a Direct Admin forum since the question your asking has more to do with both how cPanel and Direct Admin are configured rather than how Roundcube is setup since Roundcube is just webmail software and how it was setup on each system can vary.

That said I will put some general notes that maybe of use, note that all of these are how a typical cPanel server is setup but its could be configured differently. In your cPanel account the mail folder should be a typical Maildir format and hold all accounts emails/folders. The Roundcube database is stored in etc/<cPanel username>.rcube.db and is a sqlite database, that holds all the users contacts and setting.

As for how to migrate that into Direct Admin you should ask people who are more familiar with Direct Admin.

Thank you SKaero for your explanation.  That information helps a lot.  However, in the Shared Hosting environment, all folders under root (/) are unavailable to me, except for my /home folder.  I do not have root access and can't access /etc and any .db files which are not under my /home directory.

I DO have root/super user privileges on the Direct Admin host (the VPS) host, so if I could get a dump of the /etc/...../.rcube.db, I'd have no problem restoring and/or rebuilding the emails on the VPS/Direct Admin host.

I'll contact the Shared Hosting Admin and see if they'd be willing to do this .rcube.db dump for me. The Shared Hosting admins are not likely to cooperate.  I've had some terrible experiences with them to date.

I will keep all of you posted and thank you again for helping an rcube newbie.

p.s. @PlymouthExpat -- so I understand IMAP/SMTP etc, but your " ________________" don't really add anything.  What exactly are you trying to say?

Offline SKaero

  • Administrator
  • Hero Member
  • *****
  • Posts: 5,882
    • SKaero - Custom Roundcube development
Re: Transferring from a Shared Hosting Environment to VPS Hosting Environment
« Reply #8 on: December 26, 2023, 11:46:39 PM »
The etc folder I'm referring to should be in your home folder that you can access as a shared hosting user I'm not referring to the /etc folder for the whole server.

Offline PlymouthExpat

  • Newbie
  • *
  • Posts: 7
Re: Transferring from a Shared Hosting Environment to VPS Hosting Environment
« Reply #9 on: December 27, 2023, 06:40:05 AM »
p.s. @PlymouthExpat -- so I understand IMAP/SMTP etc, but your " ________________" don't really add anything.  What exactly are you trying to say?


Does your quote of a load of underline characters mean that you consider my whole post doesn't add anything? A little curt, don't you think, when I was merely trying to assist you, someone who has already admitted that Roundcube is a novelty. At no time have you mentioned the eventual location of the IMAP and SMTP servers; where your IMAP server stores its data files (emails and management files) is of greater importance to you than the location of the Roundcube installation, as Roundcube is only an IMAP client (as already mentioned).


You are moving your Roundcube web folder to a VPS, for which you have direct (and full) access. If, by that, you mean you will be using FTP to manage the VPS's facilities, I suggest you consider something like Webmin, which will not only give a graphical interface for the FTP function, but give intimate control of all other aspects of the VPS, including software installations and upgrades.


Essentially, you have given insufficient information to explain the full context of your current configuration and that of your future configuration, both of which are outside of the normal operating parameters of Roundcube.

Offline mikeklon

  • Newbie
  • *
  • Posts: 8
Re: Transferring from a Shared Hosting Environment to VPS Hosting Environment
« Reply #10 on: December 27, 2023, 09:12:51 AM »
SKaero:

Thank you for the clarification about /etc.  I did find those folders/files in the /home/etc directory.

As advertised, the tech support at the shared host were useless.  They said "copy the files from the /home/etc directory and you're done". This obviously is not a competent solution.

I did ask the tech's at the VPS hosting for a recommendation on how to transfer the emails.  They have given me a bit of a convoluted procedure using a script called "imapsync".  The procedure is quite involved.  I will report back if I have any success with it.  I'll give more details if it's successful.

I also found a possible solution using an app called Thunderbird by Mozilla.  Using this app would be a bit cumbersome because it involves creating profiles for each user, exporting all emails for all users, then making new profiles and importing the previous exports for each user. I'll test this for a single user first to see if it even works.

Also -- thank you for editing the original post by PlymouthExPat.  I only saw "_________" on his original post which is why I replied with "That doesn't add anything".

Offline mikeklon

  • Newbie
  • *
  • Posts: 8
Re: Transferring from a Shared Hosting Environment to VPS Hosting Environment
« Reply #11 on: December 27, 2023, 09:55:36 AM »
p.s. @PlymouthExpat -- so I understand IMAP/SMTP etc, but your " ________________" don't really add anything.  What exactly are you trying to say?


Does your quote of a load of underline characters mean that you consider my whole post doesn't add anything? A little curt, don't you think, when I was merely trying to assist you, someone who has already admitted that Roundcube is a novelty. At no time have you mentioned the eventual location of the IMAP and SMTP servers; where your IMAP server stores its data files (emails and management files) is of greater importance to you than the location of the Roundcube installation, as Roundcube is only an IMAP client (as already mentioned).


You are moving your Roundcube web folder to a VPS, for which you have direct (and full) access. If, by that, you mean you will be using FTP to manage the VPS's facilities, I suggest you consider something like Webmin, which will not only give a graphical interface for the FTP function, but give intimate control of all other aspects of the VPS, including software installations and upgrades.


Essentially, you have given insufficient information to explain the full context of your current configuration and that of your future configuration, both of which are outside of the normal operating parameters of Roundcube.

"________________" was what I saw on your original post, so yes, that post didn't add anything.  I see that SKaero edited the post on 12/26, which does add more information. My apologies if you thought I was being "curt".  I meant no disrespect, just communicating that "___________" is not helpful.

My apologies if you feel I didn't give enough information about what my original request.  I assumed most everyone already understood the differences between a shared hosting environment and a VPS environment.  I also thought I made it clear that we're moving to a new hosting company which means that the database and email servers WILL change by default. If you feel I didn't offer enough information, then perhaps it would have been appropriate to have asked for clarification or more information.

In general and depending on the hosting company, in a shared hosting environment the web master DOES NOT have root or super user privileges.  So even though a web master has access to the /home/etc directory where RoundCube stores it's files, there are no database privileges to do any kind of table dump so it turns out this solution isn't an option. (If RoundCube already creates database "restore" files automatically as it's backup, I as a novice user wouldn't know about them or even if they'd be compatible on a different hosting company and structure.

I presumed that changing hosting companies AND changing from shared to VPS would naturally change the way RoundCube would store and access it's files, including configuration files. This is why I asked for guidance being a RoundCube novice.  I also don't recall saying RoundCube is a novelty.

FTP (File Transfer Protocol) is not a tool I would use to perform system administration tasks anywhere (although depending on the interface, I suppose it could be done Adhoc).  In general, I usually don't use a WYSYWIG app to do system administration work. I typically use SSH in a Command Line Interface, which when used with a private/public key combination is a very secure method for communications between systems and is generally efficient and powerful.

By the way, I am not using a DOS/Windows machine locally so any DOS/Windows apps are useless for me.

Really, thank you for taking the time to contribute to the thread. 

Offline PlymouthExpat

  • Newbie
  • *
  • Posts: 7
Re: Transferring from a Shared Hosting Environment to VPS Hosting Environment
« Reply #12 on: December 27, 2023, 10:25:37 AM »
I still don't understand how you ended up seeing "_________" as my post and have asked @SKaero to explain in a PM. It was certainly not what I posted 🙄


I think we were 'talking' at cross-purposes, as I wasn't clear that you were conversant with the management of a remote server (VPS or otherwise), so offered the choice of Webmin to assist. I am a long-term Windows/Unix/Linux administrator and operate 2 VPSs managing them using Webmin. You will need to use SSH (as I did) to install Webmin but the subsequent access to all parts of the VPS will become much simpler, however experienced and competent you are with SSH. At home, I use Mac OS (with Windows 11 in a Parallels virtual PC).


The fact that you are moving to a new hosting provider for all your facilities was what prompted my remarks about the IMAP/SMTP servers. As Roundcube does not manage the emails within itself (it merely displays what it receives in exactly the same way as any other email client), the transfer of emails for all users will depend on the data you can retrieve from the files and folders within your ~/mail folder in the shared environment.


While this is a Roundcube forum, I am prepared to offer any assistance I can via PMs.


p.s. Please note that I am female! 😉

Offline SKaero

  • Administrator
  • Hero Member
  • *****
  • Posts: 5,882
    • SKaero - Custom Roundcube development
Re: Transferring from a Shared Hosting Environment to VPS Hosting Environment
« Reply #13 on: December 27, 2023, 12:34:57 PM »
@PlymouthExpat your original post had a font size set to 1 for all the text after the first sentence so it looked like something like this:
As has already been mentioned, Roundcube is a web-based facility that provides access to IMAP/SMTP facilities, whether on the same server or elsewhere; moving the Roundcube installation folder to another server will require DNS changes to point to the new location (for web access) but nothing else as long as the database and IMAP/SMTP server locations do not change. I have no doubt that you have access to the database on the shared facilities (I used to run just such a system prior to the adoption of a VPS), so will be able to dump the records and upload same to a new database on the VPS. If you need to change the location of the IMAP/SMTP servers that I assume are on the shared facilities at present, then you’ll have to seek advice in another forum as that is not a Roundcube issue. My own procedure was to set up Postfix and Dovecot/Sieve on the VPS first and ensure it was fully functional prior to any other action.

I noticed that after @mikeklon and my replies so I just removed the font size since I assumed that wasn't what you had wanted to do. Sorry I didn't post anything leading to the confusion.

Offline mikeklon

  • Newbie
  • *
  • Posts: 8
Re: Transferring from a Shared Hosting Environment to VPS Hosting Environment
« Reply #14 on: December 29, 2023, 12:54:49 PM »
Good afternoon.

Sorry for the delay reporting back on this. Other issues popped up for the conversion which delayed my getting back and reporting.  I'm attempting to make this brief.  If anyone wants more info, you can post a request on the thread and I'll reply.

I ended up using a tool called "imapxfer" per the destination tech's suggestion (turn's out there's even a man page for it).  The tool transfers all information including all emails, folders, and tags.  I chose to install it on the destination server, which was not as daunting as I presumed it would be.  The tool still requires execution for each individual user, AND both origin and destination user account passwords are required.  I wrote a small shell script and ran it for each user, making the whole process rather easy.  The most difficult part was setting up password files for each user.

It also turns out the imapxfer tool can be run multiple times on a user without producing duplicates on either server (I thought this was valuable).

Ultimately this was the "cleanest" process in "transferring" email account from one server to another irrespective of the email interface, RoundCube in this case.

Thank you all for your contributions, time, and patience. Stay healthy, stay safe, and enjoy the new year.

Mike K.