IMPORTANT NOTICE - Server Upgrades

On Thursday, March 24, 2005, we will begin moving all hosting accounts to better, faster, more reliable servers. All customers are being systematically upgraded to Dual Xeon 2.4 GHz servers with 2 GB RAM and RAID1 mirroring technology. The upgrades will be completed this weekend. This will accomplish the following:

1. You will notice an increase in the speed your website(s) are accessed
2. You will notice an increase in server reliability
3. In the event of a hard disk failure, there will be NO downtime until we replace the faulty disk. When we do, downtime will be limited to between 5 and 15 minutes (instead of the 24+ hours some have experienced).

We will no longer be running daily backups of all accounts on our servers. Instead, all servers will have dual hard drives with the RAID1 controller. RAID 1 is known as "mirroring". There are two hard disks in each server's configuration, and your server keeps identical copies of the data on both disks. In the event of a hard disk failure, the server will keep running with the data on the good disk until the faulty disk can be replaced. This dramatically increases the reliability of your server, and insures against data loss. Performance is also increased as your server can read data from your hard disks at twice the normal rate.
EDIT OUT - Please see note below - We WILL continue running backups - but we'll be backing up to an off-site backup server, thus adding another level of redundancy.

As has happened in the recent past, when a hard disk fails without having mirrored hard disks, it means your accounts are down until we can replace the faulty disk and then restore your account(s). This restore process can be very time consuming, and has resulted in downtimes in excess of 24 hours. With the new RAID1 configurations, downtimes should be limited to under 15 minutes.... the time needed to turn off the server, replace the faulty disk, then reboot. RAID1 will then automatically resume mirroring the drives.

So here's what's going to happen:
1. We'll copy your account(s) over to a new server
2. You will NOT have to update your nameservers.... they will be setup the same way on the new server
3. Once your accounts are copied, we'll update our nameservers to their new IP addresses
4. After our nameservers propagate to the new server, your domain names will be resolving to the new server, and the move will be complete. There should be no downtime for your websites. If there is, it will be minimal.
5. We'll send you an email advising of your new account IP address (for accessing CPanel), new nameserver IP addresses (for your records), and new server name and SSL link.

Considerations:
1. Your usernames and passwords (for accounts, email addresses, etc) will remain the same. All existing settings will be copied over.
2. If you access any of your site(s)' pages using an IP address, the IP address will be changing. Another email will follow with your site(s)' new IP address.
3. If you use our free SSL, then you will have to update any links that use SSL because the server name will be changing.
4. If after the move is complete you notice missing files (perhaps you updated your site while the move was taking place thus only updating the old copy of the site), let our support staff know immediately and they'll move a fresh copy of your site over to the new server. We will keep the old servers online and available for about 1 week after the move.

More information will follow, and I trust these new upgrades will meet with your approval. Any questions can be directed to info@hostedtoday.com.

We heard you loud and clear

A number of people voiced their concern about not having backups of accounts.... that RAID1 still doesn't help in the event of a server being hacked. In a nutshell, it's still a very good idea to have a backup even when running mirrored drives.

So we are going to also implement a backup plan.... a better plan than we had running before.

In the past our backups were running daily and re-copying all files every day to a backup drive in the server. With these new servers, we are instead going to run incremental backups (i.e. only the information that has changed), and we are going to backup all servers to a remote location in another data center.

This should address any backup concerns you have.

Rob this is FABULOUS news!!
Thanks so much for letting us know. Although I'm sure it will be a smooth transition, it's nice to know beforehand.
Thanks again!
Marleen

The account copy process began a couple hours ago and is on-going. There are thousands of accounts to be copied so it may take a few days for all to be finished. Emails will be going out to customers once their accounts have been successfully copied. For resellers with custom nameservers, the email will have your new nameserver IPs which we are setting up.

Further individual instructions will follow via email.

Hey Rob,

Just wondering how the update is going? It has been a few days and I have not received any update notifications yet. Thanks in advance.

The account copies are still on-going. All HostedToday accounts have been copied over to their new servers, while most ResellersToday accounts have been copied. The rest should finish up today. It's a pain-stakingly slow process.

Email notifications will be going out today for most, and as soon as the rest of the copies are done for the rest.

Thanks Rob, I will watch for the email.

Hi Rob,

I got my update email, but it said I do not have custom name servers! I do have custom name servers. Please advise.

Thanks.

This upgrade has become the biggest nightmare any host provider could imagine.

Lets start with the new ticket support system ...

1. No email has ever been received containing a copy of the support ticket.
2. No email is received with the key access to the support ticket.
3. You can not access your ticket without your key.

Lets say this ticket system is no different then the old method. So where is the improvement? You still have to submit several tickets to get one response.

New Server Setup ...

Two thumbs up to a faster server and RAID. I believe no one would disagree with that. However, was the server unpacked and just plugged in? Did a system engineer look at the old server setup and ensure that the new server would contain and have avaliable the then compile Apache modules?

Since most software and API code that deals with secure transactions, submissions, etc. use curl, curl ssl, mcrypt and zend you would think that they would have been compiled with the new server. Thank you to Pirya at support for at least getting curl, curl ssl, and mcrypt compiled. The reponse to Zend was, "it is coming soon." Now I am hedging my bet on how long it will take to have Zend installed.

Upgrade Email Information ...

The following did happen:

1. The account(s) were copied.
2. The custom name servers are the same with the IPs listed in the email received.
3. I am going to assume that the update is in progress since we are still pointing to the old server.
4. See number 3.

What did not happen:

1. The updated IP addresses to access cPanel and WHM are invalid. Yet another support ticket opened.
2. Our dedicated IP address for our site was not moved or reassinged. According to support, "it's coming soon"
3. Our SSL certificate (the reason we purchased dedicated IP addresses), is now garbage. According to support, "it's coming soon"
4. We did not get any type of notification regarding the new block of dedicated IP addresses if they were to change.
5. Surprisingly, we emailed info@resellerstoday.com on March 24, 2005 one day after the initial upgrade notice and four days before the upgrade process began regarding our dedicated IPs, SSL certificate, and requesting information on if they would be changing and would the SSL certificate be updated and reinstalled to reflect the changes. No response was received. I guess we should not be surprised.

When you look at the whole picture, even though the increased speed and redundancy is a step in the right direction, it is worthless if the migration is full of cracks, non-functioning componets, recompiles, additional lag time, and numerous attempts at support.

It is obvious that no one looked at this upgrade and ask themselves the important question, "Do we have everything set-up correctly, from the server stand point to the clients needs?", before unleashing this mess and then giving a response of "it's coming soon."

Coming soon does not answer our clients emails and telephone calls. Coming soon does empower us with solid information to make an informed decision on our responses to our clients and is definately not an acceptable answer to our investors and directors or myself.

IS THIS UPGRADE CAUSING THE SERVERS TO GO BACK IN TIME??

I run a forum and have lost a days worth of people who signed up as well as my homepage goes back to what it was before I uploaded the new page. Can you please tell me if this is the affect, I have sent to "PLEASE HELP ME" messages with no response, this is the second day and I have had the same problem. From 400 hits per day I went down to 36???? It makes no sense, please advise.

www.lumana.com

Lost data on Cyan

5 months after hearing incremental backups will be taken I just had this from support

We can understand your concern over this issue. If you have any personal backups please upload it we will restore it for you. Now we have configured the server to take the backup daily and to store a copy of the backups to another brand new hard disk also so that we can avoid such a situation in the future. Please be assured that we have taken all necessary steps to avoid such a bad time in future.