We need to reboot both web1 (sgc.gbdns.net) and web2 (daedalus.gbdns.net) to complete the latest round of Windows Updates.

Both servers will be rebooted on 14/06/2009 at 00:40 AM. This will take no more than 3-5mins per server.

All other servers within our hosting cluster were automatically rebooted in the early hours of the 10th June 2009.

Update 14/06/2009 00:50:
Both servers have been rebooted downtime was <4mins per server.

Again web1.gbdns.net is experiencing network issues. After the last outage we made the secondary NIC interface live which is still responding so we believe the primary NIC is faulty.

IP’s are being moved from one NIC to the other at the moment and we are awaiting a VLAN update from the network provider.

Service should be restored within 5-10mins maximum.

Update 12:45
Service was fully restored at 12:39.

We do apologise for the length of the outage but we chose to fix the issue fully rather than partially fixing the issue and having another outage later on to fully fix the problem.

We are currently investigating an outage of web1.gbdns.net.

This looks to be NIC related.

Update 22:48
A KVM was attached to the server, the primary NIC wasn’t passing any traffic in or out. Disabling / re-enabling the NIC didn’t solve the problem so we had to perform a reboot.

As precaution we are going to enable the secondary NIC interface in case the first fails again so we can access the server and solve the problem.

07/06/2009 01:00AM

Servers web2 and web3 within our shared hosting cluster need to be rebooted to complete software installations.

The reboot will take 3-5mins per server.

08/06/2009 01:00

We will be moving all FTP accounts from our current Serv-U FTP Server software to Gene6 FTP Server. This migration is relatively simple as we have created an application to perform the required tasks.

This will cause up to 2mins of interruption to FTP Service’s on web1 and web2.

All usernames and passwords will stay the same and you will be able to manage accounts through Helm as normal.

This work does not affect Windows 2008 users as the servers are already using Gene6.

Server SGC (web1.gbdns.net) will be rebooted on 18/05/2009 at 00:10.

This is to complete the install of the latest Windows Updates.

Downtime will be less than <7mins.

Update 18/05/2009 - 00:22
The reboot is now complete. (Start 00:18 - Finish 00:21)

We have some maintenance to complete on server Daedalus. This involves replacing both hard drives within the Raid 1 set with a new pair.

This work will be carried out in two phases. During both phases there will be a very short period of downtime. This will effect email services and websites hosted on web2.

Phase 1
Start: Thursday 14th May 2009 @ 23:00hrs
End: Thursday 14th May 2009 @ 23:10hrs
Impact: ~10 Minutes

During this window the server will be shut down and the first drive will be replaced. The system will then be booted up to Windows as normal. The Raid Array will re-sync itself during the early hours of Friday morning.

Phase 2
Start: Friday 15th May 2009 @ 23:00hrs
End: Friday 15th May 2009 @ 23:10hrs
Impact: ~10 Minutes

During this window the server will be shut down and the second drive will be replaced. The system will then be booted up to Windows as normal. The Raid Array will re-sync itself during the early hours of Saturday morning.

Update 14/05/2009 - 23:19
The server is currently shutting down and should return within approximately 10mins.

Update 14/05/2009 - 23:33
The server shutdown completed at 23:23.

The first new drive has been installed and the server has been booted to Windows. The Raid Array is currently re-syncing with the new drive, this will take 3-4hrs to complete and will be done in the background.

Update 15/05/2009 - 23:13
The server is currently shutting down and should return within approximately 10mins.

Update 15/05/2009 - 23:29
The server shutdown completed at 23:16.

The second new drive has been installed and the server has been booted to Windows. The Raid Array is currently re-syncing with the new drive, this will take 3-4hrs to complete and will be done in the background.

We are aware of web1 and web2 along with mail and mail2 being offline at the moment. This is currently being investigated.

This only affects Windows 2003 shared / reseller hosting customers.

Update 8:07am - Everything appears to be back online, total downtime 9mins. We are currently awaiting an RFO.

Update 8:15am - Both boxes appear to offline again, we have escalated this back to the network provider for resolution.

Update 8:26am - Both boxes are back online, we are waiting to hear if this issue has been resolved or if it is still ongoing.

Update 8:59am - Unfortunately everything is up and down at the moment. We have received a notification from our network provider that the issues are related to one of their routers failing to detect a second router in the VRRP pair failing hence the working router was still routing traffic to a dead router.

Update 10:08am - We still don’t have an ETA from our network provider on fully resolving this issue. We have been informed this issue is known by the router manufacture and we are awaiting a further update.

Access to our 193.219.118.X block of addresses has been restored but pings are around 17ms above normal. This means web2 and email are functioning again.

Access to our 195.74.55.X block of addresses is still unavailable.

We will continue to update the status page when we have more info.

Update 12:15 - All services have been restored. A full RFO will be sent to all customers later on today.

All email held at the backup mail servers is being processed but it may take 1-2hrs for this to arrive in users inboxes.

We will be performing a reboot of web3.gbdns.net on 10/04/2009 at 00:45 this should take no longer than 3-5mins to complete.

During the reboot all websites hosted on web3 will be unavailable along with MSSQL 2008 databases.

12/03/2009 00:15-00:30AM GMT

We will be rebooting all Windows 2003/2008 servers within our shared hosting cluster this evening. This will take no more than 3-5mins per server.

This is to complete the latest round of Windows Updates and also correct a problem with IIS Password.

Update - 00:30
All servers have now been rebooted.

IIS Password is now working correctly on all Windows 2003 Servers.