Our network provider has indentified a potential problem with one of their core distribution switches located at the Kent Science Park Data Center. The problem could potentially cause network instability at higher traffic levels. They have been advised by their hardware vendor that this can be resolved by booting into an alternative firmware as a permanent fix.

Due to the critical nature of this issue, the work will take place tonight between 23:45hrs and 00:15hrs. The new firmware has already been copied to the switch, it simply needs a restart in order to boot from it. During this time our servers in Kent will experience up-to a 30 second connectivity drop while the new firmware loads. Our network provider have tested the new firmware in a lab environment, however the old firmware will be left on the switch in order that they can revert back to it should anything go wrong.

Update - 31/12/2007 01:00 -
This work was completed without issue.

We have altered our PHP config to make it more secure.

This has been applied to all our Windows Servers.

If any users encounter any issues please raise a support ticket http://www.dhelpdesk.com

We are required to reboot both Windows Servers in our Helm Cluster. This work will be carried out on Sunday morning at 1:00AM

This is purely for windows update to complete installing December updates.

Each server will take 3-5mins to reboot.

Update - 16/12/2007 01:30
Both servers have now been rebooted.

We applied a fix to solve our slow startup issue; previous reboots would take around 7-10mins per server.

Server Atlantis started up within 2mins
Server SGC started up within 4mins

There is still an issue with startup on Server SGC. This is a config issue on our network provider’s switch, essentially the switch polls the port for activity. When the server turns off the switch turns the port off as there is no activity. Then the server starts up but the port takes 2-3mins to become active on the switch.

All though this isn’t a major issue it would decrease the time it takes for a reboot to complete.

I will escalate this issue to our network provider.

On the positive side the fix we used has worked correctly.