UK – Maintenance Notice 24/09/2009

Closed.

Brief: We are making improvements to our DNS infrastructure to increase capacity, speed and redundancy.

Timeframe: Thursday 24rd September 2009 between 22:00 and Friday 25th September 2009 01:00 ETA 1800.

Expected Impact: The control panel will be offline during the maintenance window and DNS servers will be running slower than normal while data propagates.

What you need to do: All customers are advised to check their sites availability and functionality after this maintenance window. While we will endeavour to check all sites, this is a time consuming task and user reported issues will be dealt with as the highest priority.

As ever, please feel free to contact us if you have any questions or suggestions via the control panel support system.

Updates:

22:00 – Process started.
23:30 – The control panel is still available and the update is running fine, just a lot slower than expected. We’re working on increasing the speed of the update but at this point it is more likely that this maintenance window be extended until 06:00. No services are affected by this maintenance at present.
00:50 – The update is continuing well.
08:30 – The update is continuing well and will be left running.
11:00 – The update is … still going, we’re at M in an 0 to Z run-down. It’s going much slower than hoped, but it is working and there’s no outage or noticeable slowdown caused by this work.
14:10 – The update continues without issues. There was around 120 seconds while the pipeten.co.uk DNS zone got deleted and rebuilt where you may of had problems resolving the DB or mail servers but only for those few seconds.
22:00 – So that was a long 3 hours…

In all it’s taken 23 hours to rebuild all of the name server zones across the three nameservers. We had no DNS outages other than the expected 120 seconds per zone/domain. Unfortunately due to the time of day the pipeten.co.uk. zone was rebuilt (not at midnight as planned) some customers requesting mail.pipeten.co.uk received an invalid reply which was then cached by their ISP’s recursive nameserver and gave the impression of an email outage where non existed. This maintenance period has now closed, all services are working as normal and new blog posts will be created in the coming days about changes customers can make to take advantage of the new ns0.pipeten.co.uk nameserver.