Firewall Incident on Legacy Sydney Datacenter

- Thursday, January 27, 2011

Our engineers are currently checking that we have stabilized an issue that caused a significant period of downtime for the Adobe Business Catalyst in the legacy Sydney Primus Datacenter today. I'm posting to give an explanation of what happened and will keep you updated as more details come through:

  • Issue: Business Catalyst clients were unable to access FTP and Admin Consoles for all sites hosted on the legacy Sydney Primus Datacenter. Webpages for these sites were being served very slowly with response times of up to 4000ms
  • Time of Incident Start: 26 Jan 2011 1:48AM US Pacific Time
  • Time of Incident End: 26 Jan 2011 7:00PM US Pacific Time
  • Technical Action: The Systems/Operations team have verified that there was a failure in the primary firewall for the datacenter causing heavy packet loss. We have switched over to the backup firewall which has restored services.
  • Vendor Action: No vendor action required
Although the situation is currently stable, our engineers are continuing to monitor the datacenter closely. There will be a follow-up blog post with actions taken to ensure we improve reliability for the legacy Sydney DC after the team performs an incident 'post-mortem'.