Backend servers rebooted
Incident Report for Presslabs
Resolved
This incident has been resolved.
Posted Nov 13, 2020 - 00:24 EET
Update
All sites are back up. We are continuing to monitor the situation.
Posted Nov 12, 2020 - 23:43 EET
Update
We are continuing to monitor for any further issues.
Posted Nov 12, 2020 - 22:55 EET
Monitoring
The sites are recovering and we are currently prioritizing live sites so dev sites might take some time to recover. Thank for your patience!
Posted Nov 12, 2020 - 22:54 EET
Update
The source of these issues was a latent bug, which quickly scaled down to 0 than back up the backends. We've reverted the faulty deploy and we are waiting for the sites to recover.
Posted Nov 12, 2020 - 22:37 EET
Identified
A set of cascading events led to backends being restarted. This in turn puts a heavy load on the git service. We are working on reducing the load on the git service so that the backends can be restarted.
Posted Nov 12, 2020 - 22:30 EET
This incident affected: WordPress admin on Kubernetes (Europe, US) and Git service.