APIs unstable and slow issue
Incident Report for Easyship

In this incident, users encountered the API unstable and slow issue.

All times are UTC.

20:44 A new deployment on the production site.

21:30 Monitoring Servers report a check failed issue to the DevOps team.

21:35 Developer investigates the issue.

21:46 DevOps found an error from Core API.

21:51 DevOps found Database performance slow and all API latency is very high.

22:39 DevOps decides to rollback to the previous stable version.

22:44 All services are back to normal.

Root cause:

The root cause is the new migration codes caused a lot of locks on tables and then slow down all API calls.

Posted Nov 13, 2020 - 01:37 UTC

This incident has been resolved.
Posted Nov 12, 2020 - 22:43 UTC
Core API check failed
Posted Nov 12, 2020 - 21:30 UTC
This incident affected: API (Core API).