Elevated API errors and delayed async processing
Incident Report for Orb
Resolved
This incident has been resolved.
Posted Oct 30, 2024 - 05:44 UTC
Update
API services and asynchronous processes are stable.

The query load issue has been fully isolated and we're working on improvements to prevent this issue in the future.
Posted Oct 29, 2024 - 19:39 UTC
Monitoring
API services stabilized at 12:30 UTC and async processing has been restored, and we are continuing to monitor our systems.
Posted Oct 29, 2024 - 14:12 UTC
Investigating
A query load issue has been identified starting at 2024/10/29 8:45 UTC which was causing intermittent spikes of errors accessing internal data storage, resulting in API errors and delays in async processing.

This issue has been mitigated for the majority of customers as we continue to investigate.
Posted Oct 29, 2024 - 13:09 UTC
This incident affected: API (Reads, Writes, Analytics).