Starting on 2024-10-29 at 19:15 UTC until 2024-10-30 18:15 UTC, some customers experienced delays in data exports to S3. This incident was caused by a misconfiguration in API routing that impacted the export service. The issue was resolved by correcting the configuration used for API routing.
All Insights data exports to S3 were delayed due to errors in the export service. There was no data loss, the export service continued to retry until success. Data caught up in less than an hour after the configuration was fixed.
The root cause of the incident was a human error in configuration.
To prevent similar incidents from happening again, we have taken the following actions:
· Improved our monitoring and alerting systems to detect and report anomalies in the export service's API requests.