On January 28, 2026, between approximately 1:35 PM and 8:10 PM PDT, some customers experienced delays related to ban processing. During this time, our global indexing pipeline for ban-related data fell behind after encountering invalid customer data that caused batch processing failures. We mitigated the issue by deploying a hotfix to skip invalid affected records, allowing the indexing pipeline to catch up and normal processing to resume.

Impact

During the incident window (about 6 hours and 34 minutes), ban-related indexing was delayed. As a result, scenarios that rely on timely ban updates may have taken longer than expected to take effect. We did not observe data loss; the primary impact was delayed processing. Any bans enacted during the impact period were eventually applied.

Root Cause Analysis

A customer-created record contained an invalid value that was interpreted incorrectly during decoding (in this case, an invalid date string). Because our validation did not sufficiently enforce expected formats for certain fields, the bad data made it into the ban-related data set and caused repeated batch failures in the indexing pipeline. Those failures prevented the pipeline from making forward progress until the problematic records were bypassed.

Action Items

To reduce the likelihood and impact of similar issues, we completed the following actions:

• Deployed a hotfix to safely skip invalid records so indexing can continue even when encountering bad data.
• Improved error logging in the indexing processor to speed up diagnosis if similar processing failures occur.