Delays in Drains, Observability, and Workflow in Washington (iad1)

Timeline

1. 10 Oct · 01:37 UTC 🟢 Resolved atom

   Oct 10, 01:37 UTC
   Resolved - All services have recovered.
   
   Oct 10, 01:12 UTC
   Monitoring - Delayed Drains deliveries, Observability data, and Build logs have been processed now. We are continuing to monitor and we will provide additional updates as they become available.
   
   Oct 10, 00:35 UTC
   Identified - A fix has been implemented and delivery has resumed. Delayed Drains deliveries and Observability data are being processed. We are continuing to monitor.
   
   Oct 10, 00:15 UTC
   Investigating - We've identified an issue where some customers may experience delayed drain delivery, delayed logs, metrics, and traces, and Workflow runs that remain pending in Washington (iad1).
   
   We are investigating this issue. We will provide additional updates as they become available.

2. 10 Oct · 01:12 UTC ⚪ Monitoring atom

   Oct 10, 01:12 UTC
   Monitoring - Delayed Drains deliveries, Observability data, and Build logs have been processed now. We are continuing to monitor and we will provide additional updates as they become available.
   
   Oct 10, 00:35 UTC
   Identified - A fix has been implemented and delivery has resumed. Delayed Drains deliveries and Observability data are being processed. We are continuing to monitor.
   
   Oct 10, 00:15 UTC
   Investigating - We've identified an issue where some customers may experience delayed drain delivery, delayed logs, metrics, and traces, and Workflow runs that remain pending in Washington (iad1).
   
   We are investigating this issue. We will provide additional updates as they become available.

3. 10 Oct · 00:35 UTC 🟡 Identified atom

   Oct 10, 00:35 UTC
   Identified - A fix has been implemented and delivery has resumed. Delayed Drains deliveries and Observability data are being processed. We are continuing to monitor.
   
   Oct 10, 00:15 UTC
   Investigating - We've identified an issue where some customers may experience delayed drain delivery, delayed logs, metrics, and traces, and Workflow runs that remain pending in Washington (iad1).
   
   We are investigating this issue. We will provide additional updates as they become available.

4. 10 Oct · 00:15 UTC 🟡 Investigating atom

   Oct 10, 00:15 UTC
   Investigating - We've identified an issue where some customers may experience delayed drain delivery, delayed logs, metrics, and traces, and Workflow runs that remain pending in Washington (iad1).
   
   We are investigating this issue. We will provide additional updates as they become available.