We are once again seeing normal processing times for our Async `/v2/transcript` endpoint. We will continue to monitor traffic going forward to ensure continued good performance but at this point, the issue with AWS and limited GPU capacity affecting our traffic has been resolved.
Posted Oct 25, 2023 - 19:46 UTC
Monitoring
We have been able to get increased GPU capacity from AWS as they continue to work to address the GPU capacity issues they are experiencing. We have also deployed some of our models onto other hardware and are now seeing improvements with processing times going down. We will continue to monitor this improvement and provide additional updates once we have returned to normal processing times.
Posted Oct 25, 2023 - 18:17 UTC
Update
We are continuing to stay in contact with AWS while they work to address the GPU capacity issues they are experiencing, which is the root cause of the slowdown we are experiencing. Our Engineering team is also working in parallel to deploy our models on other hardware to improve performance while AWS works to resolve the GPU shortage issues. We will continue to provide updates as this work continues.
Posted Oct 25, 2023 - 16:33 UTC
Identified
We are still working to resolve the issue that is causing slower-than-usual processing times, which has led to some requests being throttled but we have identified that this slowdown is caused by an AWS GPU capacity issue in US-WEST-2. Our Engineering team is in communication with AWS support so that we can work to get this slowdown resolved as quickly as possible. We will continue to provide updates as soon as we learn more.
Posted Oct 25, 2023 - 15:48 UTC
Update
Our team is still working with AWS to identify the root cause of the issue causing a slowdown in processing times.
Posted Oct 25, 2023 - 15:24 UTC
Update
Investigating - AWS is experiencing a server issue which is negatively impacting the processing time of our Async API. This may also lead to some increased throttling. Our team is currently investigating the issue.
Posted Oct 25, 2023 - 14:58 UTC
Investigating
AWS is experiencing a server issue which is negatively impacting the processing time of our Async API. This may also lead to some increased throttling. Our team is currently investigating the issue.
Posted Oct 25, 2023 - 14:52 UTC
This incident affected: APIs (Asynchronous API) and AWS (Database).