The transient errors that were affecting LeMUR have now been resolved and regular service has been restored. We are closely monitoring recovery to ensure continued good performance.
Posted Oct 12, 2023 - 03:00 UTC
Update
We are continuing to work to address the issues that are causing transient errors every ~10 minutes. The majority of LeMUR requests are still completing successfully so if you do experience a failed request and a 500 error we suggest retrying the request as it will likely be successful. We will continue to provide additional updates as we learn more.
Posted Oct 11, 2023 - 21:58 UTC
Update
We are still working to resolve the issue causing the transient spike of errors every 10 minutes that is affecting some LeMUR requests. A potential fix was deployed but it did not resolve the issue so we are continuing to investigate. We will continue to keep you updated as we learn more.
Posted Oct 11, 2023 - 20:07 UTC
Identified
We are still investigating issues affecting LeMUR. These issues are causing transient error spikes which seem to be occurring at 10-minute intervals. We will continue to provide updates here as we get more information from them.
Posted Oct 11, 2023 - 18:39 UTC
Update
We are still working to identify the root cause of the issue causing the increase in 500 errors but we are still seeing the large majority of requests to LeMUR completing successfully. We will continue to provide updates here as we learn more.
Posted Oct 11, 2023 - 18:13 UTC
Investigating
We are currently investigating an increase in the number of 500 errors we are seeing returned from LeMUR. The vast majority of LeMUR requests are still completing successfully but we are working to investigate the higher-than-usual 500 errors. We will share additional updates here as we learn more.