An issue was found whereby certain searches might fail due to a permission error. The issue has been resolved.
Posted Jun 30, 2021 - 21:47 UTC
Verification is currently underway for the maintenance items.
Posted Jun 30, 2021 - 21:38 UTC
An issue with node auto-discovery in the deployment script required manual intervention, resulting in a partial outage that lasted several minutes longer than anticipated. The cluster is now back online and all indices have recovered successfully. Further changes still need to be made but are not expected to result in outages.
Posted Jun 30, 2021 - 20:39 UTC
Scheduled maintenance is currently in progress. We will provide updates as necessary.
Posted Jun 30, 2021 - 20:16 UTC
We will be changing some settings on our Elasticsearch cluster that will require a full cluster reboot instead of the usual rolling reboots. This means Elasticsearch will be offline while the change takes effect. Various parts of the website will return errors while this reboot takes place.
We're aiming for a single partial outage lasting no more than a minute. Due to discrepancies between the testing environment and the production environment, we anticipate that the changes may need to be revised or reverted, resulting in additional short outages.
In either of these scenarios, partial outages will result in any features that rely on Elasticsearch returning errors, while other features may perform slower than usual. Features that will be offline include search, various recent activity feeds, and the some portions of member pages.
There is a small possibility that the search index reach an inconsistent state; this would result in incomplete search results for about 24 hours while the index rebuilds.
Posted Jun 30, 2021 - 19:37 UTC
This scheduled maintenance affected: Forums, Email (SES: Invalid email handler), and Server infrastructure (Search database).