Forum Down
Incident Report for Wacky Writers
Resolved
We have successfully migrated our images, our server, and our SSL certs, which were the big reasons we were having so many problems over the weekend. Some images may continue to be sporadically available as our server catches up with displaying everything (it's got about 500k processes it's running right now). But since the big issues are fixed and the site is now fully operational, I'm marking this as resolved. Thank you all so much for your patience.
Posted Apr 11, 2021 - 17:47 CDT
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Apr 11, 2021 - 13:28 CDT
Update
The forums will be in and out of READ ONLY mode for a few hours.
Posted Apr 10, 2021 - 12:47 CDT
Update
The forums will be going through another round of fixes today. I will put the site into read-only each time so nobody is taken by surprise, but expect the forums to be available off and on sporadically today.
Posted Apr 10, 2021 - 07:56 CDT
Identified
We are finally back up, sorta. We lost a couple days of data, but the site itself is back up. A little worse for wear and limping a bit, but it is semi-functional again. You'll see that images aren't loading, I'm working on that, but I have to reindex all posts (of which there are nearly 500k whoa) to get those images working again. I'll have to run another round of maintenance this week probably to get things back to close the way they were, but for now, you can engage again. Thanks again for your patience.
Posted Apr 09, 2021 - 20:20 CDT
Update
I am still in touch with Discourse about why our forum won't come back up. For a little more information, we needed to run a bunch of updates, one of which caused a variety of errors to fire and the Discourse application to ultimately fail. As a result, we had to completely reinstall Discourse to get some semblance of it back up and running, which of course meant we were working with a blank slate (which is why if you happened to get into the site at some point today, you may have seen a "user does not exist" error). So what we're trying to do now is restore the site from a backup made on April 6th (meaning data from the 7th and 8th will be lost), but for some reason now the server is not recognizing that Discourse is installed at all, which is why the URL is down hard right now. We do have a backup of the database, as well as all of your uploads (even through the 7th and 8th), we just can't seem to get something to connect the two of them. Thanks for being patient, I had no intention of this being down so long.
Posted Apr 09, 2021 - 15:56 CDT
Investigating
The forums are currently down as a result of maintenance upgrades. We're unsure when they'll be back up, but we're hoping for sometime this weekend. In the meantime, feel free to hang out in our Discord server. https://discord.gg/Wf66QVkY We'll keep y'all updated as frequently as we can. Thanks for your patience!
Posted Apr 09, 2021 - 08:19 CDT
This incident affected: Forums.