Forum Index > Wiki News > Disabling Message Wall
Posted by 452

Now that all Special:Forum posts have been moved to the Forum: namespace, and all blog posts have been moved to the Forum: namespace, the next obvious step is to move all Message Wall threads to User_talk pages.

I've mentioned several times in the past that I regret ever enabling Message Wall, but that we were stuck with it because it would mean losing access years of messages. But now, that issue has been solved, as my Forum migration script can easily be adapted to transfer Message Wall threads.

There are several issues I must first address:

  • Renamed users - this may or may not be an issue, but I need to investigate it.
    • It's not an issue so long as Wikia correctly re-attributed the thread when the user was renamed, which means in the case of any threads created by "user452", there is definitely a problem. Obviously in this case I can automatically correct it, but who know how many other failed renames there have been.
  • Walls which only contain a welcome message - I will be deleting all non-replied welcome message threads ahead of time, and adding the welcome message to all User_talk pages ahead of time also.
    • Note: On second thought, the welcome messages are best added at the same time as importing the converted messages.
    • After deleting all welcome messages, I forgot to disable new ones, so I'll have to do that again.
New welcome messages will be prepended to all imported User_talk pages, then sometime later I'll add them to empty User_talk pages using GlitchBot.

Since there are many more Threads tthan Board_threads, this will be a significantly larger task, and may take longer to determine that there are no bugs before performing the transfer.

Also, since editing a User_talk page generates a notification, I definitely need to ensure that there will not be any problems.

With both the forum and blog transfers, there are minor issues where pages with too many revisions crashed my browser, so transferring my own message wall is likely to be an arduous task.

As I said, this will likely not happen for a while - probably more than a month. Feedback is always welcome. 452 15:08, November 21, 2016 (UTC)

As previously announced, I have begun deleting all welcome message threads, starting with the ones left by User:GlitchBot. -452 15:44, April 27, 2017 (UTC)
Due to a small oversight, I mistakenly deleted 112 messages which shouldn't have been deleted. However, I saved the database dump prior to starting, so will be able to reconstruct the deleted messages. I will be performing a full review of all threads I have already deleted to ensure there were no other mistakes. -452 15:34, April 28, 2017 (UTC)
The good news is that I didn't mistakenly delete 112 messages. The bad news is that I've found a lot of old orphaned messages. -452 16:36, April 28, 2017 (UTC)
There's still the problem of removed and/or deleted thread messages being indistinguishable from visible messages. It was easy enough to delete them in advance for the forum, but slightly harder for Message Wall threads, since there's so damn many of them - so again I wish I'd taken care of this years ago. It's really only a housekeeping issue, but it will still look pretty messy having a bunch of removed messages suddenly reappear.
One thing is for certain: I will not be manually going through finding these. However, I may still be able to do this in a somewhat automatic fashion. I have several options, not of them particularly appealing. -452 20:20, July 7, 2017 (UTC)
All removed and/or soft deleted replies hard deleted, now re-checking the same before moving on to checking for removed and deleted top level comments. -452 23:57, July 7, 2017 (UTC)
All removed and/or soft deleted top level comments hard deleted. Now re-checking all 6266 individual thread entries again, then I'll be moving on to delete the orphans created by this step. -452 13:37, July 8, 2017 (UTC)
Orphans deleted. -452 15:44, July 8, 2017 (UTC)
Today I have deleted all old User_talk pages which were blank or contained only the old welcome message. There will be many other pages which end up with duplicate welcome messages. -452 21:22, August 5, 2017 (UTC)
Despite multiple rounds of deleting orphaned messages, I've found yet more. -452 03:09, August 9, 2017 (UTC)
Cause found: my orphan matching script was only returning 1 of out any set of orphaned threads. So, there are 24 remaining deleted threads which have a total of 110 orphan children.
I believe that these 24 threads were deleted July 8th, due to the top level comment being soft-removed. Unfortunately, when hard-deleting pages in the thread namespace, Wikia does not record the deletion summary, so it's impossible for me to be certain at this point. Thanks Wikia, for yet another failure of the Thread system. -452 13:55, August 9, 2017 (UTC)
Speaking of failures of the thread system, it would appear that CLOSED Threads had exactly the same classes and appearance as REMOVED Threads, causing me to mistakenly delete them. Fuck. Unfortunately, there are some really good closed threads that I wanted to keep, I didn't close many threads, but the ones I did showed people at their worst. I will have to go through each of the 24 threads / 110 messages to see if they need to be undeleted. Normally, undeleting a thread is pointless, as the thread is not restored to the Wall, but my conversion script works with the raw pages, so there is no problem. -452 14:08, August 9, 2017 (UTC)
Okay, orphans fixed by restoring/deleting appropriate threads. Time to do the Orphan check once again. -452 14:43, August 9, 2017 (UTC)
At this time, there are no more orphans. -452 15:08, August 9, 2017 (UTC)
After numerous tests, I finally ran the entire script with the training wheels off, and it predictably crashed when processing my wall. I'm going to have to do this on a computer with more RAM. -452 15:43, July 10, 2017 (UTC)
It also crashes on at least 1 other large wall. -452 00:01, July 12, 2017 (UTC)
The problem with it crashing on my wall is due to it exceeding the javascript string limit, so all the RAM in the world wouldn't help.
On both my wall, and the other, I'll be pre-splitting them and importing separate archive pages.
I've pre-split the existing pages. -452 21:03, August 16, 2017 (UTC)
Despite my best efforts, there were a few errors when importing my wall threads from 2012. I know the cause of the problem, and from knowing the cause, I'm fairly confident there was no loss of data.
The problem occurred because of a logic issue: I was doing 6 months at a time, but I was using reverse logic to include the first 6 months in the first batch and exclude the first 6 months in the second batch. This caused threads started in June to be missing their replies in July in the first batch, and then those replies appearing in the wrong location in the second batch. I moved these to the correct location, but the history looks a little odd, especially since the edit summary contains the wrong information.
Luckily, there was only one thread which started in June and continued into July. It just happened to be a fairly important thread in the history of this wiki.
Anyway, I've changed the logic to prevent this happening in the next batches. -452 22:12, August 17, 2017 (UTC)
Speaking of edit summaries containing the wrong information, apparently in some cases the edit summary contains the wrong section info. At this point I'm not sure why. I will look into the cause, but I won't be redoing the pages I've already imported. -452 02:34, August 18, 2017 (UTC)
...well, the edit summary always contained the name of the most recent thread, instead of the thread which was being edited. *facepalm* I was even outputting that information to the console, but never noticed the mistake. It will be fixed going forward, and I will look into redoing what I've already imported. -452 02:49, August 18, 2017 (UTC)
The main reason I didn't want to redo what I had done was because I thought that separating the 2012 archived user_talk page from the message_wall thread would be too difficult, but it wasn't. There were probably a few maintenance edit entries lost, but the content of the edits themselves was carried over by keeping the recent revision from after the 2011 split.
Anyway, all previously done has been redone, properly this time. -452 03:07, August 19, 2017 (UTC)
I have now found a major issue that previously went unnoticed during the blog transfer: comment replies in some situations were moved to the wrong places. This issue wouldn't have occurred very often with blog comments, but would happen very frequently in Message Wall comments. I will be re-reviewing all transferred blog comments to try to find and fix other cases, and I will be improving my script to deal with these. -452 03:24, July 11, 2017 (UTC)
This issue was present on 26 out of 324 transferred blog pages, which have now been fixed. -452 18:29, July 11, 2017 (UTC)
Okay, problem found and fixed: I was using the wrong method to sort the threads, resulting in some replies being out of order. I'm now using the correct method, and will continue doing extensive verification to ensure there are no other errors. -452 00:01, July 12, 2017 (UTC)
Counts after deleting all unreplied welcome messages:
  • Threads: 2151
  • Replies: 4285
-452 17:51, April 28, 2017 (UTC)
Counts after deleting all soft removed/deleted messages:
  • Walls: 1084
  • Total threads: 6175
  • First message: 2056
  • Replies: 4119
452 14:42, July 10, 2017 (UTC)
Counts today:
  • Walls: 919
  • Total threads: 6070
  • First message: 1892
  • Replies: 4178
452 15:15, August 9, 2017 (UTC)
Walls with replies: 414
452 22:17, August 15, 2017 (UTC)

Apparently Wikia have made some unannounced backend change to prevent pages moved out of the Thread namespace. This isn't a problem for the Message Wall retirement, as I wasn't going to be using the move function, but it's just one more reason to go ahead with this. -452 18:47, July 7, 2017 (UTC)

They have also prevented editing pages in the Thread namespace through the API, blocking my auto-file-renamer from automatically updating those pages. Wikia seem insistent on making Message Wall worse and worse. -452 21:56, July 29, 2017 (UTC)
This also prevents null editing. *facepalm* -452 22:25, August 4, 2017 (UTC)
And on top of that, they've also removed the edit button from talk page archives. -452 17:26, August 1, 2017 (UTC)

New slightly major problem: Until now, I've just been testing the transferred wall posts themselves, but I've now added the "retrieve and prepend existing user_talk pages" part... but the trouble is that MANY user_talk pages have had linkfixes and such during the intervening years, so just importing the threads with the current user_talk page prepended will mean that some changes appear in the history before they were actually made, and that when the edits actually do appear in the history, it will look as though I removed the imported threads in those edits instead of making the fix itself... and when multiple fixes have been made to the same page, it would appear that I've actually added the later issues in those edits.
But the most major problem this causes is that IF the most recent edit to a user_talk page happened SINCE the last thread, the thread will not be present in the current revision, as only previous revisions would be imported.
I normally find that the act of describing a problem inspires me to immediately think a solution, but this time isn't helping. I'm more just thinking of other side-effects.
At this stage, I think the most effective solution would be a duplicate final history entry with today's date. -452 22:24, August 7, 2017 (UTC) (edited 22:28, August 7, 2017 (UTC))

I have implemented this solution, only for those pages which have an old talk page. Fun fact: there are only 85 users who have both a user_talk page and threads on their wall. This number would be larger if I had not already deleted all pages/threads containing only a welcome message. -452 18:16, August 9, 2017 (UTC)

I almost forgot to omit bot edit sigs again. Right now, everything is about ready to go, I've imported the output to a test wiki, and i'm randomly inspecting dozens of pages, looking for formatting errors. -452 00:20, August 11, 2017 (UTC)

Minor issue: In threads where the second comment is not a reply to the first comment, but a later comment on the same topic, the indenting looks kinda weird.
For example: an admin issuing a warning, followed by a second admin using the same thread for a follow-up warning appears as though the second admin is speaking to the first admin and not the user. -452 21:29, August 11, 2017 (UTC)

Manually correcting these confusing instances isn't feasible, as there are over 400 walls with at least one reply. -452 22:17, August 15, 2017 (UTC)

Ugh, yet another downside to Message Wall: people using useless HTML in their comments was not immediately noticeable, so there's a bunch of bullshit in the converted pages. -452 14:30, August 18, 2017 (UTC)

After all Threads have been transferred and deleted, I will also be importing 6000 redirects to honour existing Thread links, I will then replace all of those thread links, and delete the redirects in 1 year. -452 16:49, August 20, 2017 (UTC)

Right now, I have no idea how many thread links there are, but once the redirects are imported, they will be displayed on Saints Row Wiki:UnusedRedirects. -452 16:51, August 20, 2017 (UTC)

All pages imported, only 1 error during importing, formatting errors found and fixed on 82 pages, no pages left blank... a better outcome than expected, as I'm used to a lot more unforeseen problems. It looks like spending extra time testing really paid off.
917 Walls imported + 1 reimported + 82 updated = 980 imports, just like the log says. (2 split Walls imported previously).
That's it, the hard part is done. Unfortunately, 1 person saw the "new messages" notification and saw fit to leave a reply on a 5 year old thread - which is fine, just annoying as it means I have to re-import that thread now. -452 18:19, August 20, 2017 (UTC)

I am now disabling Message Wall, hopefully for the final time. -452 18:20, August 20, 2017 (UTC)

There was one more import I almost forgot: my 2017 wall threads.
Headers updated, now importing thread redirects. -452 18:39, August 20, 2017 (UTC)

At this time, I'm going to leave all Thread pages intact, as there's no urgency to delete them. This way, if errors are found, it will be easier to investigate and repair. They'll be deleted sometime before the end of the year. -452 19:19, August 20, 2017 (UTC)
6091 thread redirects created. For some reason, these are still being detected as being within the "thread" namespace, which doesn't exist anymore. "User_blog" doesn't exist, but "Thread" and "Message_Wall" still do. The upside is that his will make them easier to delete. -452 19:53, August 20, 2017 (UTC)
Leaving Thread pages for now has helped with the link cleanup, as I'm still able to look up the pages.
However, while cleaning up Thread links, I've discovered at least one case where some replies were mistakenly deleted back in April because the user had replied to the welcome message. I had tried to exclude threads with replies from being deleted, but I guess I missed some. I could restore the conversation, but it's of no great importance, and since it was originally misposted on a talk page, I've restored it there. -452 16:02, August 21, 2017 (UTC)
While Special:WantedPages correctly contains red links for now-non-existing Threads (Which SHOULD have been listed on WantedPages long ago, but there was no destination-checking for Thread links), technically the Message Wall pages themselves still exist, and have to be deleted manually before they will appear on WantedPages.
The only reason to not delete Message_Wall pages immediately is the possibility that I might need to reactivate Message Wall for some reason, but there shouldn't be any reason, as I'm able to determine Thread destinations already. For now I will continue cleaning up Thread links, and return to this later. -452 16:11, August 21, 2017 (UTC)
After removing all links to threads, and manually checking a many user_talk pages for errors, I've now deleted all threads, as I said. I still have a full thread namespace export as a backup.
All that remains now are thread redirects, which will remain in place for a year or so, and display a message that the origin of the link needs to be updated. -452 16:14, September 19, 2017 (UTC)
It has been a little over a year already, deleting thread redirects now. -452 19:28, October 29, 2018 (UTC)
Deleted pages without an edit summary are supposed to display a "content before deletion was []" message, but that is not happening with these. But it's not particularly important. -452 19:35, October 29, 2018 (UTC)

I have discovered a small problem: When automatically deleting old user_talk pages prior to merge, I mistakenly deleted some anon user_talk pages. I should have checked the number of contributions or the size of the page, but apparently didn't. I likely only checked that the final edit to the page was the "triggering anon notification" edit by GlitchBot.
This issue will have only affected anon users who edited BOTH before AND after 2012-01-27 02:43, in other words: long term anon editors, of which there are very few, and the majority of the ones that are are vandals. So, the only thing that would have been lost is various warnings and block messages for vandals, as well as occasional questions for where anons got their information, which were rarely answered anyway. (I uncovered this while examining some old anon blocks for the purposes of compiling anon aliases.)
I already wrote a database-dump parser to extract anon and users for the purpose of verifying welcome messages, so I can modify that to give me anon edits by date. I can use this to compare "anons who edited before Message Wall was enabled" and "anons who edited after Message Wall was enabled" and hopefully the intersection of those lists will be few enough to manually restore and revert.
Yet another unexpected Message Wall complication. -452 18:38, June 20, 2018 (UTC)
Only 98 pages to manually check. -452 20:54, June 20, 2018 (UTC)
Only 15 pages were mistakenly deleted, they have now been restored and merged. -452 22:05, June 20, 2018 (UTC)
Unfortunately, I just reviewed a random selection of another 100 deleted user_talk pages, and found another 3 that shouldn't have been deleted. I messed up, because I had completely forgotten that I had had GlitchBot trigger a notification for all anon users in 2015, largely because anons could not see highlighted thread notifications.
To remedy this, I will write a script to scan all the user_talk pages of all 2000 anon pre-2012 contributors, and manually check all pages which have more than 2 revisions (All have wikia welcome + GlitchBot notifications)
A big thanks to the persistent multi-year multi-ip vandals, because I likely wouldn't have noticed this without them! -452 22:30, June 20, 2018 (UTC)
According to my own deleted contributions, there are more than 150 user_talk pages which need to be restored. -452 22:38, June 20, 2018 (UTC)
There are 219 deleted user_talk pages with more than 2 revisions. Some of these are possibly a link being removed, but the majority will probably need to be restored. -452 00:20, June 21, 2018 (UTC)
Of those 219, 111 contained non-welcome-message content, and not all of them were from me. The reason it is less than 150 were because some of them were just a link being removed, as I previously suggested.
I have, however, uncovered 1 additional as-yet-unresolved problem: I generated the list of anon user_talk pages from the full database dump. The full database dump doesn't contain deleted revisions, which means that if a user had no extant revisions, I didn't check their user_talk page. I generated them this way so I would know when they first edited. I will have to now cross-check the list of anons with extant contributions against the list of all user_talk pages, and check the remainder. -452 01:59, June 21, 2018 (UTC)
Auto-checked another 555 user pages and found another 236 to manually check. That seems unusually high, given that the last batch was only 219, but it's entirely possible that these users created poorly named pages which were deleted, and were left a message about it. -452 02:19, June 21, 2018 (UTC)
Because I knew from last time that >50% would be not restored, I adapted my script to just show me the summaries instead of having to visit the undelete pages manually. I was half-right: the majority of the user_talk pages were edited to remove the link to the page they created which was deleted.
Of those 236, only 7 should not have been deleted.
Unfortunately, I know I'm still not done. I have 1 example that I somehow missed during the previous steps, so I'm going to dump the deleted contribution summaries of all user_talk pages, just to be thorough. -452 02:43, June 21, 2018 (UTC)
Well that was much more efficient. A final 16 deleted user_talk pages have been restored. -452 03:26, June 21, 2018 (UTC)
Community content is available under CC-BY-SA unless otherwise noted.