Forum Index > Wiki News > Lazy loading and the future of blogs
Posted by 452

When lazy loading was introduced, search engines lost the ability to see blog comments.

I unfortunately didn't notice this until last year. I immediately asked Wikia Staff if this was intentional, they said they would look into it, but never got back to me.

I recently remembered this issue again, and asked for a follow-up, and a different member of Wikia Staff replied, instead of the one who was looking into it. They also said they will look into it, but that "I think that, for now, we're OK with this.".

Wikia Staff have again failed to realise that it doesn't matter what they're okay with, it's what the users of the site are okay with.

It is unacceptable for blog comments on the Saints Row Wiki not to be discoverable via search engines, as there is a great deal of content in blog comments.

I've now informed Wikia Staff if they neither fix the problem, nor disable lazy loading here, then all blogs will be moved, and the blog namespace will be disabled.

Does anyone have any questions, comments, suggestions for alternate implementations? 452, 2016-09-27T18:37:28Z
(last edited: 452, 2016-10-03T19:41:08Z)

Forum migration is now complete, the next task is migrating blogs.
Blogs are structured slightly differently than forums, so I'll have to make some updates to my migration script.
There are approximately 10 times as many Blogs and Blog comments as forum threads and thread comments, so it will probably end up being a little more complicated.
I will start by moving Blog posts in Category:Wiki News and Category:Game News first, and then move on to the rest in batches.
But I have a little bit of coding fatigue, so I'll likely be putting this aside for a little while. -452 21:41, November 5, 2016 (UTC)
  • Game News: 21
  • Wiki News: 41
  • Old help topics: 36
  • To-do topics: 102
  • Misc blog posts: 151
-452 16:13, November 6, 2016 (UTC)
  • Posted by me: 181
  • Posted by others: 165
Update: first I will be moving the 62 news posts, then the old help topics, then all blogs posted by other people, then the to-do topics, the the rest of my own. Some of my own blog posts will go back to being usersubpages. -452 19:48, November 6, 2016 (UTC)
Update: But very first, I'll move blog posts with no comments, because there's no additional processing necessary. -452 19:57, November 6, 2016 (UTC)
It turns out that I was overthinking the problem, and the existence of subcomments really doesn't make a difference. -452 22:39, November 19, 2016 (UTC)
Edit: It turns out that it did make a difference, but the issue didn't occur very often. -452 03:30, July 11, 2017 (UTC)
Moving in the aforementioned batches requires additional modification to the script which is really unnecessary at this point, so I'll just be doing them all at once. -452 23:28, November 19, 2016 (UTC)
After extensive testing, I've uncovered a previously unnoticed javascript sorting problem. Apparently sorting "1, 20, 100" resulted in "1, 100, 20".
This situation was triggered if an earlier reply in the thread was edited much later than the others, and had a revision ID which was longer.
The result of this bug is that the intervening edits would be lost, which was luckily glaringly obvious in my test data.
Apparently this bug wasn't a problem for the forum transfer script: I've double-checked all imported threads, and there's all good. Either the bug wasn't present, or I just got lucky and it was never triggered.
This is the purpose of extensively testing the scripts, of course, once this bug is dealt with, I should be ready to import.
-452 17:40, November 20, 2016 (UTC)
Just to be safe, I manually checked the page history of 123 pages to verify the problem was no longer happening. Importing all non452 blog posts into the forum namespace now. -452 21:40, November 20, 2016 (UTC)
With the forum transfer, I deliberately didn't include edit summaries, as they were not used for the forum - however, they were used for blogs. So I'll be deleting the imported pages and reimporting them with edit summaries. -452 22:50, November 20, 2016 (UTC)
Transfer done, with comments.
I've just manually inspected the history all all transferred blog posts to verify that there were no accidental blankings.
In hindsight, I should have omitted the "last edited by (bot)" sigs.
Next steps:
  • Update all posts in Category:Unsorted
  1. Change all blog posts to redirects - done
  2. Export all blog post redirects - done
  3. Delete everything in the user_blog and user_blog_comment namespaces - done
  4. Disable the blog feature - done
  5. Import the previously exported redirects so that legacy links are honoured. - done
    • These redirects will be kept for at least one year.
  6. Wait for the Special page cache to update, fix all double/broken redirects - waiting
  7. Gradually update internal links to blog redirects to point to the forum post.
-452 15:20, November 21, 2016 (UTC)
Something which I wasn't anticipating is that Special:WhatLinksHere does not properly handle when a namespace is disabled, and is currently reporting that there are no links to any user_blog pages, when I know that there are.
If I had known, I could have made a list after step 1.
As it stands, I would have to massnulledit all pages, which I don't really want to do.
It is possible, but unknown for sure, that the special page cache update may refresh all the links. Or it may not.
Another option is to re-enable blogs, re-import the user_blog redirects, get the list of links to those redirects, delete the redirects, disable blogs again. I was thinking of doing that, until typing out those steps.
The other other way is going through the database dump and finding all the links, which I think is the way I'm going to have to do it. -452 18:25, November 21, 2016 (UTC)
The special page cache update did not refresh all links, it simply added a bunch of broken links to WantedPages which shouldn't be broken due to the presence of the redirects. I had already made a list of pages which needed to be purged, but I wanted to see if the issue would resolve itself first. Purging now, hopefully those broken links will disappear in tomorrow's special cache update. -452 11:05, November 22, 2016 (UTC)

Purging all pages manually worked. I deliberately left 2 links unpurged, and apparently I missed a third. I've now fixed two, including the one I missed, and have left a final one.
Also, in hindsight, I should have added a "Comments" heading below each first post to allow easy section editing for long pages. -452 09:14, November 23, 2016 (UTC)
It wasn't specifically mentioned here, but all of the User_blog redirects showed a notice that the directs would be deleted on 2018-01-01 - I forgot about that, but have now deleted them.
Anyone arriving at the wiki using a User_blog link, such as will now be shown the deletion message, including a link to the Forum post. -452 05:08, January 23, 2018 (UTC)
Community content is available under CC-BY-SA unless otherwise noted.