Hey tropers, we are about to undergo a large update to the tvtropes' database to convert the encoding. During this time the login system will be offline. You'll still be able to read TVTropes but you won't be able to post or edit anything during the conversion.
How long will the login system be down?
My most recent test run took 9 hours (my first test took 20+ hours so this is after a lot of optimizations to get down to 9 hours). ***it will now take 11 hours. see update below
Can you give me more details?
TVTropes was originally hosted on Windows' servers back in the day and all content on the site is encoded with Windows 1252 (a superset of ISO-8859-1, aka latin1).
According to the W3Tech, only 1.3% of internet traffic is ISO-8859-1. I'm guessing a large amount of that is TVTropes considering we have millions of pages defined with that charset.
The majority of the internet is encoded in UTF8. By doing this conversion it will bring us up to modern standards with the rest of the web and allow us to more easily support other languages and icons. It will also allow us to use more modern tools to help with editing such adding a WYSIWYG editor option. It will also help with code development as we often have to add special workarounds to continue to support this long deprecated charset.
Are we changing anything else?
While we have the login system offline, we are going to upgrade the edits history database table to include a sequence number. We've always wanted to do this but it could not be done while the site was online. That table has 40M+ rows of data. With this change we'll be able to make it so we can easily jump to any page when filtering to edits from a specific page or from a specific user. We have this for the forum and that's why you can jump to page 500 for example of a long running thread and not have a long delay.
When will this happen?
UPDATE: (take two) This is now scheduled for Mon Dec 5th at 8:30PM PST until Tue Dec 6th at 7:30AM
At 8:30PM PST you will be logged-off of TVTropes so make sure to save anything before that time. Once the conversion is done the next morning, I'll point the code to the new server and everyone should be auto logged in. If not, when you see the all clear announcement at the top of the page, go ahead and try logging in again.
The process will take approximately 11 hours. I had it down to 9 hours until I found examples of utf8 encoded values inside latin1 columns so I had to add some extra testing to ensure that data doesn't get double encoded in the process.
If you have any issues during the migration please send an email to thestaff@tvtropes.org
UPDATE: (complete) The migration is officially complete. It ended up taking nearly 12 hours but we got there. All data on TVTropes (hundreds of millions of rows of data) has now been converted from Latin1 to UTF8. We now have the same encoding as 99% of other websites and can support special characters and other languages. It will also allow us to build other tools such as a WYSIWYG editor.
Edited by itcdr on Dec 6th 2023 at 8:48:40 AM
Redmess
Not sure if it was overlooked, but when I check page histories, I noticed it no longer colours changed/deleted text.
Optimism is a duty.
Big Catch, Sparkle Edition
It does on my end?
Currently Working On: Incorruptible Pure Pureness
Lost in Space
The fix for page history diffs has been added. Not sure if that also fixed the highlighting.
"It's Occam's Shuriken! If the answer is elusive, never rule out ninjas!"
Heh, Comic Sans... I'm wondering why they didn't change to Futura just now...
Giving this monospace problem another thought, it is entirely possible that when itcdr saw the editor, he might of done it on the server — where monospaced complete unicode fonts are rarely if ever installed —
I just switched to monospace on the clientside and it works like a beauty.
AFK with issues, will return
I just pushed a hotfix to decode html entities before doing the diff comparison for the history. For those of you that have seen the history showing a bunch of non-changes (ie: nothing highlighted). That was because special characters were forced to be html entities but aren't anymore so the program thought the text was changed even though when displayed it looks the same.
Let me know if you still see issues with this. Hopefully my fix filters these out so it doesn't flag a bunch of changed lines with special characters even though nothing visually changed.
uwu
Editing forum posts still has the old "edit mode" font, but editing the wiki feels like editing the Page Source. Hopefully the old edit font will be restored since the new one feels a bit too cluttered to me.
THE GOLDEN AGE WILL RETURN AGAIN!
Many thanks for the efforts you devs are putting into all this!
But is the monospace font such a big issue that requires a toggle? I simply changed the css on the clientside from sans-serif to monospaced and its fine.
AFK with issues, will return
Lost in Space
![[up]](https://static.tvtropes.org/pmwiki/pub/smiles/arrow_up.png "[up]"){kind=icon}
Moar options, moar better.
But I guarantee you that 99 percent of TV Tropes' userbase is not savvy or dedicated enough to make client-side CSS changes.
"It's Occam's Shuriken! If the answer is elusive, never rule out ninjas!"
Big Catch, Sparkle Edition
I was barely savvy enough to help with the tests as an engineer because I wasn't sure if I was running the right kinda tests ![[lol]](https://static.tvtropes.org/pmwiki/pub/images/lol2v12_4863.png "[lol]"){kind=small}
Def can't function with anything more complicated than basic markup or code.
Heh,
Friends forevermore
I'd prefer if the monospace font also shows up in history if the toggle is enabled.
(And speaking of history, long history pages e.g. Funny.TV Tropes are still broken.)
I had a dog-themed avatar before it was cool.
Okay, now I'm seeing the CloudFront/CloudFlare error
.
Edited by skewview on Dec 6th 2023 at 9:09:49 AM
AFK with issues, will return
Big Catch, Sparkle Edition
To be clear, my error seems unrelated to this since it had nothing to do with me seeing long history pages. The edits just kept triggering an error page.
Friends forevermore
It's a standard 504 error
◊, the same one I've been getting on that page for quite a while.
(EDIT: If it's relevant, Firefox on Windows.)
Edited by bwburke94 on Dec 6th 2023 at 4:32:49 AM
I had a dog-themed avatar before it was cool.
Thanks for the monospace toggle, but the switches are acting a little wonky for me. The actual setting itself is remembered correctly, but toggle switches act as if "Off" means "whatever value the setting was when the page loaded" and "On" is "whichever value it wasn't when the page loaded". I'm seeing this same behaviour on both my profile page and on the edit page.
Well, I was, but now it's gotten worse. Not sure if it's something I did or something changed with the software, but while I was writing this the behaviour reversed. Now the toggle position is correctly remembered as on or off, but I'm getting the default font no matter what the setting is instead of switching between default on monospace.
Redmess
Now that I am looking at it, the old font is a lot thinner and harder to read than the standard one. Can that be fixed?
Optimism is a duty.
Is there a way to make the monospace font toggle also apply to page histories?
"Squid has to go to market. He's had to go to market for as long as he's sucked water."
Admin
Just pushed up a fix for the toggle and working on adding it to article history pages rn.
If you're still having issues, be sure to hard refresh and clear your cache and then let us know if you're still having problems.
EDIT: Okay just pushed up the changes for article history too. If you toggle the Monospace font button on your profile or the editor it will also change article history font.
Edited by kory on Dec 6th 2023 at 2:15:42 AM
Now monitoring Wishlist and Bugs
Looks good, no problems that I can see. Great work everybody!
This has been the best Saint Nicholas in recent memory! 🥳
Now off to sleep!
AFK with issues, will return
Professional Procrastinator
Does this allow you to fix that bug I reported ages ago where internal links get a little weird (i.e. broken) in things like the discussions pages each article has? It'll take me some time but I can probably find the bug report in question. Mind you it was only happening on Firefox in the first place, from what I recall, but I'll still be quite grateful if it no longer happens *ever*.
Christian, gamer, programmer, brony, and quadriplegic (paralyzed mid-thorax down). I am filled with determination...
Was .difftype intended to be included in the monospace? It's not actually part of the diff like .diffadd and .diffdel and it looks weird for it to be different from the otherwise-identical .reason heading.
Suddenly I'm imagining an April Fool's joke where everything becomes Comic Sans for a day![[lol]](https://static.tvtropes.org/pmwiki/pub/images/lol2v12_4863.png "[lol]")
Currently Working On: Incorruptible Pure Pureness