AI policy discussion

Starting the thread as it's been long overdue.

Let me summarize what I remember from various threads.

• QNA 1: Are works made by AI tropable?
  • Unless the US law starts saying otherwise, they'll be treated as any other online content. And someone has to provide input, proofread and publish these works, so creative rights are technically assigned to that person and not the AI. Also see AI Generated Works for examples.
• QNA 2: Is using an AI assistant okay on TV Tropes?
  • No more than using a Nannybot-like grammar plugin or a script to automate editing. If it makes messy edits, the editor in charge will face full responsibility. If it doesn't, good, but still very not recommended as it can't be reliable. Also, as of currently, it's very unlikely that ChatGPT-like software, despite all the current use-cases, is even capable of troping works or posting on the forums like a troper would. If it can, it'll still be a subject of following the guidelines of style and accuracy.

Mods are free to correct this.

Edited by Amonimus on Apr 20th 2023 at 4:36:11 PM

They sound fine to me although I am not sure about making a specific policy page or anything like that but if we agree on this, I guess the thread itself would be fine.

It wasn't a suggestion to make a new policy page, but to have some place to discuss what the site generally thinks about the topic, as it was asked in various places and the stance may change in the future.

If this was made a policy, we probably already have existing ones that can be clarified.

Edited by Amonimus on Apr 16th 2023 at 8:32:31 PM

OK makes sense.

I think using AI writing for description might be ok if they fit guidelines but I don't see how troping would be possible as that requires someone to actually consume the work which is something the AI isn't doing. That sounds like more work and it would be easier and much preferred to just manually describe how the trope occurs

The issue is that AI is good for creating original content, not for re-creating pre-existing content. It's most useful for things where there is no correct answer, but there definitely is one here.

As for copyright... meh? My issue with AI-generated pics mostly came from the fact that we should be sticking to images we can source, similar to the current aversion towards unsourced photos. I don't have that same issue with writing.

Edited by FernandoLemon on Apr 16th 2023 at 2:41:18 PM

Yeah.

While the AI can provide correct summaries of things sometimes, it'd be hard to get it to write something as if it were an example for a trope. First, it needs to understand the trope, then scour the whole work to find something that is playing the trope. And that's assuming it finds the correct TVT article. Chances are, it'll probably have quite a few flip flops, such as finding the wrong page/website, or providing an incorrect example that either doesn't apply or is factually incorrect. On top of that, what can guarantee that it'll actually format it as an example? Too many variables.

If someone can prove me wrong, feel free to do so.

If we want to experiment, we should probably write prompts like this:

"Write an example of [popular trope/lesser known trope/barely known trope] from [popular work/lesser known work/barely known work]"

And that's not even talking about the contents of the instructed trope and work. A trope may be a bit difficult to understand even for a human, or a work may be filled with surreal imagery or symbolism that the AI would take too literally.

The AI would need to not just understand the work, but understanding the nuances of Trope definitions to identify the trope in the work correctly, as well as editing formatting and guidelines.

That's why I'm opposed to it. Well, that and my stance that if someone can't be bothered to make the edit themselves they shouldn't be editing, but that isn't something we can exactly enforce.

I just think there are too many issues with the idea of making edits with an AI and until those are hashed out I simply don't think it can work.

Edited by WarJay77 on Apr 16th 2023 at 1:46:36 PM

Basically my thoughts. Besides, I doubt AI can be whitty.

Plus I actually enjoy the process of editing the site and cannot fathom why you would want to automate it.

Edited by Karxrida on Apr 16th 2023 at 11:38:12 AM

I agree A.I. just doesn't work with proper descriptions.

At most it's a decent tool to see how you could possibly word something down the line, but that's it at best.

Images are a different issue as noted.

I think the current statements work fine as it holds the people responsible. Regardless, using it to avoid consuming a work absolutely is a major issue that I feel should be looked at if we see massive errors. It more or less is what is being said, that if you put incorrect content, it's poisoning the page. There's no excuse for not doing the research here.

Don't know how we would even enforce any policy. I will certainly not run every reported problematic edit through an AI content checker.

I think it would just come down to "we heavily discourage this" and a "put subpar content on the wiki, expect a punishment". Whether said subpar content is plagiarism from IMDB, misuse of the trope because GPT isn't optimized for trope example writeup, or copypasting stuff Grammarly "corrected" even if it's wrong.

somewhat unrelated but it's thought that the Netflix By Bots series isn't AI generated (or at least not entirely) and instead they're deliberate human-written parodies of how AI generated text tends to sound. anything made by AI dungeon also has to be regularly prompted by the user, so neuro-sama is the only one here that's really an autonomous work (as far as you can call streaming video games a "work") created by an AI.

fiction created entirely by AI probably won't be an issue for a long time, since AI doesn't really grasp the basic concepts of storytelling very well, and as OP mentioned AI-generated works are generally considered human creations for now. it might be more of a problem with writing descriptions or biographies since, even if it understands the nuances of its subject and the website, AI is very prone to just making facts up from thin air if it doesn't have a lot of information to work from. that definitely needs to be moderated if it ever comes to light.

Edited by NoUsername on Apr 16th 2023 at 1:42:03 AM

Would this make a lot of stuff that violates Real Life Troping for other Vtubers, like personal anecdotes, kosher to trope for Neuro-Sama since her "personal life" is all generated?

Edited by badtothebaritone on Apr 16th 2023 at 4:17:15 AM

I mean she's not a real person, so real life policy is inapplicable any way you put it. The better problem is that not all random things she says would count as meaningful content if it's not recurring.

i feel like she was only given a page for novelty purposes, yeah. she isn't really a character at all, because anything that can be gleaned about her "personality" is just an AI responding to chat input; there's no consistency to her backstory or characterization like there would be with a vtuber

Let's find out! I gave it the opening of an original work that I haven't published anywhere, so I know it's not copying the summary or trope description from somewhere else, and asked it to identify 3 tropes present in the story

In summary: I'd give it 2/3 tropes, I don't think the protagonist being introduced in a mysterious way counts as a Mysterious Stranger. I guess future is noir is debatable, but I could see it from the context.

However, it did a pretty good job comprehending the context of the story, and did not misidentify any major elements that I noticed. I am most impressed that it used real tropes instead of making up trope-sounding words, and that it seemed to get the general context of what those tropes meant.

Broadly, it can't really write a Tv tropes page on its own. I would imagine that with at best 2/3 tropes used correctly, it would be easier just to write it yourself. Additionally, it was able to write this primarily because I copy-pasted the content into its chat window— I don't see how you could use it to write a page for an obscure work it can't read directly.

It was cool to see what it can do, though!

Edited by Tremmor19 on May 3rd 2023 at 5:21:19 AM

That was a test of "can it read a story and identify tropes in it" Let's also test your question: can it write context for tropes and stories identified by the user?

I only did four, rather than nine but heres what I got:

Summary: 2/4. Got the popular tropes correct for both, but missed the obscure tropes. It did know enough about the obscure book series to identify one of the characters, but when it didn't know the answer, it just made up a new character and plot entirely. Like with the previous test, cool, but not a good idea to use for this purpose

Edited by Tremmor19 on May 3rd 2023 at 8:48:46 AM

So, the main issue here is its accuracy. It's easy to get it to make a trope list for a viral work and filter out all the misuse yourself, but other than that it would be difficult for it to write proper trope examples, especially for obscure works and tropes.

Interesting.

^ yea— and it would be very easy for someone who was say, less diligent or less familiar with TV tropes to confirm that it had the content of the work correct, but just accept the tropes it offers without checking to make sure they were being used correctly. Particularly likely for tropes with non-intuitive names.

Edited by Tremmor19 on May 3rd 2023 at 8:43:28 AM

Which is... pretty typical of how humans approach tropes. And why it would be really difficult to enforce policy stating "no using AI to write TV Tropes pages".

It seems more like a "mess around and find out" situation. Someone secretly uses AI to do the work, doesn't realize the mess it's making, gets suspended for it, and learns about all the horrible mistakes the AI made. Even if no one knows that an AI did it and not them, they'll still learn that it isn't good.

I feel like having a policy that at least discourages AI usage will at least lower the chances of those situations.

If they manage to get the AI to do stuff without being noticed or making mistakes, good for them.

Don't we already have one at least for nunnybot? Apparently not.

We can settle on some "You take full responsibility for edits done by plugins, scripts, AI tools, and bots" and bring it to Outdated Administrivia to append to Please Trope Responsibly.

Edited by Amonimus on May 3rd 2023 at 7:36:05 PM

yea, that seems like the most reasonable approach— same as "I let my little brother use my account!". We don't necessarily need a formal policy against little brothers editing tvtropes, we just need to be clear that its not an excuse for anythine.

maybe a comment explicitly pointing out that chatbots are not reliable and will make up answers— though frankly, i doubt anyone trying to use chatgpt to trope for them is going to be reading administrivia pages

Edited by Tremmor19 on May 3rd 2023 at 3:51:15 PM

I think that would be a good idea.

Appending a note to Please Trope Responsibly about using a AI to trope sounds fine to me.