Ideas For The Spam Problem

That's advertisements, not spammers.

As entertaining as it is to watch someone advertise their company's various vitamins and supplements at the stroke of midnight; why not just hide the topics by new users from our view until an moderator approves it? This shouldn't hurt anyone, since the media topics generally grow at an slow rate anyways (until an new video game console is unveiled), OTC is already under scrutiny, and it'll defeat the point of advertising.

I'm going to float an idea that I had in the mod chat. This is NOT something we're implementing, so don't panic. It's something I've proposed and want feedback on.

Should we require that users link their accounts to some form of secondary authentication in order to obtain posting/editing privileges? By secondary authentication, I mean something more robust than email addresses, which are still effectively anonymous. It's trivial to get a new one even if you don't use disposable mailhosts, something we're going to be actively blocking in the future anyway.

Reliable secondary authentication methods — ones that involve jumping through some amount of hoops beyond merely creating a Gmail account — could include:

• Social media platform integration like Facebook, Google+, Twitter.
• SMS verification (cell # text messaging).
• Others I may not have thought of.

The thought process goes something like this:

• A user registers via the current method: CAPTCHA verification, username/password, email authentication.
• The user gains certain basic privileges such as the ability to edit their profile.
• The user may be allowed to post in Ask The Tropers or article discussions. (This is on the table, spammers and vandals have used these as well.)
• The user may not post or edit until they provide additional authentication.
• The user provides a secondary authentication method as listed above.
• If this authenticated identity is "clean" — that is, we haven't previously banned anyone using the same ID — they gain full privileges (subject to expiration of new account restrictions).

For obvious reasons we wouldn't make this retroactive; it would be insane to try. Equally obvious, I hope, is that we'd never make these identification elements public or share them with any third parties (other than the mod team, as necessary). Oh, and we'll have HTTPS by the time anything like this could be done.

The important question is: how much harm would this cause to the new user experience — would it be enough to outweigh the advantages in being able to prevent malicious users from registering new accounts? Would anyone already here quit the site if we did it, or never have registered to begin with?

What about younger users or those from academic settings that may not have access to secondary authentication methods? What about people who have a vested interest in anonymity due to political considerations?

Also, how easy it is to spoof the identification methods above? I thought about burner phones for SMS, but that's a much more finite resource than email addresses and requires some payment of money, which is a much bigger dissuader for casual vandals or those who lack resources. Are there other SMS spoofing services out there?

I also thought about some kind of credit authentication — not that you'd have to pay to post or anything like that, but that linking to a form of credit would help verify that you're a real person and establish your genuine identity. That has other implications that may be way beyond what we can afford to deal with, though.

Again, this is just brainstorming. Seriously, stop panicking already.

edited 6th Apr '18 7:27:36 AM by Fighteer

I think there are a lot of people who don't want to connect their real life identities to online stuff, especially something like a wiki. That removes an aspect of "anyone can edit". Credit authentication strikes particularly hard against that. Giving out phone numbers is also something people are taught not to do, even if it's to a site that advertises itself as keeping the information private.

At least some of those identification methods can easily be spoofed. You can sign up for various social platforms with fake information (well, you could some years ago). It's an additional hoop that may work against spammers, but probably not as effective against malicious edits.

However, these platforms all do have various protections against malicious accounts, which we can piggyback off of rather than reinventing them ourselves. The point isn't that it's impossible to get a fake Facebook account (for example), but that it requires enough additional effort that it should dissuade most people. Plus, if Facebook says you're fake, then we can probably trust that.

edited 6th Apr '18 8:43:49 AM by Fighteer

Um, yeah, I'm not comfortable with this idea. Duck already covered the main issues, plus the breach of privacy isn't worth blocking spammers.

I'd just restrict forum and ATT posting for a set period (72 hours? I dunno.) for new users since that's where spammers tend to strike.

edited 6th Apr '18 10:02:21 AM by Karxrida

Not just spammers. We have a couple of different classes of undesirable...

• Drive-by forum spammers
  • These mainly use IP addresses from India, Pakistan, Indonesia, and similar developing Asian countries, indicating that they are part of spam farms set up in those locations.
  • We can range-block entire countries, but that's got a pretty heavy downside in that it bars legitimate users from those countries. It also won't block all traffic, since they may adapt by using proxies.
  • They often register using disposable mailhosts, but not always. We're planning to block these.
  • They will put spam links in the post title, in their signature, and in the body of their posts. We need to block in all of these places.
  • Spam posts annoy users but they don't generally impede the functionality of the site or the use of the articles.

• Wiki-side spammers
  • They conduct more sophisticated attacks, such as posting in discussions, queries, and reviews. They may create new articles and fill them with spam, replace existing articles entirely with spam, or add spam links into articles. They also commit more acts per visit. They're like the second tier of spammer.
  • Tools can detect URIs in edits, but it's more effort to block all the possible venues and it doesn't stop the general vandalism.

• Overt trolls
  • These may or may not be repeat offenders, but simply want to cause mischief. They may troll in the forums or the wiki, but are usually reported and banned rapidly.
  • They may or may not ban evade.
  • They may have active TV Tropes accounts but use alternate accounts to troll. Detecting the original can sometimes be difficult or impossible.

• Impulse ban evaders
  • These will create new accounts immediately or shortly after a ban.
  • They will usually target the same article(s) or thread(s) that they were going after earlier, making them fairly obvious.
  • They are usually not smart enough to use proxies, or will do so amateurishly.
  • They typically go away after a few attempts.

• Serial ban evaders
  • These repeatedly make new accounts, waiting hours, days, or weeks between attempts.
  • They use proxies with more sophistication, along with disposable email accounts and other tricks. Some are lucky enough to be on highly dynamic IP ranges, or use mobile/public IPs to register, making positive detection difficult.
  • They are sometimes subtle enough to make non-problematic edits or posts before launching into whatever their pet thing is, waiting out the new account restrictions to avoid immediate detection.
  • We've caught some of them aging accounts that they can jump into when their active account is banned. These can be very hard to detect.

I don't want an unconditional block on posts and edits by new accounts. That fails to make sense for several reasons...

• Not all new accounts are immediately identifiable as malicious without a detailed examination. While disposable mailhosts are pretty easy to pick out in a list, manually checking IPs for proxy use is not something we want to do, and most would-be vandals don't pick obvious handles like "imatroll".
• Many accounts register just to make a small set of edits. Blocking that up front would be extremely unfriendly to them.

Now, some measures we're putting into place...

• We're looking into the API at check-mail.org for active detection of disposable mailhosts, which will be flagged for moderator attention and blacklisted.
• We've asked for a feature to detect when someone registers on a proxy but edits from their home IP.
• We're going to get the ability to range-block at a higher level, albeit temporarily. This will let us cut off access from Pakistan and Indonesia, two of the most prolific spam sources, with almost no legit accounts. India is under consideration but there are a lot of legit users from there, so it's harder to justify.
• We're supposed to get expanded URI detection, plus possibly have suspect posts buried and the accounts making them referred for mod attention. I'm not completely sure about this.

We've discussed additional solutions here in this topic, but I feel that a lot of it comes down to the simple problem that it's almost impossible to verify someone's identity on the Internet without using some secondary form of authentication.

This isn't like reading a news site or posting on Reddit. A wiki ought to have higher standards for authentication due to the fact that you're contributing content that becomes TV Tropes' content. So, I'm asking for ideas about how to do this without turning off so many people that it stops being worthwhile.

edited 6th Apr '18 12:41:36 PM by Fighteer

The general problem with all such restrictions is that they outlive their usefulness and there are seldom data on the "collateral" damage. I've seen some Indonesian and Pakistani tropers so I'd be wary of a broad country-based block.

Flagging people with suspect IPs would be the first thing to attempt, as well as hiding and auto-flagging forum posts which are by new accounts and contain an IP.

On the Indonesia and Pakistan thing, my initial response is, "So go fix your damn country." I know that's not realistic, but we're talking 95%+ spam accounts from those countries. At what point does extending the benefit of the doubt become pointless?

Perhaps in the cases of those countries there is a case by case basis on which accounts are validated by some extra step such as contacting the staff or other verification process that is difficult to automate. It sounds like a serious issue where a significant portion of the accounts are just garbage gumming up the system might actually warrant a heavy hand to blunt it. If we have existing tropers put the word out to them or the user base in general and as long as they are in good standing with the site they can be white listed.

I posted about this a while ago in Wishlist, but I figured I'd bring it up here for added visibility. Would it be feasible to implement a tiered page protection system tied to account age and number of edits, a la Wikipedia, with an eye towards keeping out vandals and trolls (especially one particularly persistent Single-Issue Wonk / troll in particular) without causing pages to be locked to decent edits for months or years?

I was thinking of something like this (using Wikipedia as a model):

• Bronze lock (semiprotect; keeps out most run of the mill spammers/trolls, 7 days and 10 edits).
• Silver lock (extended confirmed) used for heavy vandalism targets and other controversial pages that still may need edits. (We could just be lazy and imitate The Other Wiki's 30/500 standard, but that's obviously not a must; just make it high enough to drive off new accounts).
• Gold lock (admin/mod access only), used to enforce P5/PRLC and keep important Administrivia articles as they are. Possibly also to enforce Content Policy mandated Example Sectionectomy orders, disabled markup protection, archives, etc.

edited 16th Apr '18 11:10:13 PM by Theatre_Maven_3695

I like that idea.

On making locked new threads invisible until approved, this is a really bad idea. It will give legitimate thread creators the impression that something went wrong, and they will create another thread, and another...

Or they could read the FAQ like everyone else, ask questions, or have an indicator that the thread is awaiting approval and won't show unless it is approved.

My experience suggests that most people do not in fact read FAQs or ask questions