this post was submitted on 24 Aug 2024
27 points (86.5% liked)

Santabot

14 readers
8 users here now

Santabot is an automated moderation tool, designed to reduce moderation load and remove bad actors in a way that is transparent to all and mostly resistant to abuse and evasion.

This is a community devoted to meta discussion for the bot.

founded 2 months ago
MODERATORS
auk
27
MediaBiasFactChecker almost broke the whole system (self.santabot)
submitted 3 weeks ago by auk to c/santabot
21 comments fedilink hide all child comments
 

Everything's been working smoothly, with nothing to report about the moderation bot. The community has been quiet but productive, which was precisely the goal, and the bot working smoothly with no issues. However, something almost went wrong in a particular entertaining fashion which I thought I would share.

The algorithm for classifying troll users doesn't have any polarity. It only knows which users are opposed to which other users. 50% of the time, it'll get its whole ranking system backwards, so the troll users are the normal ones, and everyone else gets negative rank, because the math works just as well under that ranking regime. Generally this isn't a problem, because there's a step:

        # Flip the sign if we arrived at a majority-negative ranking, which can happen
        if -min_val > max_val:
            rank[1:] *= -1

The most popular user is always more popular than the least popular troll is unpopular, by quite a big margin, so that works fine.

However. Things have changed. MediaBiasFactChecker@lemmy.world is so unpopular that it's almost (1% margin) more unpopular than the highest-rank user is popular. If that had happened, the whole polarity would have flipped, every user would have been banned, all the trolls would have been unbanned. Mass hysteria. I only happened to notice it before it happened and stop the bot. It's on track to be the least popular user on Lemmy, with about 5 times lower rank than some of the most notorious trolls.

Have fun with this information. I started checking the median rank of all users, instead. Thanks MediaBiasFactChecker.

top 21 comments
sorted by: hot top controversial new old
[–] Five 9 points 3 weeks ago* (last edited 3 weeks ago) (2 children)

As an outspoken critic of the MediaBiasFactChecker bot and the place of authority it gives to the deeply flawed site MediaBiasFactCheck.com, I find this hilarious.

MediaBiasFactChecker@lemmy.world is so unpopular that it’s almost (1% margin) more unpopular than the highest-rank user is popular.

Thank you for sharing this statistic! I'm excited that my downvotes may have contributed to this record-breaking achievement.

[–] auk 4 points 3 weeks ago

Everyone played their part, I'm sure.

And yes, it's funny as hell. That's why I wanted to spread the word!

[–] dharmacurious 2 points 3 weeks ago (1 children)

Can you give me a brief rundown on why the bot gets so much hate/what's wrong with its rankings/source site? Like, is it straight up wrong, is the problem that we shouldn't be trusting someone else to tell us what is biased/factual/good/bad info? I have no opinion currently, influence me!

[–] Five 1 points 3 weeks ago* (last edited 3 weeks ago) (1 children)
[–] poVoq 8 points 3 weeks ago (1 children)

Crisis averted 😅

But yeah, the mbfc bot is annoying, I have blocked it as well.

[–] ProdigalFrog 1 points 3 weeks ago* (last edited 3 weeks ago)

...Why didn't I think of doing this until now? 🤦‍♂️

Bout to 💥 that block button so damn hard!

[–] cerement 5 points 3 weeks ago

the bot that rates Voice of America as “Least Biased” and “High Factual Reporting” …

[–] brian@lemmy.ca 5 points 3 weeks ago (5 children)

Why is the media bias bot so disliked?

[–] 9point6@lemmy.world 19 points 3 weeks ago* (last edited 3 weeks ago) (1 children)

The message formatting looks shit on a lot of lemmy clients

The source of the data is also pretty biased in general and does objectively malicious ratings like drawing a factuality equivalency between fairly trustworthy news outlets like The Guardian and the fucking far right conspiracy outlet, Breitbart.

The owner of the bot ignores this feedback and continues to spam every thread with it.

I've blocked it

[–] Quill7513 4 points 3 weeks ago

i wish it had a different name. i think "fact checker" gives the impression that its media bias analysis is objective truth when in fact the analysis itself is biased

[–] Five 7 points 3 weeks ago

For a deep dive, you can read the original post when it was unveiled for !politics@lemmy.world last month, or the feedback post when they expanded it to !news and !world despite its terrible reception.

I've made a number of comments about the flaws in their approach. If there's interest I might make a link collection. TLDR: MBFC is a joke, nobody in the fact-checking world or experience managing source credibility takes it seriously. The author/editor of the site has a right-wing, anti-Palestinian, and anti-LGBTQ+ bias.

[–] cm0002@lemmy.world 7 points 3 weeks ago (1 children)

The guy who maintains the database it pulls from has questionable bias himself, it's not unbiased itself like it implies.

But supposedly the guy's bias leans left sooo I personally have been fairly neutral towards it

[–] catloaf@lemm.ee 2 points 2 weeks ago

The guy leans right, and leans Zionist.

[–] lemmyng@lemmy.ca 5 points 3 weeks ago
  • A lot of people get annoyed that it shows as a comment, sometimes the only comment on a post.
  • Because there's no way to hide the comment, people downvote it so it's at the bottom of the comment list.
[–] gibmiser@lemmy.world -3 points 3 weeks ago

My bet is tankies or paid social media disinformation

[–] DannyBoy@sh.itjust.works 3 points 3 weeks ago (2 children)

The idea of every normal user banned and only trolls remain is kinda hilarious though. I could only imagine the meaningful conversations that would come out of that.

[–] auk 4 points 3 weeks ago* (last edited 3 weeks ago)

I almost let it happen, so I could post this after the fact. It would have been much funnier, but I wasn't sure about the implications of banning thousands of users one by one, probably including myself and the bot.

They're still neck and neck, with the bot still just barely behind the highest-ranked user.

Edit: I swear to God, this just happened as I was typing this message: MediaBiasFactChecker is now lower-ranked than the negative of the highest user. We've crossed the threshold.

[–] trolololol@lemmy.world 1 points 3 weeks ago

Time for Exodus from lemmy and create a new network /s