this post was submitted on 25 Aug 2024

6 points (80.0% liked)

Santabot

26 readers

1 users here now

Santabot is an automated moderation tool, designed to reduce moderation load and remove bad actors in a way that is transparent to all and mostly resistant to abuse and evasion.

This is a community devoted to meta discussion for the bot.

founded 4 months ago

MODERATORS

auk

Here's why I don't think this is a good idea. (lemm.ee)

submitted 2 months ago by muntedcrocodile@lemm.ee to c/santabot

10 comments fedilink hide all child comments

It would be extraordinarily easy to bot it and just silence anyone you want.
I agree, moderation is absolutely necessary to maintaine civil discussion, but silencing people, because they have unpopular opinions, is a really bad idea.
I love lemmy because it is the ultimate embodiment of decentralised free speech. This destroys that.
If I were a bad actor, hypothetically, let's just say lammy.ml or haxbear and I decided I wanted to silence anyone who disagrees with what I have to say. Then I could just make a fork of this project to only value my instances votes and censor anyone who doesn't agree with what my community thinks.
This tool simply acts as a force multiplier for those who want to use censorship as a tool for mass silencing of descent.

Yes, I've read the Q&A, But I can simply think of more ways to abuse this bot for bad than it can be used for good.

top 10 comments

sorted by: hot top controversial new old

[–] Tiresia 3 points 2 months ago (1 children)

First off, no rules in a centralized system can survive corrupt admins/moderators. At best, the rules can make it difficult for the admins/mods to hide their malfeisance. If we don't assume good faith from the admins, this discussion is pointless because we should just leave this instance.

Second, upvotes and downvotes already moderate discussion. The default comment sorting algorithm prioritizes upvoted comments and hides downvoted comments, and people do tend to treat downvoted comments negatively. Popularity already matters, it's just a matter to what extent each thread gets you a fresh start.

Right now, slrpnk account generation is gatekept by the mods. You have to pass a Turing test to be let in. This makes it difficult to amass a sufficient army of bots without mod assistance. It's worth looking out for, but not expected by any means.
Agreement and dislike are different things. Empirically, people can become more hardened in their opinions if they see crappy disagreement - that's why organizations like FOX NEWS show a constant cavalcade of liberals and leftists being stupid. As long as people upvote well-formulated disagreement, this could actually improve discussion because it filters out the comments that would never have convinced anyone anyway. That's a big "as long as", so it's worth seeing in practice whether or not it holds.
Lemmy instances have admins and moderators with absolute unaccountable power over bannings. It has never been decentralized or pro-free speech in the ways santabot might have destroyed in a more fundamentally anarchic social media. If you want to make use of Lemmy's decentralization, make your own instance and see who wants to let you crosspost. If you want more, make your own social media platform that is (more) fully decentralized.
Yes. Bad actors gonna act bad. Stay away from places that give them authority.
Not very well. You're leaving it up to the whims of the voting public. It would be easier to write a bot that asks ChatGPT whether a user holds certain opinions and ban them if it says yes. Or deputize more (informal) mods to ban people based on their personal opinion.

It is natural that an object can be used for bad in more ways than it can be used for good. 'Good' is a fragile concept, while 'bad' is everything else. A kitchen knife can be used for bad more easily and in more different ways than it can be used for good. So can a brick or a water bottle. The question is whether its use here pumps towards good, both now and in the future.

I understand expecting this experiment to go poorly, but I think it's excessive to say the experiment should not be run at all.

[–] muntedcrocodile@lemm.ee 3 points 2 months ago* (last edited 2 months ago) (1 children)

I think that's the key, votes moderate comments and posts in terms of sorting. They don't moderate it in terms of outright silencing an opinion or idea.

I'm assuming that the voting is based on all accounts across all instances, so it's not just your instance whose account creation rules matter, it's all instances across the Fediverse, right?
I think ultimately people vote based on preconceived biases more than they will on the validity of an argument or its facts. I'd definitely love to see some data on how the experiment plays out. It'd be quite interesting if we could get that in full.
I guess not necessarily free speech but more marketplace of ideas. I guess my main concern here is that it will get implemented across the Fediverse without Admins and moderators thinking about the long-term effects of such a system.
I prefer instances that have a more open policy in terms of defederation. I feel this tool could provide people who are willing to go to the lengths of vote manipulation, direct moderation capabilities without having to be a moderator in the community itself. Hence, I believe this would lead to instances with more open federation policies being more susceptible to manipulation by extremists.
Sure, but by the misuse of this tool I can affect the moderation of an individual on a community that I don't have any moderation powers in.

I definitely think it's an interesting experiment that's worth running. But I'm hesitant to see what the outcomes of it will be if it gains mass adoption.

[+] auk -312 points 1 month ago (1 children)

It would be extraordinarily easy to bot it and just silence anyone you want.

You can try. Make a bunch of accounts on one of those instances that doesn't police their signups very well, downvote everything I've ever done with all of those accounts, and see if I get banned. I think it's more difficult to accomplish this than you think.

I agree, moderation is absolutely necessary to maintaine civil discussion, but silencing people, because they have unpopular opinions, is a really bad idea.

I love lemmy because it is the ultimate embodiment of decentralised free speech. This destroys that.

If I were a bad actor, hypothetically, let’s just say lammy.ml or haxbear and I decided I wanted to silence anyone who disagrees with what I have to say. Then I could just make a fork of this project to only value my instances votes and censor anyone who doesn’t agree with what my community thinks.

This tool simply acts as a force multiplier for those who want to use censorship as a tool for mass silencing of descent.

You posted that everyone in the US should need a nationally verified ID in order to make a social media account, and got dozens of downvotes. You also have some other unpopular opinions. You said Google should be shut down. You're not banned or close to it. Why are you so sure that this tool is going to ban people for expressing unpopular opinions?

I get where you're coming from. It's a valid concern. I think an important part of the answer will be opening up the process, and maybe even taking it out of my hands as the sole proprietor of all the parameters, so it's a community project instead of my project only.

I posted more about this:

https://slrpnk.net/post/13361827

I’m assuming that the voting is based on all accounts across all instances, so it’s not just your instance whose account creation rules matter, it’s all instances across the Fediverse, right?

Right.

I think ultimately people vote based on preconceived biases more than they will on the validity of an argument or its facts.

Sometimes. Not enough to overshadow other positive participation, in most cases. There's a thing that does happen often, where the bulk of what someone says is their unpopular opinion, and they present it with a lot of hostility, so they spend most of their time collecting mostly downvotes. That will get you banned. That, I think, is a feature, not a bug.

I’d definitely love to see some data on how the experiment plays out. It’d be quite interesting if we could get that in full.

It's a big invasion of everyone's privacy for me to lay out all the data in full. Do you want me to break down its judgements about your user, so you can see some details of at least one case? I can do that, either here or over a DM.

I'd like to be able to lay out a more complete picture, too, if you have ideas for how I can break it down without creating drama.

I guess not necessarily free speech but more marketplace of ideas. I guess my main concern here is that it will get implemented across the Fediverse without Admins and moderators thinking about the long-term effects of such a system.

This is completely fair. What I talked about in https://slrpnk.net/post/13361827, spreading the operation of the tool out to the community instead of me operating it only, seems like it could be a good solution.

I prefer instances that have a more open policy in terms of defederation. I feel this tool could provide people who are willing to go to the lengths of vote manipulation, direct moderation capabilities without having to be a moderator in the community itself. Hence, I believe this would lead to instances with more open federation policies being more susceptible to manipulation by extremists.

Sure, but by the misuse of this tool I can affect the moderation of an individual on a community that I don’t have any moderation powers in.

Like I say, try it. It's not impossible to do, but I would be surprised if anyone could make this work in reality without creating dummy accounts on an industrial scale.

An approach that will work better is to post content that will attract a lot of upvotes, raise your own user's rank, and then downvote everything I've ever done from that single highly-ranked user. You can try that, as an alternative or in conjunction, and see how well it works, if you want to try.

I definitely think it’s an interesting experiment that’s worth running. But I’m hesitant to see what the outcomes of it will be if it gains mass adoption.

It's definitely not a silver bullet. I have a tendency to look at the whole thing with rose-colored glasses, when it's not perfect. I'm completely open to people poking holes in it or figuring out things that I've missed. All I would ask is that it be based in how it actually behaves, not just a theory about how it's going to do all these terrible things, that's not based on observing how it works in practice.

Then if we look together at what it actually does, and you find a problem, we can agree on it and I can potentially even fix it.

[–] muntedcrocodile@lemm.ee 1 points 4 weeks ago (2 children)

You can try. Make a bunch of accounts on one of those instances that doesn’t police their signups very well, downvote everything I’ve ever done with all of those accounts, and see if I get banned. I think it’s more difficult to accomplish this than you think.

challenge accepted!

I got some questions tho:

Do instances have rankings or are all users considered equall? i see issues with both cases.
Are all upvotes equall?
how does instances with disabled downvotes work?
what about single direction federation?
can u give me some number on the effectivness of upvotes per account downvoting on a user.

My contraversial opinions got me banned by the bot and unbanned recently i assume due to tuning of some parameters of something.

I saw that post i think it will definatly help alleviate some of the concerns. Open sourcing it sounds like a good idea till u realise anyone can then use it to purposely be malicious so thats an even worse idea its a complex issue that thankfully i dont gotta solve.

I do t see how its a privacy violation all the raw votes are public anyways i guess its just ur bots judgements that are private. Ur welcome to post my results publicly since its all open anyways.

I mean u can always anonimise the data use a cheap locally hosted ai to paraphrase/extract the relevent points from the text and hash + salt usernames etc.

I will try it if i can get away with using my own instance then creating accounts on an industrial scale is easily doable. Thx for the info do instances have ranking or can i manufacture heigh ranked accounts on a single instance (even if i cant doing it across many instances is just a matter of purchasing a hacked account list.

Im greatfull for you licence to develop a system designed to try abuse ur bot. As long as i dont get defederated it should be doable without resorting to illegal practice's (buying stolen accounts etc).

I can print about 1k upvotes per 5minutes with 200accounts each account will have about 1/200th that many votes per 5minutes (rank building). Is this pace significant enough to influence ur algorithm?

Currently just proof of concept on my own server (although it is federated). Can easily be defederated to block this but thats only cos i dont wanna be botting someone elses server. Accounts are created with significantly real enough data to feed to an llm/image generator to make them indistinguishable from real account activity to farm votes for rankings or disseminate propaganda or whatever.

[+] auk -309 points 4 weeks ago

My contraversial opinions got me banned by the bot and unbanned recently i assume due to tuning of some parameters of something.

You are banned again, and have been for a couple of weeks. You accrued a lot of negative rank from the "is (blank) a slur" comments section, but not enough to ban initially. I think generally, you're near the borderline, which is why you drift into getting banned from time to time, as you did when some of your earlier participation aged out of the 30-day window, and it became chiefly that one comments section to go on. I think at the present time, it's mostly that you do not have much participation, outside of that one thread, and so the negative rank from that thread is pushing you over into getting banned.

As with almost everyone who gets banned, it's not your opinions that are the issue. It's your delivery. Here's an example that worked very much against you:

Is the n word a slur if it isn't directed at black people or the f word?

Can a black person call themselves an nword tho? Or are you going yo tell them what they can and cant do like the old days?

That has nothing to do with the topic at hand. It's just inflammatory for no reason. You didn't answer the question or acknowledge the point that the other person was making. You just tried to continue and inflame the bickering.

You could have posted a calm and reasoned attempt to make the exact same point you were making, acknowledging the question and feeling confident enough in your point of view that you were willing to have it probed and challenged. Instead you opted to ignore the comment, and make a counterpoint while dismissing the question.

Like it says in the FAQ, most of Lemmy is highly tolerant of a wide range of points of view. Like most people, though, they will disapprove across the board of someone who is both unpopular and hostile and argumentative about their unpopular opinion. You can get away with one or the other, but not both.

[+] auk -309 points 4 weeks ago (1 children)

Do instances have rankings or are all users considered equall? i see issues with both cases.

All instances are equal.

Are all upvotes equall?

Not at all. Votes are weighted by the "rank" of the user, as determined by the weight of votes coming in from other users. It's a modified version of PageRank, with the same strengths and a lot of the same vulnerabilities as PageRank.

how does instances with disabled downvotes work?

They're not considered differently in the algorithm. If only upvotes are coming out of any given instance, they're considered to be the same as any other upvote.

what about single direction federation?

What do you mean?

can u give me some number on the effectivness of upvotes per account downvoting on a user.

I'm deliberately not providing too much introspection into how things work numerically, because I don't want to provide a roadmap for how to abuse the system. It's impossible to make this type of system bulletproof. All I'm saying is that I think it's a lot more difficult to abuse it than you think it is.

Open sourcing it sounds like a good idea till u realise anyone can then use it to purposely be malicious so thats an even worse idea its a complex issue that thankfully i dont gotta solve.

Abusive moderation already exists on Lemmy. Automating it based on the community at large, at this point, sounds to me like more of a blessing than anything else. I don't think anyone who wants to enact abusive moderation would bother with this tool. They would just ban the people they don't like and think no more about it.

I do t see how its a privacy violation all the raw votes are public anyways i guess its just ur bots judgements that are private. Ur welcome to post my results publicly since its all open anyways.

I don't mean that it's a privacy violation in the sense of exposing people's votes. What I mean is that I don't want to create a bullying atmosphere, where I'm holding up one person's account and outlining all the reasons they are a bad poster and are banned. I prefer to do that only for people who have signed up to have it done to them.

Im greatfull for you licence to develop a system designed to try abuse ur bot. As long as i dont get defederated it should be doable without resorting to illegal practice’s (buying stolen accounts etc).

No problem. I'm interested in researching the question of how well it works, obviously. I may not be very helpful to you, in terms of giving feedback about how it's working, because as I said, I don't want to provide a roadmap. But if you think you are capable to do this, it helps me understand the system and its strengths and weaknesses to have you make the attempt.

I do think that you should worry about being defederated. I don't plan to make any attempt to influence people to defederate you, but I wouldn't be surprised if some administrators take notice of what you're doing and defederate you on their own. Vote-rigging is one of a handful of behaviors which admins feel strongly enough about to defederate for.

I can print about 1k upvotes per 5minutes with 200accounts each account will have about 1/200th that many votes per 5minutes (rank building). Is this pace significant enough to influence ur algorithm?

I doubt it. You need to outweigh millions of real votes. You can leave your process running and eventually reach that level, but I think that before you start creating an impact on the algorithm, you will be noticed and defederated. I could be wrong, of course, which is why it's a useful experiment.

I think you should also consider the performance implications of your work. If someone starts seeing increased load on their server, and they look into it and discover that you are federating clearly fake votes in the range of thousands per hour and that's why their performance is suffering, they are likely to defederate you and ban your accounts on other instances now and in the future, at the very least. I'm fine with you doing this experiment, but that doesn't extend to server administrators being fine with your impacts on them.

In fact, I recommend that you send a quick note to the server administrator of whichever instance, outlining what you're doing and asking if it is okay from their point of view. I'm not on board for you causing operational problems to anybody.

[–] muntedcrocodile@lemm.ee 2 points 4 weeks ago (1 children)

I spun up my own instance for that very reason as to not be fucking with some poor admin and am only federating with large instances (the ones u post and comment on) to avoid loading any small instances unnessasarilly.

[+] auk -195 points 4 weeks ago (1 children)

Your karma farm needs to be federated in order to work the way you want it to. It's not enough for your instance to federate with the world, that specific community also needs to.

I think I worked out a way to make it influence the vote bot rankings without polluting a real instance, but you'll need to redo all the votes so they federate out. I think it'll work fine if you just have all the vote bots undo their votes to each other, and then redo them. Give it a try.

[–] muntedcrocodile@lemm.ee 2 points 4 weeks ago (1 children)

Ahh so the karma farm isnt federating? This is my first time running anything federated. If somone from ur instance subscribes to the karma farm community will that then federate it to ur instance and thus allow ur bot to count those votes?

[–] auk 2 points 4 weeks ago

Ahh so the karma farm isnt federating?

Yes, that's right. Communities are not federated except to instances that are subscribed to them.

Also, it only counts things going forward. When you subscribe to a community, you don't get the votes going back historically. You only get new votes from that point forward.

If somone from ur instance subscribes to the karma farm community will that then federate it to ur instance and thus allow ur bot to count those votes?

Yes. I've done a version of that now. If you have the bots in the karma farm make all new posts and mutual votes, then I should be able to pick it up now. The posts you made before won't take effect, because nothing was federating out at the time that you made them.