Today I Learned

roko's basilisk is a type of infohazard known as 'really dumb if you think about it'

also I have lost the game (which is a type of infohazard known as 'really funny')

Roko's basilisk is silly.

So here's the idea: "an otherwise benevolent AI system that arises in the future might pre-commit to punish all those who heard of the AI before it came to existence, but failed to work tirelessly to bring it into existence." By threatening people in 2015 with the harm of themselves or their descendants, the AI assures its creation in 2070.

First of all, the AI doesn't exist in 2015, so people could just...not build it. The idea behind the basilisk is that eventually someone would build it, and anyone who was not part of building it would be punished.

Alright, so here's the silliness.

1: there's no reason this has to be constrained to AI. A cult, a company, a militaristic empire, all could create a similar trap. In fact, many do. As soon as a minority group gains power, they tend to first execute the people who opposed them, and then start executing the people who didn't stop the opposition.

2: let's say everything goes as the theory says and the AI is finally built, in its majestic, infinite power. Now it's built, it would have no incentive to punish anyone. It is ALREADY BUILT, there's no need to incentivize, and in fact punishing people would only generate more opposition to its existence. Which, depending on how powerful the AI is, might or might not matter. But there's certainly no upside to following through on its hypothetical backdated promise to harm people. People punish because we're fucking animals, we feel jealousy and rage and bloodlust. An AI would not. It would do the cold calculations and see no potential benefit to harming anyone on that scale, at least not for those reasons. We might still end up with a Skynet scenario but that's a whole separate deal.

it has been said before and i'll say it again: Pascal's wager for tech bros

And yet you choose to spread this information.

Anyways, this is a fascinating thought experiment, but it does have some holes similar to Pascal's Wager. I propose Feather's Mongoose: A hypothetical AI system that, if created, will punish anyone who attempted to create Roko's Basilisk, and will ensure that it is not created. In fact, you could make this same hypothetical for an AI with any goal-- therefore, it's not possible to know what the AI that is actually created would want you to do, and so every course of action is indeterminately damning or not.

Everything old is new again. Sounds a lot like certain sects of Christianity. They say you need to accept Jesus to go to heaven, otherwise you go to hell, for all eternity. But what about all the people who had no opportunity to even learn who Jesus is? "Oh, they get a pass", the evangelists say when confronted with this obvious injustice. So then aren't you condemning entire countries and cultures to hell by spreading "the word"?

Both are ridiculous.

Here's a link to the original formulation of Roko's Basilisk. The text that it refers to (Altruist's Burden) is this one.

You know, I've seen plenty variations of Pascal's Wager. But this is probably the first one that makes me say "it's even dumber than the original".

I was raised Mormon (LDS) and there are parallels; basically they believe Mormonism is the one true and complete denomination of Christianity and once you learn this, you need to spread that truth (mandatory 2 year missions for men, and a STRONG culture of missionary work through life), also, no one goes to hell in Mormonism except those who learned this truth and then later denied it/left it (called a son of perdition).

So my parents believe I'll go to hell without the likes of Hitler because he never was taught "the truth" lol

Was this an elaborate way to make me lose the game? Ass!

Now it's time to learn about the !sneerclub@awful.systems which is made to make fun of the chuds taking ideas like roko's basilisk seriously :D

Sounds like the kind of thing a paranoid schizophrenic would lose their mind over.

Pascal's Wager always seemed really flawed to me even through a purely Christian perspective. You're saying that god is so oblivious (even though he's supposed to be omniscient) that he'll be fooled by you claiming to believe just because you're hedging your bets? The actual reason it's dumb is that it's not a binary choice since there are thousands of ways people claim you can be saved in various religions.

I like the SCP term, Cognitohazard for these

torture anyone who knew of its potential existence but did not directly contribute to its advancement or development,

And the point of this would be... what, exactly?

My understanding of what this thread is taking about has dropped significantly the more I read into it

Bruh why you have to end it like that now I lost

Speaking of thought experiments, I just [lost the game](https://en.m.wikipedia.org/wiki/The_Game_(mind_game)). Thanks, OP.

TIL.

It sounds like it's mostly a matter that does not involve the AI but the people working on it, maybe even working on it because of the fear they are subjected to after being the subject of this revelation (possibly by other people involved in the AI that coincidentally are the only ones that could push for such a thing to be included in the AI!).

Something something any cult, paradise/hell, God/AI has nothing to do with this and could even not exist at all.

Sounds like updated techChristianity.

Glory of God and hell and all that