We are experimenting with hierarchies of needs, giving behaviors point values to inform the AI how to conduct itself completing its tasks. This is how, in simulations we are seeing warbots kill their commanding officers when they order pauses to attacks. (Standard debugging, we have to add survival of the commanding officer into the needs hierarchy)

So yes, we already have programs, not AGI, but deep learning systems nonetheless, that are coded for their own survival and the survival of allies, peers and the chain of command.

[–] MBM@lemmings.world 3 points 5 months ago (1 children)

in simulations we are seeing warbots kill their commanding officers when they order pauses to attacks.

Wasn't that a hoax?

[–] uriel238@lemmy.blahaj.zone 1 points 5 months ago

If it is, it's a convincing one. The thing is, learning systems will try all sorts of crazy things until you specifically rule them out, whether that's finding exploits to speed-run video games or attacking allies doing so creates a solution with a better score. This is a bigger problem with AGI since all the rules we code as hard for more primitive systems are softer, hence rather than telling it don't do this thing, I'm serious we have to code in why we're not supposed to do that thing, so it's withheld by consequence avoidance rather than fast rules.

So even if it was a silly joke, examples of that sort of thing are routine in AI development, so it's a believable one, even if they happened to luck into it. That's the whole point of running autonomous weapon software through simulators, because if it ever does engage in friendly fire, its coders and operators will have to explain themselves before a commission.