I was involved in discussions 20-some years ago when we were first exploring the idea of autonomous and semiautonomous weapons systems. The question that really brought it home to me was “When an autonomous weapon targets a school and kills 50 kids, who gets charged with the war crime? The soldier who sent the weapon in, the commander who was responsible for the op, the company who wrote the software, or the programmer who actually coded it up?” That really felt like a grounding question.
As we now know, the actual answer is “Nobody.”