Is it Possible to be Fully "In the Loop" with AI?

Much of the agita around deploying autonomous robotics and AI systems on the modern battlefield centers around keeping humans “in the loop”. But MIT Tech Review poses an important question – can we really remain “in the loop” if we don’t fully understand the thought process of artificial intelligence?

It seems self-evident – few want war to be fought entirely by machines with zero human oversight or input. It goes against our pride as a species (not to mention raising a gazillion ethical questions and skirting the laws of war). And our policies reflect that.

According to DODD 3000.09, all systems, including Lethal Autonomous Weapon Systems (LAWS), must "allow commanders and operators to exercise appropriate levels of human judgment over the use of force."

However, “human judgment” is intentionally left vague, and as the Congressional Research Service (CRS), itself, points out, it absolutely doesn’t mean direct human control. Instead, it places humans in more of a management role, with “broader human involvement” in decisions.

And therein lies the conundrum. The same government officials who wouldn’t cede full control of the military to AI also wouldn’t explicitly require direct human input at every level of warfare. Doing so would potentially create exploitable weaknesses on the battlefield and concede the advantage to rivals.

So we’re stuck in this weird middle ground, where the full extent of AI’s responsibilities is decided on a case-by-case basis.

And even if we do relinquish direct control, trusting autonomous systems to follow preset rules, AI’s logic is, in a word, inhuman.

In the automotive space, there’ve been examples of autonomous vehicles behaving contrary to human behavior – say, speeding up to avoid an accident – and that’s in a realm where the preservation of human life is emphasized. Imagine the logical irregularities when the goal often involves taking human life.

The laws of war might seem straightforward, but belligerents (or NGOs) often have very different interpretations. Even if humans have the final say, will AI’s cold, pragmatic logic cause it to commit war crimes against the express wishes of the flesh-and-blood overseers? Is it even possible for humans to fully understand AI’s intentions? 

Whether it’s ultimately possible or not, it certainly doesn’t seem possible right now. And that’s why MIT Tech Review called for a massive paradigm shift, where we create tools to “characterize, measure, and intervene in the intentions of AI agents before they act.”

They suggest combining mechanistic interpretability (breaking neural networks down into human-understandable components) with “insights, tools, and models from the neuroscience of intentions.”

Whatever our stratagem for understanding AI, we have to do something, or risk not fully understanding our most destructive – and inscrutable – weapon.