A drone that killed three Ukrainian civilians, including a 19-year-old student, was flying with an experimental navigation system that uses artificial intelligence to hone in on targets.
The Russian drone in question was reportedly attempting to target a gas station in an attack in Zaporizhzhia when it struck a wall. It exploded on contact, killing 19-year-old student Tetiana Bubynets and two others.
The drone’s navigation system was programmed by human operators who key in on a generalized target — a particular building, a military base, or a gas station — and the AI is supposed to fine-tune the target once it’s in range of the drone, according to a report published Sunday by The New York Times. A drone sent by human programming to a specific military base, for example, might then use AI to hone in on propane tanks that it was programmed to “recognize” as a target.
Forensic investigators first determined the drone was using AI after they evaluated debris from the deadly Zaporizhzhia strike and several other drone attacks. What they found were onboard minicomputers, made by tech giant Nvidia, which had the capability to use AI to make targeting decisions in real time. That, coupled with the fact that they lacked the antennas necessary for receiving external commands, led them to believe that the final targeting decisions were made using AI.
As the report noted, both Russia and Ukraine have been using AI in drones for some time — but only in a “last mile” capacity in which a remote human operator has selected the final target. Rumors had been circulating that Russia was using fully autonomous AI, and according to Kateryna Bondar — senior fellow at the Wadhwani AI Center at the Center for Strategic and International Studies in Washington — the presence of the Nvidia module in recent drone attacks is clear evidence of that. The deadly strike in Zaporizhzhia marks the first time that a drone using fully autonomous AI targeting resulted in civilian deaths.
The benefits of such a system are obvious: an AI targeting system can be trained to recognize specific things like railroad tracks, tanks, or people — and can track them with greater precision than a human. But the risks are also obvious: AI is far less likely to be able to distinguish between civilians and soldiers — meaning that the use of fully autonomous AI could put a nation on the wrong side of the laws of war — and the inability for a remote human operator to communicate with the drone means that there is no way to override the AI system’s decisions in real time.
Col. Serhiy Minaiev, the commander of air defenses in Zaporizhzhia, sounded the alarm, arguing that taking such decisions out of the hands of human operators heralded the kind of dystopian scenarios usually only seen in movies: “This is a risk for the whole world. In a few years, we will be living in a ‘Terminator’ movie. It’s no joke. Machines are making decisions to strike.”
.png)
.png)
