Red Cross warns of urgent need to regulate autonomous weapons
Tech Desk
The International Committee of the Red Cross (ICRC) and various rights organisations have cautioned that it will soon be impossible to put the “genie back in the bottle” as the creation of autonomous weapons moves faster than regulatory efforts. This severe warning followed a week-long conference in Geneva, where over 120 member states of the UN Convention on Certain Conventional Weapons met.
Advertisement
Despite significant pressure, a proposal to discuss a regulatory instrument for these weapons over the next two years failed to meet the urgent requirements demanded by rights groups. Dr Cordula Droege, director of international law, movement and partnerships at the ICRC, stated: “We’re starting to see artificial intelligence … being fielded in the battlefield and perhaps also already in weapon systems. If we leave this unregulated, it will be much more difficult to put the genie back in the bottle.”
Autonomous weapons are defined as military systems that operate without human oversight, frequently utilising artificial intelligence or robotics. As AI technology has advanced exponentially, manufacturers and militaries have produced increasingly sophisticated offensive and defensive systems. These include AI-generated target banks, utilised by the Israeli military in Gaza, and semi-autonomous drones in the Ukraine-Russia war that possess “last-mile capability” where AI manages the final approach to a target.
ICRC president Mirjana Spoljaric Egger and UN secretary general António Guterres have advocated for a legally binding instrument containing clear prohibitions and usage restrictions. The ICRC specifically seeks a ban on using autonomous weapons against personnel, while ensuring systems behave predictably and limiting their use to military objectives. Experts suggest that failing to establish adequate regulations given the rapid pace of development could lead to dangerous consequences.
“It’s not about banning all autonomous weapon systems, but to set limits in an international treaty of red lines that should never be crossed,” Droege noted. “We’re very close to crossing those red lines, particularly targeting human beings through machines should be a moral red line.” Other groups warned that delays could allow private industry and militaries to dictate the norms governing how these weapons are deployed.
Advertisement
Autonomous weapons have already caused civilian casualties. In July, a Russian drone guided by an AI system killed three civilians in Zaporizhzhia, south-eastern Ukraine, after missing its intended target—a petrol station—and striking a wall instead. In Gaza, Israel has utilised AI-assisted targeting to populate target banks at speeds exceeding human capability since the war began, which rights groups claim has led to the erroneous killing of civilians and individuals with weak ties to militant groups. Furthermore, in Iran, a US strike that killed 156 civilians, including 120 schoolchildren at an elementary school, was reportedly caused by faulty intelligence from AI systems.
Anna Hehir, head of military AI governance at the Future of Life Institute, warned that delays in creating a strong treaty could lead to “more cases of mass civilian harm that we’ve seen over the past 18 months to two years”, potentially surpassing previous events. She highlighted that manufacturers are developing “swarm” capabilities, where hundreds or thousands of autonomous drones operate simultaneously. “This has the potential for high damage, just because of the sheer mass of systems that would be autonomous.
All it takes is for one of them to face algorithmic error and then the entire swarm is utilised and led down a path based on that error … particularly if they’re programmed to target humans.”
There is also a growing risk of proliferation as non-state actors increasingly use AI to program weapons, such as the Houthi movement in Yemen using Claude to “vibe code” ballistic missiles. Additionally, armed Islamist groups in the Sahel region of Africa have reportedly used AI tools for operational planning and improving drone ballistics. A senior diplomat familiar with the UN discussions commented: “This is the only thing that is important this week. It’s not really the substance of laws. What’s on the agenda here, after 13 years, is the political will finally present to start to get serious about regulation of autonomous weapons systems.”