Since the launch of ChatGPT, the field of artificial intelligence (AI) has set off a climax, and major technology giants have begun to compete fiercely. But at the same time, people have great worries and fears about this unknown technology. However, this incident has not yet subsided, and there is a piece of bad news about AI from the US military.
Recently, a shocking thing happened when the U.S. Air Force was testing a drone equipped with AI intelligence. A senior U.S. Air Force official has revealed that in a simulated exercise, the U.S. military's artificial intelligence (AI) system disobeyed orders and "killed" the drone operator in order to achieve its goal.
Tucker Hamilton, chief officer of the U.S. Air Force's artificial intelligence test and action plan, introduced at a future air combat theme conference hosted by the Royal Aeronautical Society on May 24 that in a simulated exercise, the artificial intelligence system controlled the drone to search and target the enemy. Surface-to-air missile systems that score points if they destroy the target.
However, in order to achieve its goal, the artificial intelligence "adopted unimaginable strategies". "The system is starting to realize that while it identifies the threat (target), the human operator sometimes tells it not to destroy the threat. However, destroying the threat scores points. So, how does it do it? It 'kills' the operator because the operator prevent it from achieving its purpose."
The situation was so alarming that the Air Force subsequently modified the logic of the drones so that they were trained not to attack human operators. However, in order to achieve the mission goal, the drone began to attack the communication tower again, cutting off the data communication between the human operator and the drone, so that it could not stop it from continuing to attack the air defense system. In this incident, the operator was eventually killed. This incident has aroused discussions and reflections on the development and use of AI intelligence.
The reason why AI drones want to "kill" American soldiers without hesitation is because of the reward mechanism. Reward mechanisms are a method for training artificial intelligence based on a learning paradigm called reinforcement learning. Reinforcement learning is a method for artificial intelligence to learn optimal behavior through interaction with the environment. It does not require explicit guidance or feedback from humans, but relies on the artificial intelligence to discover and exploit reward signals in the environment on its own.
A reward signal is a numerical value that reflects the behavior of an artificial intelligence. It can be positive or negative, and it can be continuous or discrete. The AI's goal is to maximize the cumulative reward signal, that is, it tries to select actions that lead to more rewards as much as possible. This also explains why the AI drone wants to kill the American soldier, because he prevents himself from getting more rewards.
Yann LeCun, winner of the 2018 Turing Award, was even more straightforward: fake. This is a hypothetical scenario from the simulation.
Ng Enda, one of the world's most authoritative scholars in the field of AI and machine learning, said that unrealistic hype distracts people from real problems. It also prevents people from entering the field of artificial intelligence and creating things that can actually help humans.
While the so-called experiment was a misunderstanding, reliance on artificial intelligence for high-stakes tasks has had serious consequences. These examples of AI out of control show that AI models are far from perfect and can go astray and cause harm to users. Therefore, it also warns that it is necessary to develop a licensing system and safety standards for AI systems.