Scientists worry, as long as the data minor adjustments, the neural network can be fooled by a "confrontation ***" rather than help.
Last year, the US Food and Drug Administration (FDA) approved a retinal image can be captured and automatically detect lesions of diabetic blindness equipment.
The new artificial intelligence technology is spreading rapidly in the medical field. Scientists are developing a system which can recognize the signs of various diseases, as well as evidence of disease in various images, X-rays from the lungs to the brain CAT scan (CAT). Compared with the past, these systems are expected to help physicians more efficient and more economical way to evaluate the patient.
Similar forms of artificial intelligence may enter the medical regulatory agencies, accounting firms and computer systems used by insurance companies from hospital. As artificial intelligence will help the doctor to check your eyes, lungs and other organs, as it will also help the insurance company determine the amount paid and the cost of the policy.
Ideally, such a system will enhance the efficiency of the health care system. But a team of researchers at Harvard University and the Massachusetts Institute of Technology warned that they may have unintended consequences.
Thursday a paper in the journal "Science" published, the researchers raised the possibility of "adversarial ***" is. *** antagonism refers to an operation to change the use of tiny digital data artificial intelligence system behavior. For example, by changing the number of pixels on the lung scan, you can fool the artificial intelligence system, let it see a disease does not really exist, or can not see a real disease exists.
The author believes that software developers and regulatory agencies in the construction of artificial intelligence techniques and assess the next few years, we must consider these scenarios. What is more worrying is that *** may cause the patient to be misdiagnosed, although just this possibility. More likely scenario is that doctors, hospitals and other organizations can manipulate the operation of artificial intelligence software in the bill or insurance, in order to maximize their income.
Samuel Finlayson is a researcher at Harvard Medical School and Massachusetts Institute of Technology, co-author of the paper. He warned that because so much money flow trading in the health care industry, stakeholders have been lured by changing the billing system code and other data in a computer system cleverly, tracking the number of health care. AI may exacerbate the problem.
He said: "The ambiguity of medical information itself, coupled with the often competing financial incentives, making high-risk decisions can swing in a very subtle message."
This new paper exacerbated by growing concerns about the possibility of such ***, *** possible for these services from face recognition, unmanned vehicles to all areas of an iris scanner and fingerprint reader.
*** confrontational advantage of the many fundamental aspects of a design and build artificial intelligence systems. Artificial intelligence is increasingly being driven neural network. A neural network is a complex mathematical system, by analyzing large amounts of data, research tasks largely independently.
For example, by analyzing thousands of eye scanning, the neural network can learn to detect signs of diabetic blindness. The scale of this "machine learning" is so great - human behavior data is defined by a myriad of unrelated - that it can generate its own unexpected behavior.
2016, a research team at Carnegie Mellon University uses a printed pattern on the frame to spoof facial recognition systems, so that they mistakenly think that the wearer is a celebrity. When the researchers put these frames, the system will mistakenly believe that they are celebrities, including Milla Jovovich and John Malkovich.
A group of Chinese researchers have done similar experiments, they infrared light is projected from beneath the brim of the hat to the person's face. This light is invisible to the wearer, but it can spoof the facial recognition system into thinking the wearer is a musician Moby, he was white, not Asian scientists.
The researchers also warned that confrontational *** likely to deceive autonomous vehicles, so that they see something that does not exist. By doing some small signs of change, they fooled the car, it was detected that the yield signs instead of stop signs.
Late last year, Tandon New York University School of Engineering, a team of the company has created a virtual fingerprint, the fingerprint can deceive the fingerprint reader at 22% of the cases. In other words, all using this reader mobile phone or PC, 22% probability that can be unlocked.
Considering the growing popularity of biometric security devices and other systems of artificial intelligence, which is far-reaching. India implemented the world's largest fingerprint-based identification system for payment of government benefits and services. Bank is introducing facial recognition functionality to ATMs. With Google to belong to a parent Waymo an example of some of the companies are testing autonomous vehicles on public roads.
Now, Mr. Finlayson and his colleagues in the medical field issued a similar warning: As regulators, insurance companies and billing companies to start using artificial intelligence in the software system, companies can learn to use the underlying algorithms.
For example, if an insurance company uses artificial intelligence to assess medical scan results, then a hospital scan results can be manipulated to improve health care spending. If the regulatory body to establish artificial intelligence systems to evaluate new technologies, equipment manufacturers may alter images and other data, trying to cheat the system, so obtaining regulatory approval. Dalian Women's Hospital ranked http://www.bohaifk.com/
The researchers demonstrated, in their paper, by changing a small number of pixels in the image of benign skin lesions, a diagnostic system of artificial intelligence may be deceived, to determine the lesion as malignant. They found that, simply rotate the image can produce the same effect.
Written description of the patient's condition to make some small changes could change the diagnosis of AI: "alcohol abuse" could produce "alcohol dependence" different diagnostic results, and "back pain" may produce different diagnostic results and "back pain."
In turn, in one way or changed in a manner such diagnosis it is easy to make the insurance company and the ultimate profit medical institutions benefit. The researchers believe that once the artificial intelligence is deeply rooted in the health care system, enterprises will gradually be able to take the most revenue behavior.
Mr. Finlayson said, the end result might harm the patient. In order to satisfy insurance companies use artificial intelligence, doctors or other medical scanning changes made to patient data, it may eventually become a permanent record of the patient and affect future decisions.
Doctors, hospitals and other organizations have sometimes manipulating software systems that control the flow of funds throughout the industry billions of dollars. For example, in order to improve medical expenses, doctors subtly modified the bill code - for example, the simple X-ray scanning described as more complex.
Assistant Professor Wharton School, University of Pennsylvania Hamsa Bastani studied manipulate the health care system, he believes this is a major problem. "Some behavior is unintentional, but not all," she said.
As an expert in machine learning system, she questioned whether the introduction of artificial intelligence will make the problem worse. *** confrontational implementation is difficult in the real world, but it is unclear whether regulators and the insurance company will use machine learning algorithms vulnerable to this type of ***.
But, she added, (AI) concern. "There will always be unintended consequences, especially in the field of health care." She said.