A MIT research team has now designed a stretchable, skin-like device that can be connected to the face of ALS patients (amyotrophic lateral sclerosis (ALS) people’s ability to control muscles gradually declines, They often lose the ability to speak), and can measure small movements such as twitching or smiling. Using this method, patients can convey various emotions, such as "I love you" or "I'm hungry" through small movements that the device measures and interprets.
The researchers designed a skin-like device that can measure the small facial movements of patients who have lost the ability to speak. They hope that the new device will enable patients to communicate in a more natural way without having to rely on bulky devices. The wearable sensor is so thin that it can be disguised as a cosmetic to match any skin tone, so it is inconspicuous.
Canan Dagdeviren, the head of the research team (assistant professor of media arts and sciences for MIT-LG Electronics Career Development), said: "The device is not only malleable, flexible, portable and disposable, but more importantly, it has good The visual camouflage makes people feel that there is nothing on the skin." And all the components in the sensor device are easy to mass produce, so the researchers estimate that the cost of each device is about $10
The researchers tested the initial version of the device in two ALS patients (a female and a male to ensure gender balance) and showed that the device can accurately distinguish three different facial expressions: smile, open mouth and squeeze mouth.
MIT graduate student Farita Tasnim (Farita Tasnim) and its scientist Sun Tao are the main authors of the research project, which was published in Nature Biomedical Engineering. MIT undergraduate Rachel McIntosh, postdoc Dana Solav, scientist Zhang Lin and laboratory senior manager David Sadat. The authors of the article also include Gu Yuanyuan, Nikta Amiri, Mostafa Tavakkoli Anbarani of the A*STAR Institute of Microelectronics in Singapore, and M. Amin Karaami of the University of Buffalo.
Skin-like sensor design:
The "Conformal Decoding" research group in Dagdeviren's laboratory specializes in the development of conformal (flexible and stretchable) electronic devices that can be adhered to the body for various medical applications. After meeting with Stephen Hawking in 2016, she became interested in studying how to help patients with neuromuscular diseases communicate. At that time, a world-renowned physicist visited Harvard University, and Dagdeviren was the Harvard University. Junior researcher of the Researchers Association.
Hawking died in 2018, suffering from a slow-moving ALS. He was able to communicate using an infrared sensor, which could detect the twitching of the cheeks, which caused the cursor to move between the rows and columns of letters. Although effective, this process can be time-consuming and requires huge equipment.
Other ALS patients use similar devices to measure the electrical activity of nerves that control facial muscles. However, this method also requires bulky equipment and is not always accurate.
The device they created consists of four piezoelectric sensors embedded in a silicone film. The sensor is made of aluminum nitride and can detect the mechanical deformation of the skin and convert it into a voltage that is easy to measure. All these components are easy to mass produce, so the researchers estimate that the cost of each device is about $10.
The researchers used a process called digital imaging correlation on healthy volunteers to help them choose the most useful location for the sensor. They randomly drew black and white spot patterns on the face, and then used multiple cameras to take many images in the area when the subject performed facial actions (such as smiling faces, twitching cheeks or spitting out certain letter shapes). The image is processed by software that analyzes how the dots move relative to each other to determine the amount of strain experienced in a single area.
McIntosh said: "We asked the subjects to perform different actions and created strain maps of various parts of the face." "Then, we checked the strain maps and determined where on the face we saw the correct strain of the device. Level, and determined that the equipment is a suitable location for our experiments."
The researchers also used measurements of skin deformation to train machine learning algorithms to distinguish between smiling, opening and pursing. They used this algorithm to test the equipment on two ALS patients and were able to distinguish between these different actions with approximately 75% accuracy. The accuracy of healthy subjects is 87%.
Continuous monitoring of facial movements plays a key role in the nonverbal communication of patients with neuromuscular diseases. Currently, the mainstream method is camera tracking, which poses a challenge to continuous and portable use. The wearable thin and light piezoelectric sensor developed by the MIT group can reliably decode facial changes and predict facial kinematics, improving recognition Accuracy.
"Artificial Intelligence + Wearable" Enhance Communication
Based on these detectable facial actions, a phrase or word library can be created to correspond to different combinations of actions. It is also possible to create customized messages based on the actions of each person. The determining factor is the configuration of the entire word bank, which can be designed for a specific patient or a group of patients. "
Information from the sensors is sent to a handheld processing unit, which analyzes it using an algorithm trained by the researcher to distinguish facial movements. The researchers said that in the current prototype, the device is connected to the sensor, but it can also be connected wirelessly.
The technology has already applied for a patent for this technology, and they now plan to test it on other patients. In addition to helping patients communicate, the device can also be used to track the progress of a patient's disease or measure whether the treatment they receive is effective. There are many clinical trials testing whether a particular therapy is effective in reversing ALS. "The device can not only rely on patients to report that they feel better or feel stronger, but it can also provide a quantitative method to track the effects."