THURSDAY, Aug. 2, 2018 (HealthDay News) — Donji Cullenbine’s young son, Alex, has autism, but when he put on a pair of Google Glass smartglasses they helped him recognize the emotions of others through their facial expressions.
“Within two, maybe three weeks, I caught him flicking a glance at me,” said Cullenbine. “It was stunning because it was spontaneous. I had nothing to do with it. And then there were more. I thought this is a change. This is different. And he kept doing it, and it became more common,” she said.
“They were usually very short glances, but they were real. He was looking for information. He wanted to know what was on my face,” Cullenbine added.
“The study had helped him overcome his anxiety, and taught him he could recognize what was there. At one point he said, ‘Mommy, I can read minds.’ And I thought, he’s getting it! He’s getting why you look at faces,” said Cullenbine, of San Jose, Calif.
“It not only didn’t fade, but this icebreaker for him has made it so that he routinely glances around at faces,” she said.
Alex, who is 9, was part of a pilot study that used the smartglasses to provide a small group of children real-time feedback about the emotions being conveyed in others’ faces, researchers explained.
“We’re seeing an average gain that’s consistent and very strong,” said senior researcher Dennis Wall, an associate professor of pediatrics and biomedical data science with Stanford University School of Medicine. “The kids are getting more social and making more eye contact, on average, and are appreciating and understanding emotion better, on average, than when they started.”
The struggle to connect
Children with autism often struggle to read the subtle nonverbal cues contained in a person’s expression. It’s so uncomfortable that these kids routinely fail to make eye contact and grapple with even basic social interactions.
As a result, Cullenbine explained, the world becomes an incredibly scary place for these children, full of people who act in apparently unpredictable ways.
Wall said that early autism therapy can help kids better understand emotions and social cues, but a lack of autism practitioners means that many kids aren’t treated within the window of time where their brains are most adaptable.
“It’s heartbreaking to think about kids on waiting lists passing through those windows of brain development where the interventional therapy can have a major impact,” Wall said.
Facial recognition app makes emotions easier to read
As a means of getting early help to these children, Wall and his colleagues created a smartglass app that could scan faces and recognize eight core facial expressions: happiness, sadness, anger, disgust, surprise, fear, neutral and contempt.
The app’s facial recognition feature was trained with hundreds of thousands of photos featuring faces showing the eight expressions, the study authors said.
The glasses read the faces of those with whom the child is interacting, and displays in their peripheral vision an emoticon reflecting the emotion that’s being expressed, Wall noted.
Researchers designed three ways to use the smartglasses:
- “Free play” recognizes facial cues as children interact or play with their families.
- “Guess the emotion” asks the child to read the face of a parent acting out one of the eight core facial expressions.
- “Capture the smile” asks children to give another person clues about the emotion they want to elicit, until the other person acts it out. This helps gauge a child’s understanding of different emotions.
Pilot study delivers promising results
The 14 kids in the pilot study used the smartglass app for at least three 20-minute sessions per week over a six-week period. By the end, 12 of the 14 children were experiencing meaningful improvement in their eye contact, Wall said.
The children, on average, showed improvement in their autism symptoms. Six of the 14 improved so much that they actually moved up in the autism spectrum classification — four from severe to moderate, one from moderate to mild, and one from mild to normal.
The therapy is based on applied behavior analysis, a well-studied autism treatment in which a clinician uses flash cards depicting faces with different emotions.
Although this therapy has proven effective, it has limitations, the researchers said. Flash cards can’t always capture the full range of human emotion, and children may struggle to transfer what they learn to their daily lives.
This new smartglass technology could be a “breakthrough” in helping kids with autism learn from the world around them, rather than in a sterile environment, said Thomas Frazier, chief science officer for Autism Speaks, an autism advocacy organization.
Technology a “game-changer” for kids with autism
“That’s a game-changer in the sense that it would be real-time. There would be immediate feedback from real-world interactions,” Frazier said.
“It also increases independence,” he added. “They don’t have to have an adult or therapist or a peer model right next to them all the time, prompting their behavior. The glasses themselves can prompt their behavior.”
Wall pointed out that the smartglass app is 97 percent accurate at reading faces, but concerns over that remaining 3 percent led the researchers to add a “reset” feature to the program.
Odd lighting can affect the recognition software, and so can faces captured at odd angles, Wall explained. The reset feature captures a few images of a parent’s face in a calm, non-expressive state, and uses those neutral images to reset the software.
“It’s not often parents have to use it, frankly, but we felt it was important that it be there,” he said.
Wall and his colleagues just completed a full-fledged clinical trial of the smartglass app, and are preparing it for publication.
“The middle- to high-functioning kids may progress to a point even on this program between the ages of 4, 5 and 6 where they no longer need intensive behavioral therapy,” Wall said. “That’s the goal. That’s the hope.”
The new report was published Aug. 2 in the journal npj Digital Medicine.
The study was funded by Stanford, the U.S. National Institutes of Health and a number of philanthropic organizations.
More information
The Autism Society has more about autism and social interaction.