ICDS News

Patient with hearing aid and smartphone on light background, closeup. Space for text

Study suggests smart assistant design improvements for deaf users

by Jessica Hallman; originally published on Penn State News

UNIVERSITY PARK, Pa. — Despite the inherent challenges that voice-interaction may create, researchers at the Penn State College of Information Sciences and Technology recently found that deaf and hard-of-hearing users regularly use smart assistants like Amazon’s Alexa and Apple’s Siri in homes, workplaces and mobile devices.

The work highlights a clear need for more inclusive design, and presents an opportunity for deaf and hard-of-hearing users to have a more active role in the research and development of new systems, according to Johnna Blair, an IST doctoral student and member of the research team.

“As smart assistants become more common, are preloaded on every smartphone, and continue to provide benefits to the user beyond just the ease of voice activation, it’s important to understand how deaf and hard-of-hearing users have made smart assistants work for them and the realistic challenges they continue to face,” said Blair.

In their study, Blair and Saeed Abdullah, assistant professor of information sciences and technology, conducted in-depth interviews with deaf users of smart assistants and collected survey data from individuals with mild to profound hearing loss.

They found that users from these populations — even those with profound deafness — used smart assistants to accomplish daily tasks, such as checking the weather, setting reminders and initiating GPS directions. They also identified common challenges these users face in their interactions with smart assistants, such as the default higher-pitched female voices used on many smart assistants being incompatible with hearing aids, and difficulty of use in public places with background noise that competes for their attention, such as nearby conversations or the average commotion in a grocery store.

“When we started working on this specific research problem, I initially had no idea how individuals with hearing loss would potentially use these smart assistants given they were relying mostly on audio and voice interactions,” said Abdullah. “So, it is really interesting to see how different people have quite different ways of engaging with smart assistants.”

Added Blair, “By empathizing with these users, we want to highlight new design features that can make smart assistants more accessible to hearing needs.”

Using smart assistants with all these constraints — in a quiet home, facing the device and dropping all other tasks — hinders the naturalistic, “imbedded in daily life” experience that people specifically seek from smart assistants. Therefore, the researchers suggest future design features that could better accommodate hearing needs, such as customizable voice characteristics; tangible information about common commands and error codes; and improved visual feedback, including more intuitive light patterns on devices or screen-based feedback.

“Hearing loss is not a straight line. There is a lot of fluctuation in both directions — hearing loss can worsen as people age, but hearing aids can also improve over time — so devices need to be able to evolve with the user right in step with these changes,” said Blair.

The researchers suggest recalibrating smart assistant voices through a means similar to how hearing devices are recalibrated by audiologists — to map out the ideal pitch range for individual users.

“With improvements, many saw promise in this type of technology to address other accessibility concerns in their daily lives, such as providing assistance in large work meetings or helping them fill in communication gaps when talking to others in loud public spaces,” said Blair.

She concluded, “Better accommodating the needs of hard-of-hearing users could also provide new opportunities for use, such as speech therapy, and speak to the needs of the wider population as they change over time. Making smart assistants more flexible and customizable for one group of users can help improve accessibility for all.”

The work was published in the December 2020 proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT). The Tronzo Medical Informatics Endowment partially funded the research.

Share

Related Posts

• Featured Researcher: Nick Tusay
• Multi-institutional team to use AI to evaluate social, behavioral science claims
• NSF invests in cyberinfrastructure institute to harness cosmic data
• Center for Immersive Experiences set to debut, serving researchers and students
• Distant Suns, Distant Worlds
• CyberScience Seminar: Researcher to discuss how AI can help people avoid adverse drug interactions
• AI could offer warnings about serious side effects of drug-drug interactions
• Taking RTKI drugs during radiotherapy may not aid survival, worsens side effects
• Cost-effective cloud research computing options now available for researchers
• Costs of natural disasters are increasing at the high end
• Model helps choose wind farm locations, predicts output
• Virus may jump species through ‘rock-and-roll’ motion with receptors
• Researchers seek to revolutionize catalyst design with machine learning
• Resilient Resumes team places third in Nittany AI Challenge
• ‘AI in Action’: Machine learning may help scientists explore deep sleep
• Clickbait Secrets Exposed! Humans and AI team up to improve clickbait detection
• Focusing computational power for more accurate, efficient weather forecasts
• How many Earth-like planets are around sun-like stars?
• Professor receives NSF grant to model cell disorder in heart
• SMH! Brains trained on e-devices may struggle to understand scientific info
• Whole genome sequencing may help officials get a handle on disease outbreaks
• New tool could reduce security analysts’ workloads by automating data triage
• Careful analysis of volcano’s plumbing system may give tips on pending eruptions
• Reducing farm greenhouse gas emissions may plant the seed for a cooler planet
• Using artificial intelligence to detect discrimination
• Four ways scholars say we can cut the chances of nasty satellite data surprises
• Game theory shows why stigmatization may not make sense in modern society
• Older adults can serve communities as engines of everyday innovation
• Pig-Pen effect: Mixing skin oil and ozone can produce a personal pollution cloud
• Researchers find genes that could help create more resilient chickens
• Despite dire predictions, levels of social support remain steady in the U.S.
• For many, friends and family, not doctors, serve as a gateway to opioid misuse
• New algorithm may help people store more pictures, share videos faster
• Head named for Ken and Mary Alice Lindquist Department of Nuclear Engineering
• Scientific evidence boosts action for activists, decreases action for scientists
• People explore options, then selectively represent good options to make difficult decisions
• Map reveals that lynching extended far beyond the deep South
• Gravitational forces in protoplanetary disks push super-Earths close to stars
• Supercomputer cluster donation helps turn high school class into climate science research lab
• Believing machines can out-do people may fuel acceptance of self-driving cars
• People more likely to trust machines than humans with their private info
• IBM donates system to Penn State to advance AI research
• ICS Seed Grants to power projects that use AI, machine learning for common good
• Penn State Berks team advances to MVP Phase of Nittany AI Challenge
• Creepy computers or people partners? Working to make AI that enhances humanity
• Sky is clearing for using AI to probe weather variability
• ‘AI will see you now’: Panel to discuss the AI revolution in health and medicine
• Privacy law scholars must address potential for nasty satellite data surprises
• Researchers take aim at hackers trying to attack high-value AI models
• Girls, economically disadvantaged less likely to get parental urging to study computers
• Seed grants awarded to projects using Twitter data
• Researchers find features that shape mechanical force during protein synthesis
• A peek at living room decor suggests how decorations vary around the world
• Interactive websites may cause antismoking messages to backfire
• Changing how government assesses risk may ease fallout from extreme financial events
• Symposium at U.S. Capitol seeks solutions to election security
• ICS co-sponsors Health, Environment Seed Grant Program
• Differences in genes’ geographic origin influence mitochondrial function
• ICS Affiliate named AGU fellow