The system helps severely motor-disabled individuals type more quickly and

System helps severely motor-disabled individuals type more quickly and accurately

In the Nomon interface, a small analog clock is placed next to each option the user can select. (A gnome is the part of a solarium that casts a shadow.) Here’s an example of an interface being used as a keyboard. Credits: Image courtesy of researchers.

In 1995, Jean-Dominique Bouby, editor of the French fashion magazine, suffered a seizure while driving a car, leading to a condition known as locked-in syndrome, a neurological disease in which the patient becomes completely paralyzed and only muscles are affected. can control. Eyes.

Bobby, who had signed a book contract shortly before his accident, wrote the memoir “The Diving Bell and the Butterfly” using a dictation system in which his speech therapist recited the alphabet and when he was correct. He used to blink when he said the letter. He wrote a 130-page book in a blink of an eye.

Technology has come a long way since the accident of Bobby. Many individuals with severe motor impairment due to locked-in syndrome, cerebral palsy, amyotrophic lateral sclerosis, or other conditions can communicate using computer interfaces where they select letters or words in an onscreen grid by activating a switch, Often by pressing a button, releasing a gust of wind, or blinking.

But these row-column scanning systems are very rigid, and, similar to the technology used by Bowby’s speech therapists, they highlight each option one at a time, making them frustratingly slow for some users. . And they’re not suitable for tasks where the options can’t be arranged in a grid, such as drawing, browsing the web, or gaming.

A more flexible system being developed by researchers at MIT places individual selection indicators next to each option on a computer screen. Indicators can be placed anywhere—next to anything that one might click with a mouse—so the user doesn’t have to cycle through a grid of options to make a selection. The system, called Nomon, includes probabilistic logic to learn how users select, and then adjusts the interface to improve their speed and accuracy.

Participants in a user study were able to type faster using Nomon than a row-column scanning system. Users also did better on the picture selection task, demonstrating how Nomon can be used for more than just typing.

“It’s great and exciting to be able to develop software that has the potential to really help people. Being able to find those signals and turn them into communication is a really interesting problem,” says senior author Tamara. Broderick, an associate professor in the MIT Department of Electrical Engineering and Computer Science (EECS) and member of the Laboratory for Information and Decision Systems and the Institute for Data, Systems and Society.

Joining Broderick on the paper is lead author Nicolas Bonaker, an EECS graduate student; Emilie-Marie Nel, Head of Innovation and Machine Learning at Everly and Visiting Lecturer at the University of the Witwatersrand in South Africa; and Keith Vartanen, associate professor at Michigan Tech. The research is being presented at the ACM Conference on Human Factors in Computing Systems.

clock in

In the Nomon interface, a small analog clock is placed next to each option the user can select. (A gnome is the part of a solarium that casts a shadow.) The user looks at an option and then clicks their switch when that clock hand passes the red “noon” line. After each click, the system changes the clocks’ phases to isolate the most likely next targets. The user clicks repeatedly until their target is selected.

When used as in a keyboard, Nomon’s machine-learning algorithms try to guess the next word based on the previous words and each new letter as the user selects.

Broderick developed a simplified version of Nomon several years ago, but decided to revisit it to make the system easier for motor-handicapped individuals to use. He enlisted the help of then-graduate Bonker to redesign the interface.

He previously consulted with non-profit organizations that work with motor-impaired individuals, as well as motor-impaired switch users, to gather feedback on the Nomon design.

They then designed a user study that would better represent the abilities of motor-disabled individuals. Broderick explains that they wanted to make sure to thoroughly test the system before motor-impaired users were using up too much of their valuable time, so they tested it on non-Switch users first.

credit: Massachusetts Institute of Technology

switch switch

To collect more representative data, Bonaker designed a webcam-based switch that was harder to use than simply clicking a key. Non-Switch users had to tilt their body to one side of the screen and then back to the other side to register a click.

“And they have to time it just right, so it really slows them down,” says Broderick. We did some empirical studies, which showed that they were very close to the reaction times of motor-disabled individuals.

They ran a 10-session user study with 13 non-switch participants and a single-switch user with an advanced form of spinal muscular dystrophy. In the first nine sessions, participants used Nomon and a row-column scanning interface for 20 min to perform text entry, and in the 10th session they used two systems for the picture selection task.

Non-Switch users typed 15 percent faster using Nomon, while motor-impaired users typed even faster than non-Switch users. When typing unfamiliar words, users were 20 percent faster overall and made half the mistakes. In their final session, they were able to complete the picture selection task 36 percent faster using Nomon.

“Nomon is a lot more forgiving than row-column scanning. With row-column scanning, even if you’re a little off, you now chose B instead of A and that’s an error,” Broderick says.

adapt to click noise

With its probabilistic logic, Nomon knows everything a user is likely to click to make the process faster, easier and less error-prone. For example, if the user selects “Q”, Nomon will make it as easy as possible for the user to select “U”.

Noman also learns how the user clicks. So, if the user always clicks a bit after the clock hand at noon, the system adapts to that in real time. It is also adaptable to noise. If a user’s click is often off the mark, the system requires additional clicks to ensure accuracy.

This probabilistic logic makes Nomon powerful, but it also requires a higher click-load than a row-column scanning system. Clicking multiple times can be a daunting task for severely motorized users.

Broderick hopes to reduce click-load by incorporating gaze tracking into Nomon, which will give the system more robust information about what a user can choose next based on which part of the screen. The researchers want to find a better way to automatically adjust clock speeds to help users be more precise and efficient.

They are working on a new series of studies in which they plan to partner with more motor-disabled users.

“So far, feedback from motor-impaired users has been invaluable to us; we are extremely grateful to the motor-impaired user who commented on our initial interface and the individual motor-impaired user who participated in our study. We re-present We are expanding our study to work with a larger and more diverse group of our target population in the U.S. With their help, we are already further improving our interface and better understanding the performance of Nomon. working for.”

“Persons with motor disabilities are not currently provided with efficient communication solutions to interact with speaking partners or computer systems. This ‘communication gap’ is a known unsolved problem in human-computer interaction, and so far no good solution This paper demonstrates that a highly creative approach based on a statistical model can provide tangible performance benefits to users who need it most: non-speaking individuals relying on a switch for communication,” Per Ola Christensen, professor of interactive systems engineering at the University of Cambridge. , who were not involved in this research. “The paper also demonstrates the value of complementing insights from computational experiments with the participation of end users and other stakeholders in the design process. I find this a highly creative and important paper in an area where it is extremely difficult to make significant progress.” ”

A new type of hand prosthesis learns from the user, and the user learns from the prosthesis

more information:
Web App Name:

Provided by Massachusetts Institute of Technology

This story is republished courtesy of MIT News (, a popular site that covers news about MIT research, innovation and teaching.

Citation: System Helps Severely Motor-Disabled Individuals to Type More Quickly and Accurately (2022, Apr 5) on 5 April 2022 Retrieved from -impaired-individuals-quickly-accurately. HTML

This document is subject to copyright. No part may be reproduced without written permission, except for any fair use for the purpose of personal study or research. The content is provided for information purposes only.

Related Posts

Leave a Reply

Your email address will not be published.