New York — When Matthew Sherwood shops for clothes, he often needs assistance to ensure he picks the right color or style. Despite being blind for over 15 years, with a successful career in investing, a family, and a guide dog named Chris, daily tasks like shopping can still challenge his independence.
Artificial intelligence (AI) promises to significantly change that.
Sherwood sometimes uses an app called Be My Eyes, which connects visually impaired users with sighted volunteers via live video to help with tasks such as checking if a shirt matches an outfit or confirming the expiration date on a milk carton. However, advances in AI technology are beginning to reduce the need for volunteer assistance.
Last year, Be My Eyes partnered with OpenAI to integrate AI capabilities into its app. Instead of relying on a human volunteer, the AI can now “see” and describe what is in front of the user. In a recent OpenAI product demonstration, a person used the AI-powered Be My Eyes to hail a taxi, with the app instructing the user on precisely when to raise their arm. Similarly, Google announced a feature for its Lookout app, designed to aid visually impaired users in various tasks.
These applications for visually impaired users are just one example of how AI is advancing assistive technology—tools designed to help disabled or elderly individuals. Tech giants like Apple and Google have introduced an array of AI-driven tools to improve accessibility, from eye-tracking technology that allows physically disabled users to control their iPhones with their eyes to detailed voice guidance for blind users navigating Google Maps.
Since the launch of ChatGPT over a year ago, the potential for AI to reshape our world has been evident, altering how we work, communicate, and perceive reality. For disabled individuals, however, AI has the potential to be life-changing in profound and unique ways.
“It used to be that if you were in business and blind, you needed an assistant to read to you,” Sherwood said. “Now, you have this new power… For some, this is great technology. For blind people, it’s an opportunity to gain employment, compete in business, and succeed.”
The Benefits of AI for Accessibility
Tech companies have long used early forms of AI to enhance accessibility, such as automated closed captioning on videos and screen readers. However, recent advances in AI models, supported by large datasets and powerful computing systems, are accelerating possibilities in the assistive tech space. For example, for an AI tool to reliably help blind individuals hail taxis, it must accurately recognize a taxi, which requires extensive training on numerous examples.
Google has also enhanced its tools for visually impaired users. Its screen-reading tool now features a “question and answer” capability, leveraging generative AI to provide detailed descriptions of what is displayed on a user’s screen.
“The promise of AI has been evident for many years, but it must reach a high quality level to be viable for products,” said Eve Andersson, Google’s senior director of product inclusion, equity, and accessibility, in an interview with CNN.
Generative AI tools are particularly promising for accessibility because they can understand and produce information in various formats, such as text, audio, photos, and videos. This capability allows AI to serve as an intermediary, converting information into the most suitable format for the user. For instance, AI can transform audio into written text for hearing-impaired users.
“People’s accessibility needs vary, but many disabilities involve how information is perceived,” Andersson explained. “There are hearing disabilities, vision, motor, speech, cognitive, and all can require different modalities of information. AI excels at translating between these modalities.”
Designing Inclusive AI Systems
Ensuring that AI systems remain inclusive requires continuous investment. Since AI models are trained on human-created data, they can inherit human biases. Early instances of bias have emerged, such as AI image generators struggling with racial concepts or algorithms displaying gendered stereotypes in job advertisements.
To address this, companies like Apple, Google, and Microsoft have partnered with researchers at the University of Illinois Urbana-Champaign to develop a diverse training dataset for AI speech recognition tools. Known as the Speech Accessibility Project, this initiative collects recordings from volunteers with conditions like Parkinson’s, Down Syndrome, and ALS. With over 200,000 recordings, the project has reduced the error rate of a sample speech recognition tool from 20% to 12%.
“The more diverse types of speech we can include in these systems, the better they will understand individuals with non-standard speech,” said Clarion Mendes, a speech-language pathologist and project leader. “Many people face significant barriers to participation in life due to their communication challenges. Assistive technology can dramatically increase their independence and opportunities.”
Andersson added that investing in AI for accessibility is both ethically right and financially sensible.
“We don’t want to leave people behind. Technology can level the playing field,” she said. “Additionally, there are financial benefits, such as the ability to sell products to government entities and educational institutions.”
