An assistive learning application for children with Auditory Processing Disorder (APD) made in Android. The application provides an assistive reading tool, comprehension exercises and data analytics to analyze a user's progress.
Auditory processing disorder undermines language mastery – the gateway to all learning. Children suffering from APD will struggle with reading, learning, thinking and social interaction; increasing the cognitive load required of them to complete many tasks. This is frustrating for the child as tasks can be increasingly fatiguing, making school a negative experience. APD causes a person’s brain and ears to not coordinate effectively. The brain typically attributes sound to words when reading thus people affected with APD will have a challenging time understanding text at the same rate as people without. The disorder is lifelong but those who experience it can be aided through the remediation of their deficit skills through tools and techniques, which is where our proposed solution comes in.
Our problem to solve is assisting young children suffering from APD so that they can find success in their schoolwork. There are two approaches to auditory training which are the “bottom up” approach and “top-down” approach. Our solution would focus on the “top-down” approach which includes the training of cognitive abilities as well as the use of assistive technology. Elementary schools would especially benefit from a potential solution to this problem as it would assist students who are struggling, increasing equity in the classroom. Speech and Hearing pathologists would also benefit from a potential solution as they would have a tool they could recommend to children and their parents to assist them when completing work. Specifically, Hearsay Speech and Hearing Centre in Milton Ontario would benefit from a potential solution.
Our solution to the problem mentioned above involves the use of mobile computing, cloud computing and a few advanced areas of computer science. We are developing an assistive mobile application that can be utilized by young students to help improve their reading comprehension. One functionality using image to text allows the user to take a photo of what they are reading and bring it into the application. The application provides features such as highlighting parts of speech (nouns, verbs etc.) or text to speech to help breakdown the text and improve the user’s comprehension of it. In this functionality as well as throughout the application, colorful semantics are used which is a psycholinguistic approach that is often used to develop children’s speech and writing abilities. This involves the use of highlights of certain parts of speech with a specific color. For example, in the application nouns are highlighted yellow while verbs are highlighted green. This helps the reader to establish the structure of sentences and where to expect specific parts of speech ultimately providing them with tools for thinking about language.
Another key accessibility feature for reading comprehension with APD is text to speech. As the text is spoken to the user, the word being spoken is highlighted. This is significant to people with APD as they struggle to create a connection between words and the sounds they produce. The student is also able to practice their reading comprehension on context reading problems. Pathologists will often work with children on context reading problems like. They often have similar sounding or rhyming words that can be difficult for a child with APD. Practicing with these readings often helps to improve their phonemic awareness and reading abilities. Using AI, our solution can generate similar readings dynamically. This is significant as it helps improve problem variety. Currently, many similar apps targeted towards improving a child’s reading comprehension use a static collection of questions. This is a significant issue as when the child encounters a question they have completed previously, they may memorize the answers. This hinders their ability to practice comprehension and may also lead those involved (parents, teachers etc.) to believe the child is improving. We have included data analytics based off the student’s performance on the quizzes they complete to help their parents gauge the development of their child. Training auditory processing and sound discrimination skills requires consistent efforts and time. The data analytics functionality helps important people in the child’s life visualize and manage their progress. We have also included a supporting web application which allows admin accounts to generate problems that are available to all mobile users for practice.
Login and Sign up:
From left to right, the Home, Practice, Camera, Analytics and Profile Fragments:
The left image displays the RenderedText fragment highlighting parts of speech. The right image displays the Quiz fragment where the user is performing a noun quiz:
A user’s data breakdown presented in the application:
For a full breakdown of the project view the .pdf below: