Creating an App to Assist the Visually Impaired with Public Transportation

 Joseph Tooker, Verrazzano Class of 2022, completed major in Computer Science 

For my capstone project, under the mentorship of Professor Zhanyang Zhang I developed MOVESmart, a mobile navigation application that assists people who are visually impaired.

Initially, the plan was for the app to work with airports but we both agreed that public transportation travel is much more common than air travel, so that is where we focused. I developed this app for Apple devices using the xCode IDE (Integrated development environment), and the Swift programming language.

With the app being developed for visually impaired users, the most important aspect of the app was accessibility. Any text written in the app, such as instructions on how to use it, would be read to the user. Upon opening the app, users would be instructed to tap the top of the screen to open the camera, or tap the bottom of the screen to access the map. Users could take a snapshot of what their phone would be pointing at, and any signs or text in the photo would be read to the user. This was done with the help of the AWS Rekognition API (Application programming interface). When a picture is taken, it would be sent to the AWS Rekognition server, and the AI would be able to detect any text that is in the image. Once the text is detected, it would get sent back to the app.

Unfortunately, the map feature of the app was unfinished. The intention of it was for the user to be able to have public transportation information read to them, and be able to plan a trip. If there were any delays or cancellations, the user would be notified. The map would be displayed using the Google Maps API.

Looking to the future, there are definitely some areas where the app could improve. The first area for improvement is the user interface(UI). While the app is mainly intended for visually impaired users, it is still important for the app to look nice. I mainly focused on the technical side of the app. Another area for improvement is having functionality with the Google Transit API. I struggled getting this API to work, so I wasn’t able to utilize it. With additional time and research, I should be able to have functionality to the app.