Integrated Multimodal Braille Translation and Recognition System

Abstract

Assistive technologies for visually impaired individuals often rely on cloud-based processing, specialized hardware, or single-purpose solutions that limit accessibility, compromise user privacy, and reduce usability in real-world environments. This paper presents BrailleVision, an integrated multimodal Braille translation and recognition system that combines browser-native Edge Artificial Intelligence (Edge AI), computer vision, multilingual translation, and conversational interaction within a unified assistive framework. The proposed system performs all major inference tasks locally on the user's device using WebGPU, TensorFlow.js, and WebAssembly, thereby eliminating dependency on remote servers while enabling low-latency and privacypreserving operation. BrailleVision incorporates a custom YOLOv8-based object detection model for real-time recognition and decoding of Grade-1 Braille characters from live camera streams, a vision-language model for contextual scene understanding, an on-device multilingual translation engine supporting multiple languages, and an intelligent conversational interface that provides speech-based interaction and guidance. To improve usability in practical environments, the system integrates image-quality assessment, voice-assisted camera alignment, hazard-aware scene feedback, spatial memory, and an interactive Braille learning module within a single browser application. Experimental evaluation demonstrates reliable Braille recognition performance on the Roboflow Braille Detection Dataset while maintaining real-time responsiveness suitable for assistive applications. By combining multiple accessibility functions into a unified browser-based framework, BrailleVision provides a scalable, privacy-preserving, and user-centric solution that enhances independent information access for visually impaired users without requiring dedicated hardware or continuous internet connectivity.