Fusion of Human Gaze and Machine Vision for Predicting Intended Locomotion Using Deep Learning

Authors

  • DYVALA MEGHANA, V.SARALA Author

DOI:

https://doi.org/10.62643/

Keywords:

Human Gaze, Machine Vision, Locomotion Prediction, Deep Learning, Convolution Neural Network (CNN), GENet, Computer Vision, Behavioral Analysis, Video Processing, Human-Computer Interaction

Abstract

Predicting human locomotion intent is a crucial task in the domains of assistive robotics, autonomous navigation, and human-computer interaction. The integration of human gaze information with machine vision offers a promising pathway to enhance prediction accuracy and contextual understanding of human movement. This project proposes a deep learning-based framework that combines visual data processing with behavioral inference to predict intended locomotion modes in real time. The system utilizes a Convolutional Neural Network (CNN)-based architecture, specifically a customized GENet model, to extract spatial features from input images and video frames. The dataset consists of labeled images categorized into different locomotion types such as walking, running, standing, and turning. These images are preprocessed through normalization and resizing techniques to ensure uniformity and improve model performance. The processed data is then split into training and testing subsets to evaluate the generalization capability of the model. The GENet architecture employed in this study includes multiple convolutional layers, pooling layers, batch normalization, and fully connected dense layers. These components work together to learn hierarchical feature representations from the input data. The model is trained using categorical cross-entropy loss and optimized using the Adam optimizer. Performance metrics such as accuracy, precision, recall, and F1-score are used to assess the effectiveness of the model. Additionally, confusion matrices are generated to visualize classification performance across different locomotion categories. A key feature of the system is its ability to process real-time video input for forecasting intended locomotion. By capturing frames from a video stream, the model predicts the next locomotion state dynamically. This capability makes the system suitable for realworld applications such as assistive devices for visually impaired individuals, smart surveillance systems, and autonomous robots. The fusion of human gaze and machine vision enhances contextual awareness, allowing the system to make more accurate predictions compared to traditional vision-only approaches. The proposed system demonstrates significant improvements in classification performance and real-time prediction capabilities. Overall, this research contributes to the advancement of intelligent systems that understand human intent by combining visual perception and behavioral cues. Future work may focus on integrating additional modalities such as depth sensing and eye-tracking hardware to further improve prediction accuracy and robustness.

Downloads

Published

04-04-2026

How to Cite

Fusion of Human Gaze and Machine Vision for Predicting Intended Locomotion Using Deep Learning. (2026). International Journal of Engineering Research and Science & Technology, 22(2), 949-965. https://doi.org/10.62643/