A SEMANTIC WEIGHT ADAPTIVE MODEL BASED ON VISUAL QUESTION ANSWERING

Authors

  • Dr. M. V. NARAYANA1 , NITHIN REDDY2 , RAHUL3 , HARSHA VARDHAN REDDY4 , PRAVEEN 5 Author

DOI:

https://doi.org/10.5281/zenodo.21373396

Abstract

This project presents a multilingual Visual Question Answering (VQA) web application built with Flask, integrating deep learning and NLP techniques. It utilizes the Salesforce BLIPVQA model, which combines a Vision Transformer and text decoder to answer questions based on uploaded images or videos (up to 60 seconds, given sufficient GPU and memory). Users can interact in various Indian languages like Hindi, Tamil, and Telugu, with a mock translation service converting questions to English for processing and translating answers back to the original language. The system supports secure file uploads, validates inputs, and extracts keyframes from videos to generate context-aware answers. This real-time, multilingual application demonstrates the potential of transformer-based VQA models in accessible and scalable web environments, with room for future integration of services like IndicTrans2. A key highlight of the system is its multilingual support, allowing users to ask questions in a variety of Indian languages such as Hindi, Tamil, Telugu, and others. To achieve this, a mock translation service is integrated to simulate the translation of the user's input question to English (the primary language used by the VQA model) and then translate the modelgenerated English answer back to the user's original language. This simulates an end toend multilingual pipeline and demonstrates the feasibility of extending the system with real translation services like in dictrans.

Downloads

Published

15-07-2026

How to Cite

A SEMANTIC WEIGHT ADAPTIVE MODEL BASED ON VISUAL QUESTION ANSWERING. (2026). International Journal of Engineering Research and Science & Technology, 22(2(4), 1774-1783. https://doi.org/10.5281/zenodo.21373396