Transformer-Driven Intelligent Image Understanding and Question Answering System
DOI:
https://doi.org/10.62643/Keywords:
Image Captioning, Visual Question Answering (VQA), Deep Learning, BLIP Model, ResNet50, Transformer Models, Medical Image Analysis, Natural Language Processing, GUI Application, Knowledge-Based SystemsAbstract
The rapid advancement of artificial intelligence has significantly transformed the field of image analysis, enabling systems to not only recognize visual content but also interpret and describe it in natural language. This paper presents an intelligent image analysis and Visual Question Answering (VQA) system that integrates deep learning and transformer-based models to provide automated image understanding, caption generation, and user-interactive query responses. The system is designed to handle both medical and general image datasets, making it versatile for multiple real-world applications such as healthcare diagnostics, wildlife recognition, and intelligent assistants. The proposed system utilizes a convolutional neural network, specifically ResNet50, for image classification tasks. Pretrained models are fine-tuned to categorize images into predefined classes such as medical conditions or animal types. To enhance interpretability, the system incorporates the BLIP (Bootstrapping Language-Image Pretraining) model for generating human-like captions that describe the content of the image. This enables users to gain contextual understanding of visual data without manual inspection. Furthermore, the system integrates a transformer-based language model (FLAN-T5) to support Visual Question Answering. Users can input queries related to the selected image, and the system generates meaningful answers by leveraging natural language processing capabilities. This combination of vision and language models creates a knowledge-based interactive system that bridges the gap between image recognition and semantic understanding. A user-friendly graphical interface is developed using Tkinter, allowing seamless interaction with the system. Users can upload images either from local storage or via URL, input questions, and receive results including captions, classifications, and answers in real-time. Additionally, a SQLite database is employed to manage user authentication and store analysis results, ensuring data persistence and usability. Experimental evaluation demonstrates that the system effectively combines multiple AI techniques to deliver accurate image classification, meaningful captions, and context-aware responses. The integration of deep learning and transformer models enhances the system’s performance and adaptability across different domains. In conclusion, this work highlights the potential of combining computer vision and natural language processing for intelligent image analysis. The proposed system provides a scalable and efficient framework for developing advanced AI applications capable of understanding and interacting with visual data.
Downloads
Published
Issue
Section
License

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.













