An Enhanced RAG Chatbot Using Multi-Metric LLM Evaluation

Abstract

Retrieval-Augmented Generation (RAG) improves the factual accuracy of Large Language Model (LLM) chatbots by grounding responses in external knowledge. However, challenges such as hallucinations, incomplete answers, and poor evaluation methods remain. This project presents an Enhanced RAG-Based Chatbot with Multi-Metric LLM-Based Answer Evaluation. The framework combines efficient document retrieval, context-aware response generation, and automated multi-dimensional evaluation. Using embeddingbased similarity search, it retrieves relevant information and generates responses with an LLM. A multi-metric evaluation module assesses answers based on criteria like semantic correctness, contextual relevance, fidelity to sources, completeness, and language quality, allowing for scalable and consistent assessments with minimal human input. Experimental results show significant improvements in answer reliability and evaluation over traditional single-metric RAG systems, making it suitable for enterprise knowledge systems and domainspecific applications where accuracy is essential.