CLARITYGATE: A FASTAPI-BASED DUPLICATE DETECTION AND TAGGING SYSTEM

Abstract

This project presents a Python-based AI service for detecting duplicate or near duplicate text and providing basic tagging across web platforms. Built with FastAPI, scikit learn, and NumPy, it uses TF-IDF vectorization and cosine similarity to compare new content against an existing corpus in real time. The system exposes simple REST APIs, includes an embeddable JavaScript widget for background checks while users type, and supports configurable thresholds. A proof-of-concept dataset (22 posts) demonstrates functionality; tagging is currently a placeholder endpoint designed for future NLP upgrades. The solution is portable, privacy-friendly (self-hostable), and easy to integrate into forums, support portals, and e-commerce Q&A