CLOUD CLEANSE: INTELLIGENT DUPLICATE DATA DETECTION IN CLOUD STORAGE

Abstract

Cloud storage systems have become essential for
managing large-scale data, but they face challenges
related to storage efficiency and data redundancy.
Data deduplication is a widely adopted technique
that eliminates duplicate data by storing only a
single copy, thereby reducing storage consumption
and network bandwidth usage. In particular, blocklevel
deduplication offers higher efficiency by
operating on smaller data units, enabling finegrained
redundancy elimination. However,
implementing secure deduplication in encrypted
environments remains a critical challenge.
Traditional encryption techniques generate different
ciphertexts for identical plaintexts, preventing
duplicate detection. To address this limitation,
message-locked encryption (MLE) has been
introduced, where encryption keys are derived from
the data itself, allowing identical data to produce
identical ciphertexts. Despite its advantages,
existing block-level MLE schemes are vulnerable
to brute-force and dictionary attacks due to the low
entropy of small data blocks. This project proposes
an enhanced privacy-preserving deduplication
system that improves security while maintaining
storage efficiency. The system eliminates the need
for additional trusted key servers and supports
dynamic operations such as data modification,
insertion, and deletion. By integrating secure
hashing, efficient block management, and
encryption techniques, the proposed model ensures
both confidentiality and scalability. The system
architecture includes client, deduplication proxy,
and storage server components to optimize
performance. Experimental analysis demonstrates
improved resistance against attacks, reduced
storage overhead, and efficient data handling.