Management of a rapidly evolving large volume of data in a scalable way in distributed environments is very challenging because the transparency of data is not known to the end-user. Two major aspects of database management are to store and query the data in an efficient way. This paper describes a Multi-Block Compressed Data (MBCD) algorithm to handle large volumes of data, incorporating data compression. We evaluate the performance of our proposed (MBCDS) compressed database structure in a centralized system, comparing it to other existing techniques in terms of time and storage space. The compressed database structure presented provides direct addressability in a distributed environment, thereby reducing retrieval latency when handling large volumes of data. We also develop an updating algorithm for the database structure without recompressing the whole database.
International Journal of Computational Science Vol. 3, Issue 4, p. 456-471