A Hand Book on Hadoop : Real World Approach for Infinite Scalability

In this Exa byte scale era, data increases at an exponential rate. This is in turn generating a massive amount of metadata in the file system. Hadoop is the most widely used framework to deal with big data. Due to this growth of huge amount of metadata, however, the efficiency of Hadoop is questioned numerous times by many researchers. Therefore, it is essential to create an efficient and scalable metadata management for Hadoop. The book principally deals with the Single Point of Failure (SPOF) of Hadoop by providing a suitable distributed metadata management. This book presents a circular Metadata management mechanism named Dynamic Circular Metadata Splitting (DCMS). DCMS preserves metadata locality using consistent hashing as well as locality-preserving hashing, keeps replicated metadata for excellent reliability, as well as dynamically distributes metadata among the NameNodes to keep load balancing. DCMS removes Hadoop's SPOF as well as provides an efficient and scalable metadata management. This book demonstrates the various experiments with DCMS, and shows how the new framework is an improved solution compared to the existing one, both mathematically and theoretically.