Azure Data Lake- Implementations - Benefits

Question

Hey Everyone Could explain about the Azure Data Lake how to implement this what are the benefits of same

Manikandan Murugesan · Answer

Benefits: Because you want to know the benefits before using it. 1. True HDFS Compatibility Azure Data Lake is a true Hadoop file system compatible with the major Hadoop distributions like Hortonworks Data Platform and Cloudera Enterprise Data Hub and projects like Spark Storm Flume Sqoop Kafka etc. 2. Unlimited Data Size Other cloud storage options have file and account size limitations of only a few terabytes. This may sound like a lot but in big data environments this is quite small. Azure Data Lake removes this constraint: There is no fixed size limit on files or accounts. You can store single or multiple files at terabyte and petabyte scale and process them with Hadoop. This is an ideal situation because Hadoop is at its best when working with very large files. 3. Fault-Tolerant and Available In a typical Hadoop implementation data is replicated three times in the cluster. Azure Data Lake adopts this practice in order to ensure the data is available in the event of hardware failure or interruption. 4. Designed for Parallel Processing One of the key benefits of the Hadoop file system is it keeps the data close to the compute. In an on premises implementation this is done by having dedicated disk in each physical node and moving data between nodes as little as possible. Azure Data Lake is designed to support throughput for parallel processing scenarios in order to provide performance similar to that of an on premises Hadoop implementation. 5.Optimized for High-Speed Throughput One of the hottest topics in data management is the Internet of Things (IoT). IoT boils down to data streaming off devices such as phones sensors and equipment. This type of data consists of very small transactions at very high volume. Ingestion of this data into a data management system has traditionally been very difficult both from the application development and hardware and software support standpoints. Azure Data Lake supports extremely high speed data ingestion at huge scale. 6. Enabling Hadoop for the Cloud I’m excited to see what Azure Data Lake does for Hadoop in the cloud. Many enterprises implementing Hadoop choose an on premises implementation because of the perceived limitations of cloud infrastructure. Azure Data Lake should bring cloud of limitations of Hadoop more in parity with traditional installations. For Implementation: Refer the following Link : https://docs.microsoft.com/en-us/azure/data-lake-store/