Get a quick insight of a data lake, Is Datalake is a flexible platform for BigData Management?? A data lake can be a much more flexible repository than the traditional data warehouses. Or it can be a trash dump that grows and grows Data lake defined A data lake holds a vast amount of raw, unstructured data in its native format. Data lake vs. data warehouse Data repositories are nothing new; data warehouses have been around for decades. And while it is natural to compare data warehouses to data lakes, there are fundamental differences that separate data warehouses from data lakes, ranging from the kind of data stored to how it is processed. Data lakes don’t require specialty hardware One of the key differences between a data lake and a data warehouse is that a data lake does not require special hardware or software, unlike a data warehouse. Data lakes are more flexible As noted, a data lake holds a vast amount of raw, unstructured data in its native format, whereas the data warehouse is much more structured into folders, rows, and columns. As a result, a data lake is much more flexible about its data than a data warehouse is. A flexibility-related difference between the data lake and the data warehouse is schema-on-read vs. schema-on-write. A schema is a logical description of the entire database, with the name and description of records of all record types. A data warehouse applies schema-on-write, so you have to know exactly how to structure the data before you save it. That means a lot of preparation before intake, or at least before storage. By contrast. data lakes apply schema-on-read, so you can format it as you read and process it. Schema-on-read means you can throw everything into a bucket, like log files, web files, or things with no meaningful structure, and then figure it out later. Unlike a data warehouse, data lakes don’t have an underlying database. Instead, data lakes use a flat file system. With a database, you have to choose data and columns before you write to it. The trade-off is that it might take a while to insert the data into a database, but when you do a query it is a lot faster than in a data lake, which has to process the data as it is read. “With a data lake, you can put data into a store any way you like. That allows you to write data with a flexible schema and query later, but orders of magnitude slower, ” said Stein. “The one element those servers don’t do well is metadata management. Things like what goes in which folder, when is it aged out. You have to roll your own when doing a service like that.” That has since changed, and traditional companies lke TeraData and Oracle offer commercial data lake products, as do specialized big data vendors like Hortonworks and Cloudera. Amazon, Microsoft, Google, and IBM all offer a variety of data lake tools along with their basic cloud storage services, so you can build your data lake on premises or in the cloud. Other commercial data lake products include: When to avoid a data lake A data lake is not for everyone. Some companies may not need it, and it might make things worse. For example, Hiskey says data lakes are not for real-time work.