Skip to main content

What is Data Lake and Azure Data Lake

What is data lake ?
A data lake is a storage repository that holds a vast amount of raw data in its native format until it is needed. While a hierarchical data warehouse stores data in files or folders, a data lake uses a flat architecture to store data. Each data element in a lake is assigned a unique identifier and tagged with a set of extended metadata tags. When a business question arises, the data lake can be queried for relevant data, and that smaller set of data can then be analyzed to help answer the question.

A data lake, on the other hand, maintains data in their native formats and handles the three Vs of big data — volume, velocity, and variety — while providing tools for analyzing, querying, and processing. Data lakes eliminate all the restrictions of a typical data warehouse system by providing unlimited space, unrestricted file size, schema on read, and various ways to access data (including programming, SQL-like queries, and REST calls).

What is Azure Data Lake ?
Azure Data Lake is a Bigdata and data-warehouse platform on azure cloud. It is a highly scalable public cloud service that allows developers, scientists, business professionals and other Microsoft customers to gain insight from large, complex data sets. As with most data lake offerings, the service is composed of two parts: data storage and data analytics.

The Azure Data Lake service was released on November 16th, 2016. Azure Data Lake is built on the learnings and technologies of COSMOS, Microsoft’s internal big data system. COSMOS is used to store and process data for applications such as Azure, AdCenter, Bing, MSN, Skype and Windows Live. COSMOS features a SQL-like query engine called SCOPE upon which U-SQL was built.



Thanks for reading
Plz dont forget to like Facebook Page..
https://www.facebook.com/pages/Sql-DBAcoin/523110684456757

Comments

Popular posts from this blog

mongoDB error : aborting after fassert() failure

What to do when facing errors on mongoDB “aborting after fassert() failure”

I like errors, in mongoDB this is the first error I faced and luckily many times. This error i faced during restoring name-space on local and restarting db system. I am still searching the exact root cause of this issue but i am able to resolve the current problem through below steps.

Remove all relevant namespace files from data-file route path..Now repair mongo instance using mongod process.mongod --repair ////////// execute command from bin folder path Then start server using mongd process, if started server successfully then ..mongod  ////////// execute command from bin folder path Restore last backups as normal process.Now check database by connecting mongo shell. Thanks for reading, 
Please comment your experience if you faced and also share knowledge if you have better steps to resolve...


https://www.facebook.com/pages/Sql-DBAcoin/523110684456757

SQL71562: external references are not supported when creating a package from this platform

Last week I got this error from one of developer who was trying to deploy his project from Testing server to SQL Azure QA server. He was using “Deploy Database to SQL Azure” option from SSMS Tool-Task option.

After connecting to SQL Azure portal when operation started to deployment below errors occurs.

Validation of the schema model for data package failed. Error SQL71562: Error validating element xx.xxx.xx:function .dbo.xxx has an unresolved refrence to object xx.dbo.xxxx external refrences are not supported when creating a package from this platform.



Reason: The reason of the this error was; some functions of project was dependent on master database and only single database was being deploy to SQL Azure. DACFx must block Export when object definitions (views, procedures, etc.) contain external references, as Azure SQL Database does not allow cross-database external references So, this error was coming.

Solution : I suggested him to create those function to locally on local database what…

How to recover msdb database from suspect mode

It was Monday 9th Jun 47 degr. temperature of Delhi-NCR. Temperature was like boiling me and database. When I reached my office( @ 8.45 am) got an alert from one of Server.
“MSDB is in suspected mode” At the same time comes in my mind, this issue will boil me today.. I just tried to cool my self through cold drink then connected server from my local system using windows authentication mode..