For one of my batch where I was teaching AWS data engineering, I gave this architecture as a mini capstone project.

Idea was to give students the idea of different services, how you can process batch and stream data using PaaS.
The processing would have been done using EMR…

This is a standard data lake architecture that I suggest to my clients.

If you have followed this, I’m sure your data pipeline would be robust.

This is called a medallion architecture. Databricks is promoting the medallion architecture for data Lakehouse.

However this has been in place for many years now.
Earlier people had 4 layers - raw/bronze, (refined and integrated)/silver, presentation/gold

Reply to know more…

Focus on interaction and less on views. Anyone who makes more than 5% should come forward if they have the courage

People follow to gain followers and then unfollow!

This should be illegal 😅

I asked Grok when will I reach 500 followers! This is what it replied -

3 short of 100 followers!
Can I get 3 followers today? 🙏🏻🤞🏻

Why Poisson Distribution is not treated the same way as Normal Distribution 👀

I was inconsistent on X:

Reasons(made up) -
🔹I am stuck into 9-5(10-9 in my case)
🔹I have to give time to my family
🔹I have to meet friends
🔹I don’t get enough sleep

What I changed -
🔹Posting 5 times a day
🔹Have a clear direction on what to post
🔹Share my experiences