How to Unlock Value from DataThe old cliche, ‘Data is the New Oil’, is even more relevant now in the age of AI revolution as it was in the age of Big Data revolutionAug 22, 2024Aug 22, 2024
Automating Data Engineering Workloads Using Generative AIPart -1: A Brief Overview of Large Language ModelsNov 9, 2023Nov 9, 2023
How to Design & Build Data Governance & Quality FrameworkData governance is an important milestone in any company’s data journey and anytime resources & time are allocated for data governance…Apr 10, 2023Apr 10, 2023
We have been working with data for 40+ years. Why are most data systems still chaotic & complex ?Databases and associated systems have been around in one form or another since 1980s. The technology has been widely adopted and used…May 11, 20221May 11, 20221
Changing Winds in Data Analytics, Data Engineering & Machine LearningCloud has enabled big data processing at scaleMar 23, 2022Mar 23, 2022
Cloud Data Architecture: Guiding Principles & Best Practices1. Separation of Storage from ComputeFeb 24, 2022Feb 24, 2022
A Brief Primer On Data LakeData Lakes & Warehouses have been around for a while, and there is a lot of literature & theory on what works and what doesn’t. I have been…Nov 12, 2021Nov 12, 2021
Cloud Data Platform: Design ChoicesSay you are enterprising data engineer working for an upwardly mobile and tech savvy company. You have made all the right choices in tech…Jun 21, 2021Jun 21, 2021
Design Philosophy for building Big Data SystemsSoftware Engineering has come a long way from the dark days of top down management, waterfall approach, MS Project, Gantt chart (shiver!)…Sep 25, 2020Sep 25, 2020
Data Strategy for Big Data AnalyticsData strategy is the bedrock of any BI Analytics solution, in particular we are guided by the following principlesJan 2, 2020Jan 2, 2020
Database Evolution — from RDBMS to Big DataDesigning a data pipeline can be a serious business, building it for a Big Data based universe, however, can increase the complexity…Mar 7, 2019Mar 7, 2019
Design & Strategies for Building Big Data Pipelines(Root of all data engineering evil lies in the unnecessarily complex data processing pipeline)Feb 4, 20197Feb 4, 20197
Next Generation Data Integration Tools for Big Data ETL PipelineIn the realm of Big Data, there has been an explosion in number of sources that can generate data and targets that can consume. As a…Jan 29, 2019Jan 29, 2019
