NTT DATA
|Senior Data Engineer
Pune, Maharashtra, India
Summary
Led data engineering initiatives for ExxonMobil's Supply Chain Analytics, driving cloud adoption and optimizing data workflows across global teams.
Highlights
Spearheaded a global team of 7-8 junior data engineers across India, US, and Brazil, leading ExxonMobil's Supply Chain Analytics initiatives and fostering team development.
Directed UAT phases onsite in Brazil and Vietnam, successfully migrating legacy on-prem databases to Snowflake, achieving seamless cloud adoption and enhancing client confidence.
Engineered robust data pipelines using Azure Data Factory to ingest diverse structured and semi-structured data (CSV, JSON, Parquet) into Fabric Lakehouse, leveraging PySpark and Python/Pandas for transformations to create curated Delta tables in the Silver layer of a Medallion architecture.
Architected Gold layer models within the Data Warehouse using dimensional modeling (Star and Snowflake schemas) on curated Delta tables, implementing partitioning, clustering, and performance tuning to optimize BI tool connections.
Optimized long-standing Snowflake data flow code, reducing runtime and compute usage by 49.4% and achieving monthly cost savings of ~$3,631 USD, earning executive recognition.
Enabled secure data sharing across partner teams, eliminating data pipeline and duplication efforts, thereby enhancing cross-business data collaboration and efficiency.
Developed and automated a complex email notification system for Snowflake task success/failure using Snowpark, Azure Data Factory, Azure Data Lake Storage, and Logic Apps, improving operational visibility.
Enhanced data delivery capabilities by implementing advanced Snowflake features including Snowpipe, Stream, Time Travel, Cloning, Sharing, Tasks, Materialized Views, External Tables, Procedures, and CDC.