Summary
Overview
Work History
Education
Skills
Websites
Certification
Timeline
Generic
Anay Mishra

Anay Mishra

Data Engineer
Berlin

Summary

Highly motivated Data Engineer with over 5 years of experience across various industries. Proficient in designing data solutions, building and maintaining data pipelines, developing interactive dashboards, and creating data models. Experienced in Azure and AWS cloud platforms for data engineering, including data integration, storage, and processing. Currently pursuing Master’s in Data Analytics, seeking a part-time or working student role in Germany to advance my expertise and contribute to impactful projects.

Overview

6
6
years of professional experience
6
6
years of post-secondary education
7
7
Certifications
3
3
Languages

Work History

Senior Data Engineer

KPMG India
Gurugram
03.2022 - 04.2024
  • Collaborated with Data Architects and Engineers to develop and deploy solutions in Azure Databricks using Python, PySpark, and Spark SQL, following Medallion Architecture principles.
  • Engineered ETL pipelines in Azure Data Factory and Synapse reducing operational effort by 50%.
  • Implemented a robust logging mechanism using Synapse pools and SQL Server Management Studio (SSMS), cutting troubleshooting time by 60% and enabling Power BI dashboards for real-time monitoring.
  • Designed interactive and dynamic dashboards in Power BI, showcasing key performance indicators (KPIs) and actionable insights for stakeholders.
  • Utilized DAX expressions, data modeling, and Power Query to ensure accurate and efficient data representation.
  • Integrated data from Azure Synapse, Data Lake, and external sources to create consolidated dashboards, improving business decision-making.
  • Created row-level security (RLS) configurations to ensure data confidentiality and appropriate access levels for users.
  • Designed bronze, silver, and gold data layers for ingestion from Azure Blob Storage and SFTP servers into Azure Data Lake, ensuring efficient data processing and governance.
  • Utilized Docker and Docker Compose to containerize solutions and implemented CI/CD pipelines for seamless deployments across environments.
  • Oversaw the end-to-end lifecycle of solutions from Development to UAT and Production, ensuring seamless environment transitions, strict adherence to deployment standards, and consistent delivery of high-quality solutions.
  • Authored comprehensive project documentation, adhering to business guidelines to facilitate knowledge transfer and workflow continuity.
  • Mentored new team members, fostering a collaborative team environment and ensuring successful onboarding and project delivery.

Data Engineer

Tata Consultancy Services
Noida
02.2018 - 02.2022
  • Interacted with end customers to gather requirements for designing and developing a common architecture for enterprise-wide customer data storage and building a Data Lake in Azure Cloud.
  • Designed and implemented scalable data pipelines using Azure Data Factory and Azure Databricks, reducing processing time by 45%.
  • Executed advanced data cleaning techniques, including de-duplication, normalization, and validation, ensuring high-quality and accurate datasets.
  • Planned and developed a web scraping architecture to extract data from PDFs, online tables, and Excel files on hourly, daily, and weekly basis, reducing manual effort by 80%.
  • Re-architected, developed, and automated more than 25 critical pipelines using Azure Data Factory, Databricks, PySpark, and Spark SQL, achieving a 30% project cost reduction.
  • Developed and maintained data warehouses and data lakes with Azure Synapse Analytics and Azure Data Lake Storage, ensuring optimal performance and scalability.
  • Monitored active pipelines in production, promptly resolving issues to ensure smooth data flow and data integrity.
  • Designed custom Power BI reports and dashboards tailored to meet specific business needs, delivering comprehensive insights into operational and strategic data.
  • Connected diverse data sources, such as Azure Data Lake, Synapse Analytics, and Excel-based datasets, to build unified dashboards with consistent and accurate data.
  • Enhanced dashboard performance by using optimized measures, aggregations, and pre-processed datasets, reducing load times by up to 40%.

Education

Masters in Data Analytics - Data Analytics

Berlin School Of Business And Innovation
Berlin
05.2024 - 11.2025

Bachelor of Technology - Electronics And Communication

Pranveer Singh Institute Of Technology
Kanpur
05.2013 - 06.2017

Skills

Database: MS SQL, Oracle SQL, NoSQL

Programming Language: Python, Apache PySpark, Spark SQL

Cloud Ecosystem: Azure: Databricks,Azure Synapse,ADF,ADLS Gen2

AWS: EC2, Redshift, S3, QuickSight

Web Scrapping: Selenium, BeautifulSoup

Python Libraries: Pandas, Numpy, Matplotlib, Seaborn, Tabula-py

Visualization Tools: Power BI, Excel

CI/CD: GIT, Jenkins

Certification

Microsoft AZURE AZ-900(Azure Fundamentals)

Timeline

Masters in Data Analytics - Data Analytics

Berlin School Of Business And Innovation
05.2024 - 11.2025

Senior Data Engineer

KPMG India
03.2022 - 04.2024

Data Engineer

Tata Consultancy Services
02.2018 - 02.2022

Bachelor of Technology - Electronics And Communication

Pranveer Singh Institute Of Technology
05.2013 - 06.2017

Similar Profiles

CREATE PROFILE
Anay MishraData Engineer