Gobu Shanmugam

Experienced Senior Data Engineer with over 16 years of expertise in Data Engineering, designing and architecting Cloud solutions in AWS, building complex data pipelines, designing and developing SAAS platforms across diverse domains. Proficient in AWS Sevices, SQL, Python, ETL processes, and big data technologies. Demonstrated success in managing complex data engineering projects and leading cross-functional teams to achieve business objectives.

• Application Integration (Life Sciences), 03/2024 - Present, Architected and implemented an automated interface between Veeva CRM and Zoom using AWS services like Lambda, Glue, and Step Functions to eliminate manual tasks. Leveraged REST APIs for seamless integration, ensuring real-time data synchronization. Defined CI/CD pipelines with Azure DevOps for continuous integration and deployment. Implemented automated data health checks using Step Functions, enhancing the data processing pipeline. Utilized Terraform for infrastructure-as-code, ensuring consistent and scalable deployments.
• Data Integration (Life Sciences), 01/2023 - 02/2024, Architected and Led the development of multiple interfaces between diverse systems within the CPH landscape, including Veeva CRM, Veeva Align, Veeva Network, SFMC, Content Factory, CDP, SAP, and Cvent. Utilized AWS services such as Glue, Lambda, and Step Functions to streamline data integration and processing. Employed SOAP and REST APIs to facilitate data exchange between systems. Enhanced the parsing of configuration files using Python, providing greater flexibility in configuration options. Ensured robust security and compliance measures across all integrations.
• Talend to AWS Migration (Life Sciences), 01/2021 - 11/2022, Designed and architected the migration of data integration interfaces from Talend to AWS services, including Glue and Lambda, using Python. Designed and implemented complex data pipelines to ensure efficient data processing and transformation. Leveraged AWS serverless services to enhance scalability and reduce infrastructure costs. Employed agile methodologies and tools like Azure DevOps for project management and continuous delivery. Ensured thorough documentation and provided training to team members, fostering a culture of knowledge sharing and continuous improvement.
• MasterCard DW clearing system (Banking), 06/2019 - 12/2020, A migration project from Oracle on-prem to PostgreSQL database in AWS cloud. Built complex data pipelines in Python and PostgreSQL stored procedures to clone the existing pipeline but also provide a modernized cloud adoption by reducing the time to load data and having cost savings in place