Senior Specialist, External and Scientific Engagement (ESE) Data Engineer

Job Description

Senior Specialist, ESE Data Engineer

The Opportunity

· Based in Hyderabad, join a global healthcare biopharma company and be part of a 130- year legacy of success backed by ethical integrity, forward momentum, and an inspiring mission to achieve new milestones in global healthcare.

· Be part of an organisation driven by digital technology and data-backed approaches that support a diversified portfolio of prescription medicines, vaccines, and animal health products.

· Drive innovation and execution excellence. Be a part of a team with passion for using data, analytics, and insights to drive decision-making, and which creates custom software, allowing us to tackle some of the world's greatest health threats.

Our Technology Centres focus on creating a space where teams can come together to deliver business solutions that save and improve lives. An integral part of our company’s IT operating model, Tech Centres are globally distributed locations where each IT division has employees to enable our digital transformation journey and drive business outcomes. These locations, in addition to the other sites, are essential to supporting our business and strategy.

A focused group of leaders in each Tech Centre helps to ensure we can manage and improve each location, from investing in growth, success, and well-being of our people, to making sure colleagues from each IT division feel a sense of belonging to managing critical emergencies. And together, we must leverage the strength of our team to collaborate globally to optimize connections and share best practices across the Tech Centres.

Role Overview

The Data Engineer will serve as a key internal expert responsible for modern data engineering across our company Products. This role provides a unified, scalable, and high-quality approach to data across multiple business areas and strengthening consistency in data pipelines, data models, and architecture patterns.

The Data Engineer is responsible for designing, building, and operating reliable, scalable data pipelines and curated data layers that enable analytics and data-driven products. This role partners closely with upstream data owners, QA, business users and DevOps to ensure data is delivered accurately, tested, and deployed.

The Data Engineer owns technical design documentation and source-to-target mappings, resolves production issues, and provides timely status updates through Jira.

In addition, this position will lead and mentor an offshore development team, coordinating tasks and priorities, ensuring code quality and on-time delivery. Strong communication and team-working skills are essential: the role requires clear, proactive communication across time zones, the ability to provide constructive feedback, and the capability to foster collaboration and knowledge sharing among onshore and offshore teammates.

What will you do in this role

Data Engineering & Pipeline Development

· Design, implement, and operate scalable, reliable data pipelines and curated consumption layers to meet analytics and business needs using the team’s Databricks platform and PySpark/Spark Data Frame transformations.

· Develop clean, modular Python code and SQL-based models for data processing, validation, and consumption; build and maintain automated tests to ensure correctness and prevent regressions.

· Orchestrate, schedule, and monitor end-to-end workflows using Airflow; ensure pipelines are idempotent, resilient, and have clear retry and alerting behavior.

· Optimize performance and cost by applying best practices in Spark processing, SQL tuning, and Delta Lake/table management on Databricks.

· Use GitHub for source control, code reviews, and CI/CD integration; maintain clear commit history, pull requests, and versioned releases.

Data Modeling & Architecture

· Develop and maintain data models (relational, analytical, or lakehouse) that support high-quality, well-structured data.

· Lead the definition of data integration patterns and cross-domain consistency—reducing fragmentation between several business areas.

· Partner with architecture and product teams to align evolving data needs with the technical roadmap.

Performance, Quality, & Governance

· Monitor, tune, and enhance data performance across ingestion, transformation, and compute layers.

· Implement best practices for quality assurance, including automated data quality checks, lineage capture, and validation processes.

· Contribute to documenting data architecture, data definitions, and engineering standards to support strong governance.

Collaboration & Leadership

· Work closely with product teams, analysts, and data consumers to translate business requirements into scalable engineering solutions.

· Collaborate with DevOps and cloud engineering teams to ensure data pipelines (database schema deployments, version control, infrastructure-as-code, etc.) operate seamlessly within CI/CD environments.

· Lead and mentor offshore developers: delegate tasks, review code, provide feedback, run knowledge transfers, and maintain effective cross–time-zone communication.

· Collaborate with upstream source teams, QA, business stakeholders, and DevOps to coordinate schema changes, support testing/UAT, and validate deployments.

· Troubleshoot and resolve production issues and bugs; perform root-cause analysis and implement fixes and preventive measures.

· Author and maintain technical design documents, source-to-target mappings, runbooks, and operational playbooks; keep JIRA updated daily with progress, blockers, and estimates.

· Enforce data quality, governance, and security standards across pipelines and datasets, and ensure lineage and metadata are maintained.

What should you have

· Bachelor’s degree in data engineering, Computer Science, Information Systems, or related field preferred.

Primary Skills: Python,PySpark,SQL,DBT/Data bricks/Airflow/Spark, Technical Lead

Secondary Skills: Unity Catalogue, Apache Atlas ,Delta Lake

Technical Expertise

· 7+ years of hands-on experience in data engineering or large-scale data pipeline development.

· Strong proficiency in SQL and modern data processing tools (Databricks, Airflow, DBT, Spark, or similar).

· Experience designing and maintaining production-grade data pipelines in cloud environments.

· Solid understanding of ETL/ELT principles, data modeling, distributed processing, and data orchestration frameworks.

· Experience implementing dbt for transformation layering and model testing.

· Ability to troubleshoot complex data issues across ingestion, transformation, and processing layers.

· Familiarity with DevOps concepts, CI/CD integration, and source-controlled data workflows.

· Expert T-SQL: Mastery of DDL, DML, CTEs, and window functions for complex data manipulation.

· Azure SQL Database: Deep knowledge of PaaS offerings, including scaling, security, and backup/recovery.

· Azure Data Factory: Experience with building pipelines, triggers, parameters, and linked services for orchestration.

· Automation: Proficiency in database management, elastic job agent, and infrastructure automation.

· Infrastructure as Code (IaC): Familiarity with ARM Templates, Bicep, or Terraform.

· Familiarity with CI/CD pipelines and modern build, deployment, and release practices.

· Strong documentation habits and experience producing technical design documents, source-to-target mappings, and runbooks.

· Excellent communication and collaboration skills, with experience coordinating across onshore and offshore teams and supporting QA/UAT activities.

Soft Skills

· Strong analytical problem-solving skills with a focus on long-term maintainability.

· Clear communicator capable of working effectively with cross-functional technical and business teams.

· Proactive, ownership-driven, and committed to high-quality engineering practices.

---

Preferred Qualifications

· Hands-on experience with Databricks Delta Lake features (time travel, ACID transactions, ZORDER) and Spark performance tuning.

· Familiarity with monitoring and observability tools (Databricks metrics, Prometheus, Grafana, or cloud-native equivalents).

· Experience with data governance, lineage, and catalog tools (Unity Catalog, Apache Atlas, or similar).

· Prior experience leading or mentoring offshore development teams.

Our technology teams operate as business partners, proposing ideas and innovative solutions that enable new organizational capabilities. We collaborate internationally to deliver services and solutions that help everyone be more productive and enable innovation.

Who we are

We are known as Merck & Co., Inc., Rahway, New Jersey, USA in the United States and Canada and MSD everywhere else. For more than a century, we have been bringing forward medicines and vaccines for many of the world's most challenging diseases. Today, our company continues to be at the forefront of research to deliver innovative health solutions and advance the prevention and treatment of diseases that threaten people and animals around the world.

What we look for

Imagine getting up in the morning for a job as important as helping to save and improve lives around the world. Here, you have that opportunity. You can put your empathy, creativity, digital mastery, or scientific genius to work in collaboration with a diverse group of colleagues who pursue and bring hope to countless people who are battling some of the most challenging diseases of our time. Our team is constantly evolving, so if you are among the intellectually curious, join us—and start making your impact today.

#HYDIT2025

Required Skills:

Data Engineering, Data Management, Data Modeling, Data Visualization, Design Applications, Information Management, PySpark, Python (Programming Language), Software Development, Software Development Life Cycle (SDLC), System Designs

Preferred Skills:

Current Employees apply HERE

Current Contingent Workers apply HERE

Search Firm Representatives Please Read Carefully 
Merck & Co., Inc., Rahway, NJ, USA, also known as Merck Sharp & Dohme LLC, Rahway, NJ, USA, does not accept unsolicited assistance from search firms for employment opportunities. All CVs / resumes submitted by search firms to any employee at our company without a valid written search agreement in place for this position will be deemed the sole property of our company.  No fee will be paid in the event a candidate is hired by our company as a result of an agency referral where no pre-existing agreement is in place. Where agency agreements are in place, introductions are position specific. Please, no phone calls or emails. 

Employee Status:

Regular

Relocation:

VISA Sponsorship:

Travel Requirements:

Flexible Work Arrangements:

Hybrid

Shift:

Valid Driving License:

Hazardous Material(s):

Job Posting End Date:

04/22/2026

*A job posting is effective until 11:59:59PM on the day BEFORE the listed job posting end date. Please ensure you apply to a job posting no later than the day BEFORE the job posting end date.