Work Flexibility: Hybrid
Lead Engineer, EDT Data & Analytics Cloud Engineering
The Lead Cloud Platform Engineer supports the operational excellence, security, and reliability of the enterprise data & analytics platform on Azure and Databricks. The role combines platform operations, DevOps automation, and Infrastructure as Code across the core stack - Azure Databricks, Synapse, ADLS Gen2, Azure SQL, Azure Data Factory, Key Vault, and Unity Catalog - with a reliability-engineering mindset: automate repeatable work, apply security best practices by default, and help keep the platform highly available for global data engineering and analytics teams.
What you will do:
DevOps Pipeline Management & CI/CD
- Design, build, test, and operate Azure DevOps CI/CD pipelines that package code, run automated quality checks, deploy infrastructure and data workloads, and promote releases across development, testing, and production environments.
- Own day-to-day pipeline execution and release readiness by monitoring runs, diagnosing failed builds and deployments, correcting pipeline definitions, and improving rollback, approval, and promotion controls.
Infrastructure as Code & Environment Provisioning
- Write, review, and maintain Terraform modules for repeatable provisioning of Azure and Databricks platform components; test changes through plan and deployment cycles and keep modules versioned and reusable.
- Provision and configure workspaces, compute, storage, networking, private connectivity, identities, and access through automated IaC processes; detect configuration drift and implement corrective changes.
Platform Security & Access Management
- Configure and troubleshoot RBAC, service principals, managed identities, secrets, storage permissions, cluster policies, and Unity Catalog grants across Azure Data Factory, Databricks, Synapse, Azure SQL, ADLS Gen2, and Key Vault.
- Apply and verify least-privilege access, identity integration, and secrets-management controls through hands-on configuration reviews, access testing, and remediation of security findings.
Platform Operations & Reliability Engineering
- Build and tune monitoring, alerts, dashboards, and operational runbooks; participate directly in incident response, perform root-cause analysis, and implement preventive automation and reliability fixes.
- Monitor integration runtimes, clusters, jobs, pipelines, and platform services; inspect logs and metrics, reproduce failures, resolve runtime and performance issues, and validate service recovery against SLAs.
Data Engineering Operations
- Work directly with Data Platform and Data Product teams to deploy and operationalize ingestion pipelines, orchestration, Databricks jobs, and supporting platform services; provide hands-on support through production rollout.
Collaboration
- Translate stakeholders into implementable technical changes, contribute code and configuration, lead technical working sessions, and document the resulting solution and operational handoff.
What you need:
- Bachelor’s or master’s degree in computer science, Engineering, Information Systems, Business, or a related technical discipline.
- 5-7 years of experience designing, engineering, and supporting cloud-based data and analytics platforms, data warehouses, and modern data ecosystems.
- Strong hands-on expertise with Azure data services, including Azure Databricks, Synapse Analytics, ADLS Gen2, Azure SQL, Azure Data Factory, Key Vault, and Unity Catalog.
- Proven experience implementing platform security controls, governance frameworks, and Azure data platform best practices to ensure secure, scalable, and compliant solutions.
- Proficiency in Infrastructure as Code (IaC) using Terraform and Azure DevOps CI/CD pipelines, with experience automating deployments across multiple environments.
- Demonstrated ability to monitor, optimize, and troubleshoot cloud data platforms, including performance tuning, alerting, incident management, and root cause analysis.
- Excellent analytical, problem-solving, and communication skills, with the ability to collaborate effectively across engineering, operations, security, and business stakeholders.
- Proactive mindset with a track record of driving automation, process optimization, operational improvements, and scalable platform enhancements that increase efficiency and reliability.
- Cloud Engineering certifications on Azure/Databricks platform is a plus.
Travel Percentage: None