Principal data engineer

Role Value Proposition:

We are seeking a skilled Principal Data Platform Engineer with 5+ years of experience in designing, building, implementing, and managing data, analytics, and MLOps platform on Azure Cloud. Will be responsible for automating infrastructure, ensuring platform reliability, optimizing costs, and supporting data engineering, analytics, and machine learning workloads.

Key Responsibilities:

Platform Architecture & Design:

• Design and architect scalable, secure, and cost-effective data, analytics, and MLOps platforms on Azure

• Define cloud architecture patterns and best practices for data workloads

• Create technical specifications and solution designs for complex cloud infrastructure

• Evaluate and recommend new technologies and tools to enhance platform capabilities

Infrastructure Automation & Management:

• Implement Infrastructure as Code (IaC) using Azure Resource Manager (ARM), Bicep, or Terraform

• Develop and maintain CI/CD pipelines for infrastructure deployment and application delivery

• Automate provisioning and configuration management using tools like Azure DevOps, GitHub Actions, or Jenkins

• Design and implement automated backup, disaster recovery, and high availability solutions

Data Platform Operations:

• Deploy and manage Azure data services including Azure Data Factory, Azure Synapse Analytics, Azure Databricks, and Azure Data Lake

• Implement data governance, security, and compliance frameworks

• Monitor and optimize data pipeline performance and resource utilization

• Ensure data quality, lineage, and metadata management across the platform

MLOps Platform Development:

• Build and maintain MLOps infrastructure using Azure Machine Learning, MLflow, or similar platforms

• Implement model deployment, versioning, and monitoring solutions

• Develop automated ML pipeline orchestration and model lifecycle management

• Ensure reproducibility and scalability of machine learning workloads

Security & Compliance:

• Implement security best practices including identity and access management (IAM)

• Configure network security, encryption, and data protection measures

• Ensure compliance with regulatory requirements (GDPR, HIPAA, SOX, etc.)

• Conduct security assessments and vulnerability management

Cost Optimization & Performance:

• Monitor and optimize cloud resource costs through rightsizing and automation

• Implement cost allocation and chargeback mechanisms

• Performance tuning of data processing and analytics workloads

• Capacity planning and resource forecasting

Collaboration & Support:

• Work closely with data engineers, data scientists, and analytics teams

• Provide technical guidance and mentoring to junior team members

• Participate in on-call rotation for platform support and incident response

• Document processes, procedures, and system architecture

Essential Business Experience and Technical Skills:

Required:

• Bachelor's degree in Computer Science, Engineering, or related technical field, or 7+ equivalent professional experience will be considered

• Cloud Platforms: 5+ years of experience with Microsoft Azure, including deep knowledge of Azure services (Azure SQL, Cosmos DB, Azure Data Factory, Synapse, Databricks, Data Lake, Machine Learning, etc.)

• Infrastructure as Code: Proficiency in ARM templates, Bicep, Terraform, or similar tools

• Containerization: Experience with Docker, Azure Container Instances, and Azure Kubernetes Service (AKS)

• DevOps Skills

• CI/CD: Strong experience with Azure DevOps, GitHub Actions, Jenkins, or similar platforms

• Automation of infrastructure, application deployments using Azure DevOps Pipeline / Github actions.

• Programming: Proficiency in Python, PowerShell, Bash, and/or other scripting languages

• Monitoring & Logging: Experience with Azure Monitor, Log Analytics, Application Insights, or similar tools

• Data Engineering: Knowledge of data pipeline orchestration tools (Databricks, Azure Data Factory)

• Soft Skills

• Excellent problem-solving and analytical skills

• Strong communication and collaboration abilities

• Ability to work in fast-paced, agile environments

• Leadership and mentoring capabilities

• Customer Service Orientation

Preferred:

• Certifications

• Microsoft Azure Solutions Architect Expert (AZ-305)

• Microsoft Azure DevOps Engineer Expert (AZ-400)

• Terraform Associate or Professional certification

• Experience with multi-cloud environments (AWS, GCP)

• MLOps

• Experience with MLOps frameworks and tools (MLflow, Azure ML, Kubeflow

• Understanding of machine learning lifecycle and model deployment strategies

• GenAI exp. is preferred

• Knowledge of data mesh and modern data architecture patterns

• Data Technologies: Experience with SQL, NoSQL databases, data warehousing, and big data technologies

• Security & Compliance: Strong understanding of cloud security principles and best practices

• Experience with Azure Active Directory, RBAC, and identity management

• Knowledge of data privacy regulations and compliance requirements

• Experience with security scanning and vulnerability assessment tools

• Experience with streaming data technologies (Apache Kafka, Azure Event Hubs)

• Background in financial services, healthcare, or other regulated industries Experience with agile/scrum methodologies

At MetLife, we're leading the global transformation of an industry we've long defined. United in purpose, diverse in perspective, we're dedicated to making a difference in the lives of our customers.

Equal Employment Opportunity/Disability/Veterans

If you need an accommodation due to a disability, please email us at [email protected]. This information will be held in confidence and used only to determine an appropriate accommodation for the application process.

MetLife maintains a drug-free workplace.