Senior Data Engineer - Healthcare Domain

Job Title: Senior Data Engineer – Healthcare Domain

Location: New York, NY
Job Type: Contract

Job Description:

We are seeking an experienced Senior Data Engineer with a strong background in healthcare data systems and hands-on expertise in Python, Apache Spark, and SQL. This role involves building robust data pipelines, transforming and integrating data from various sources, and supporting scalable data solutions that drive insights and compliance in the healthcare domain.

Key Responsibilities:

• Design, develop, and maintain scalable ETL pipelines and data architectures using Spark, Python, and SQL.
• Integrate and process data from various healthcare sources including EHRs, claims systems, and HL7/FHIR interfaces.
• Build and optimize data models in cloud environments (AWS, Azure, or GCP) and support data lake/lakehouse platforms.
• Work closely with Data Scientists, Analysts, and business teams to ensure data accuracy, consistency, and compliance.
• Implement data validation, quality checks, and transformation logic across large datasets.
• Collaborate with cross-functional teams to ensure data privacy, HIPAA compliance, and audit readiness.
• Troubleshoot data pipeline issues, perform root cause analysis, and ensure high data availability and performance.
• Participate in Agile/Scrum ceremonies and support sprint planning and delivery.

Required Qualifications:

• 8+ years of professional experience in Data Engineering or related roles.
• 5+ years of hands-on experience with Python, Apache Spark (PySpark), and advanced SQL.
• Strong experience working with healthcare data (e.g., EDI 837/835, HL7, FHIR, claims, EMR/EHR).
• Expertise in building data pipelines in cloud environments (AWS Glue, EMR, Redshift, S3, Azure Data Lake, etc.).
• Experience working with large-scale structured and unstructured datasets.
• Solid understanding of data warehousing concepts, data governance, and privacy regulations (HIPAA).
• Proficient in Bash/Shell scripting, version control (Git), and CI/CD tools.
• Familiarity with tools like Airflow, Informatica, or DBT for orchestration and transformation.

Preferred Skills:

• Experience with healthcare interoperability standards (HL7 v2, CDA, FHIR).
• Knowledge of data cataloging and lineage tools.
• Exposure to data quality frameworks and observability platforms.
• Background in working in Agile/Scrum teams and cloud-native architecture.

Soft Skills:

• Excellent communication, collaboration, and stakeholder management skills.
• Strong problem-solving abilities and attention to detail.
• Self-driven, proactive, and able to work in a fast-paced environment.