Lead Data Quality Engineer

Job Description

Job Description

Overview

The Lead Data Quality Engineer plays a pivotal role in ensuring the integrity, accuracy, and reliability of data across the organization. As the primary authority on data quality, this position entails developing and implementing robust data quality frameworks, standards, and metrics that align with the organizational data strategy. The Lead Data Quality Engineer will collaborate closely with data engineering, data science, and business teams to understand data needs and identify quality issues early in the data lifecycle. This role carries the responsibility of driving initiatives that lead to improved data governance and compliance while reducing data-related risks. Through mentoring and leadership, the engineer will cultivate a culture of quality among teams, ensuring that data quality remains a priority at every phase of data handling. With solid experience and expertise, the Lead Data Quality Engineer will be instrumental in enhancing trust in data and supporting data-driven decision-making processes within the organization.

Key Responsibilities:

• Design and implement data quality (DQ) checks and rules to ensure data accuracy and consistency.
• Conduct data profiling to analyze data sources, identify anomalies, and understand historical trends.
• Monitor and validate data quality results, track trends, and identify significant changes or fluctuations in key performance indicators (KPIs).
• Define and maintain thresholds for detecting and flagging anomalies in KPIs.
• Collaborate with stakeholders to:
• Translate both explicit and implicit requirements into actionable data quality measures.
• Recommend process improvements to enhance data accuracy and governance.

Required Skills Expertise:

• Proficiency in SQL for data analysis and performing quality checks.
• Strong programming skills in Python for data manipulation and process automation.
• Practical experience with Spark and Snowflake for data processing and storage.
• Strong analytical abilities to identify data issues and propose solutions.

Preferred Skills (Nice-to-Have):

• Familiarity with Airflow for workflow orchestration.
• Experience with Looker or other BI tools for data visualization.

Qualifications:

• Proven experience in data quality engineering, data analysis, or related fields.
• Strong communication skills to effectively collaborate with business leaders and technical teams.
• Ability to work independently and in a cross-functional team environment.