Data Engineering Manager

Location:

Toronto, Ontario, Canada (Hybrid/Onsite as required)

Employment Type: Full-Time

About The Role

We are seeking a highly skilled Data Engineering Manager to lead the design, development, and optimization of our Databricks and AWS-based data ecosystem. In this role, you will oversee a team of data engineers, drive engineering best practices, and ensure the performance, reliability, and scalability of the organization's data platforms. This position is ideal for a hands-on technical leader with deep expertise in Databricks, Spark, and modern cloud data frameworks.

What You'll Do

• Lead and mentor a team of data engineers in architecting and maintaining the Databricks Lakehouse Platform, including Delta Lake versioning and large-scale data processing.
• Manage and optimize Databricks infrastructure, including cluster lifecycle management, cost control, and integration with AWS services (S3, Glue, Lambda, and more).
• Architect and implement scalable ETL/ELT pipelines using Spark (Python/Scala), including streaming workloads when necessary.
• Drive technical excellence through advanced Spark performance tuning, cluster optimization, and I/O efficiency improvements.
• Implement strong data security and governance using Unity Catalog, ensuring alignment with industry and internal compliance standards.
• Champion Agile development practices, conduct code reviews, and serve as a technical advisor across cross-functional teams.
• Build and maintain monitoring and alerting solutions for data pipeline reliability, including SLAs, KPIs, and operational health checks.
• Develop and manage end-to-end CI/CD workflows using Git, automated testing, and continuous deployment pipelines.

What You Bring

• Bachelor's degree in Computer Science, Engineering, or a related field.
• 5+ years of hands-on experience with Databricks and Apache Spark, including building production-grade data pipelines.
• Proven experience leading and mentoring data engineering teams in a fast-paced environment.
• Deep expertise with AWS services: S3, EC2, Glue, EMR, Lambda, Step Functions.
• Strong programming proficiency in Python (PySpark) or Scala, with advanced SQL and data modeling skills.
• Experience with Infrastructure-as-Code tools such as Terraform or AWS CloudFormation.
• Strong understanding of data warehousing concepts, dimensional modeling, and RDBMS platforms (e.g., Postgres, Redshift).
• Proficiency with Git and CI/CD tools, including automated testing and deployment workflows.
• Excellent communication and stakeholder engagement capabilities.
• Demonstrated experience leveraging AI tools or techniques throughout the engineering lifecycle.
• Knowledge of the financial industry is preferred.

Jeanhel Acuba

Senior Technical Recruiter

PRI Technology

P:973.528.1947

[email protected]