Senior data engineer
Title: Data Engineer
Location: New York,Dallas,Chicago (Hybrid)
We are seeking an experienced Data Engineer to design and maintain scalable data pipelines in Databricks. The role involves end-to-end ownership of data workflows, from ingestion to production, ensuring performance, quality, and reliability.
Responsibilities
- Build and optimize pipelines in Databricks (PySpark, SQL, Delta Lake).
- Ingest data from databases, APIs, PDFs, Excel, flat files, and web scraping.
- Implement data quality checks, validation, and testing frameworks.
- Deploy, monitor, and optimize pipelines in Databricks/AWS.
- Collaborate with data science/analytics teams and document data processes.
Qualifications
- 5+ years of Data Engineering experience.
- Hands-on expertise in Databricks (PySpark, SQL, Delta Lake).
- Strong experience with structured & unstructured data.
- Knowledge of data quality/testing tools (Great Expectations, Deequ, dbt tests).
- Proven ability to optimize large-scale Spark/Databricks workloads.
Email : [email protected]
Job Types: Full-time, Contract
Work Location: In person
Recommended Jobs
Program Assistant
Job Description Job Description MISSION STATEMENT Are you ready to give back to the community while pursuing your passion? For over 50 years, Acacia Network and its affiliates have been commit…
Billing Specialist | New York
Our client is a distinguished national legal and professional services firm with eleven offices across the United States, boasting over 450 legal experts and 350 business professionals. This unique, …
Kitchen Help
Job Description Job Description Looking for experienced line cook that can handle busy work environment. Duties include, Prepping for dinner, preparing appetizers, salads, and assisting the head …
Medical Director Internist
Medical Director Internist - Brooklyn, NY (#1650) Medical, Dental, and Vision insurance Flexible Spending Account Paid Time Off Retirement Savings Commuter Benefits program …
Freelance Software Developer (Java) - Quality Assurance (AI Tutor)
This opportunity is only for candidates currently residing in the specified country. Your location may affect eligibility and rates. The Mindrift platform connects specialists with AI projects from …
Phlebotomist
Job Description Job Description Pride Health is hiring a Phlebotomist to support our client’s medical facility based in East Syracuse, NY 13057, USA This is a 3-month assignment with the possib…
Retail merchandiser
Retail Coverage Merchandiser - Walmart General Information Company: ACO-US Location: MONROE, New York, 10950 Ref #: 115351 Function: Merchandising Employment Duration…
Solution Architect, Utilities, Google Cloud Platform
Are you ready to make an impact? West Monroe is looking for a GCP Solutions Architect with proven experience designing and implementing cloud-based data and analytics platforms. This role will fo…
Senior Manager, Account Planning
Chanel seeks a Senior Manager for Account Planning in its Fragrances & Beauty division in New York. The role involves leading sales and inventory strategies, collaborating with cross-functional teams,…
Customer Service Representative
Do you desire to make a difference in your team, your clients, and your patients' lives while feeling supported? Culture is king in our practices! What does that mean? It means we hire, reward, a…