Senior Data Engineer
Job Summary
We are looking for a Senior Data Engineer with strong experience in real-time data streaming using Apache Kafka. The ideal candidate will design and build scalable data pipelines, streaming architectures, and data platforms to support real-time analytics and data-driven applications.
Key Responsibilities
· Design, develop, and maintain real-time data pipelines using Apache Kafka.
· Build scalable stream processing solutions using Kafka Streams, Spark Streaming, or Flink.
· Develop and optimize ETL/ELT pipelines for large-scale data processing.
· Integrate streaming pipelines with data lakes, data warehouses, and analytics platforms.
· Work with cross-functional teams including Data Scientists, Product Managers, and Analysts.
· Ensure data reliability, scalability, and performance optimization.
· Implement data governance, monitoring, and alerting for streaming systems.
· Troubleshoot production issues and optimize streaming architecture.
Required Skills
· 8+ years of experience in Data Engineering.
· Strong hands-on experience with Apache Kafka / Kafka Streaming.
· Experience with Python, Java, or Scala.
· Strong knowledge of real-time streaming architectures.
· Experience with Spark, Flink, or Kafka Streams.
· Strong SQL and data modeling skills.
· Experience working with cloud platforms (AWS, GCP, or Azure).
· Familiarity with data lake technologies (S3, Delta Lake, Snowflake, etc.).
Preferred Skills
· Experience with Databricks or Snowflake.
· Experience with containerization (Docker, Kubernetes).
· Knowledge of CI/CD pipelines and DevOps practices.
· Experience with monitoring tools like Prometheus, Grafana, or Splunk.
Nice to Have
· Experience with event-driven architecture.
· Experience building high-throughput, low-latency data systems.
Recommended Jobs
Executive Sous Chef
Job Details Barbuto, the West Village dining landmark created by Chef Jonathan Waxman, is seeking a Executive Sous Chef to join our leadership team. Known for ingredient-driven, Italian-inspired coo…
Senior Scheduler Data Centers
Senior Scheduler - Data Center - New Paltz, NY Our client is a Commissioning Company that has a national footprint and specializes in MEP critical facilities in the data center industry. They pr…
Director, Head of Balance Sheet & Capital Management
Group Treasury | Head of Balance Sheet & Capital Management | Director | New York City About ING : In the Americas, ING’s Wholesale Banking division offers a broad range of innovative finan…
Pulmonologist
Job Summary Position Title: Pulmonologist Job Summary: We are seeking for an experienced Pulmonologists to assess, diagnose and treat patients with severe or chronic breathing issues. The Pu…
HD Service Manager
Service Manager – Heavy Duty Truck & Trailer Repair Portville, NY Salary + Bonus Medical, Dental & Vision Day ONE Weekly Pay | Free Uniforms FleetPride is hiring an experienced Servi…
Job Offer: Lead Installer and HVAC Technician
Malcarne HVAC is Urgently Hiring Lead Installer and HVAC Technician Lead Installer » 10 years of practice » $35/hr - $45/hr baised on practice HVAC Technician »…
Registered Service Associate
POSITION SUMMARY: Registered Service Associates perform operational support and oversight on behalf of the Complex as well as general management support functions and special projects. The role is…
Tax Manager
Tax Manager ?? Hybrid | 3 Days Onsite / 2 Days Remote ?? Tax Accounting | Insurance & Reinsurance About the Role We are looking for a highly skilled Tax Manager to lead consolidate…
FID, Municipal Securities, Syndicate / Debt Capital Markets - Associate
Morgan Stanley is a leading global financial services firm providing a wide range of investment banking, securities, investment management and wealth management services. The Firm's employees serve c…
Hospitalist Physician
(Finger Lakes Region) $285,000 – $325,000 + Sign-on Bonus + Student Loan Repayment The Opportunity You'll join an established, collegial hospitalist team providing high-quality inpatient ca…