We Work Remotely
home_work
Remote
Azure, Databricks & ML Pipelines | Remote
Senior Data Engineer
location_on
Remote / Worldwide
calendar_today
6 hours ago
bolt
Apply & Run AI Match
We’ll save this job to your dashboard and score how well your resume matches.
description Job Description
About the Role
We're looking for a Senior Data Engineer to design, build, and maintain scalable data pipelines and ML-ready infrastructure on Azure and Databricks. This is a hands-on engineering role: you'll own the full data pipeline lifecycle — ingestion, transformation, orchestration, and deployment — while supporting machine learning workflows with clean, reliable data. If you're comfortable owning infrastructure decisions and writing production-quality Python at scale, this role is built for that.
What You'll Do
•
Design, build, and maintain data pipelines using Databricks and Azure-native data services
•
Develop and optimize ETL/ELT processes to support analytics and machine learning workloads
•
Build and maintain CI/CD pipelines for data engineering and ML deployment workflows
•
Write clean, efficient, production-quality Python for data processing and pipeline automation
•
Support machine learning teams with well-structured, high-quality datasets and feature pipelines
•
Design and manage data architecture across Azure services (e.g., Azure Data Factory, Azure Data Lake, Azure Synapse)
•
Monitor pipeline performance, troubleshoot data quality issues, and implement reliability improvements
•
Implement data governance, security, and access control best practices
•
Collaborate with data scientists, analysts, and software engineers to align data infrastructure with business needs
•
Participate in code reviews, architecture discussions, and technical planning
What You Bring
•
Strong hands-on experience with Azure cloud data services
•
Proven experience building and maintaining pipelines on Databricks
•
Solid experience designing and managing CI/CD pipelines for data or ML workflows
•
Strong Python skills for data engineering and pipeline development
•
Working knowledge of machine learning workflows and how data engineering supports them
•
Experience with SQL and relational/distributed data systems
•
Understanding of data pipeline orchestration, monitoring, and reliability practices
•
Strong problem-solving skills and ability to work independently on complex data infrastructure challenges
•
Solid communication skills for collaborating with data science and engineering teams
Nice to Have
•
Experience with MLOps practices and tools (MLflow, Azure ML)
•
Familiarity with Spark internals and performance tuning within Databricks
•
Experience with infrastructure-as-code (Terraform, Bicep, ARM templates)
•
Exposure to real-time/streaming data pipelines (Kafka, Event Hubs, Structured Streaming)
•
Relevant Azure or Databricks certifications
Why This Role
•
Full pipeline ownership: Own data infrastructure end to end, from ingestion through ML-ready delivery
•
Modern data stack: Work with Azure and Databricks, leading platforms in enterprise data engineering
•
Cross-functional impact: Directly enable machine learning and analytics outcomes, not just move data
•
Flexibility: Remote-friendly engagement structure
How to Apply
Ready to bring your data engineering expertise to Azure and Databricks-powered ML infrastructure? Apply through Toptal here:
To apply:
We're looking for a Senior Data Engineer to design, build, and maintain scalable data pipelines and ML-ready infrastructure on Azure and Databricks. This is a hands-on engineering role: you'll own the full data pipeline lifecycle — ingestion, transformation, orchestration, and deployment — while supporting machine learning workflows with clean, reliable data. If you're comfortable owning infrastructure decisions and writing production-quality Python at scale, this role is built for that.
What You'll Do
•
Design, build, and maintain data pipelines using Databricks and Azure-native data services
•
Develop and optimize ETL/ELT processes to support analytics and machine learning workloads
•
Build and maintain CI/CD pipelines for data engineering and ML deployment workflows
•
Write clean, efficient, production-quality Python for data processing and pipeline automation
•
Support machine learning teams with well-structured, high-quality datasets and feature pipelines
•
Design and manage data architecture across Azure services (e.g., Azure Data Factory, Azure Data Lake, Azure Synapse)
•
Monitor pipeline performance, troubleshoot data quality issues, and implement reliability improvements
•
Implement data governance, security, and access control best practices
•
Collaborate with data scientists, analysts, and software engineers to align data infrastructure with business needs
•
Participate in code reviews, architecture discussions, and technical planning
What You Bring
•
Strong hands-on experience with Azure cloud data services
•
Proven experience building and maintaining pipelines on Databricks
•
Solid experience designing and managing CI/CD pipelines for data or ML workflows
•
Strong Python skills for data engineering and pipeline development
•
Working knowledge of machine learning workflows and how data engineering supports them
•
Experience with SQL and relational/distributed data systems
•
Understanding of data pipeline orchestration, monitoring, and reliability practices
•
Strong problem-solving skills and ability to work independently on complex data infrastructure challenges
•
Solid communication skills for collaborating with data science and engineering teams
Nice to Have
•
Experience with MLOps practices and tools (MLflow, Azure ML)
•
Familiarity with Spark internals and performance tuning within Databricks
•
Experience with infrastructure-as-code (Terraform, Bicep, ARM templates)
•
Exposure to real-time/streaming data pipelines (Kafka, Event Hubs, Structured Streaming)
•
Relevant Azure or Databricks certifications
Why This Role
•
Full pipeline ownership: Own data infrastructure end to end, from ingestion through ML-ready delivery
•
Modern data stack: Work with Azure and Databricks, leading platforms in enterprise data engineering
•
Cross-functional impact: Directly enable machine learning and analytics outcomes, not just move data
•
Flexibility: Remote-friendly engagement structure
How to Apply
Ready to bring your data engineering expertise to Azure and Databricks-powered ML infrastructure? Apply through Toptal here:
To apply:
Ready to apply?
Create a free account to apply with an AI-tailored resume.