To Apply for this Job Click Here
Location: Hybrid Wayne, PA (2 days on site/ week required)
We are seeking a Data Engineer with strong hands-on experience in Azure Databricks and modern data pipeline development. This role will focus on building and optimizing data pipelines, developing scalable data solutions, and working with structured and semi-structured data in cloud environments.
Key Responsibilities
- Design, develop, and maintain data pipelines using Azure Databricks and related Azure services
- Build and optimize ETL/ELT workflows to support data ingestion, transformation, and processing
- Work with large datasets using PySpark and Databricks notebooks to deliver efficient data solutions
- Implement data transformations, validations, and performance optimizations across pipelines
- Collaborate with data architects, analysts, and business stakeholders to understand data requirements
- Assist in troubleshooting data pipeline issues and improving system reliability and performance
- Support integration with Azure Data Factory for orchestration and pipeline workflows
- Ensure data quality, consistency, and accuracy across datasets and downstream systems
- Participate in code reviews and contribute to best practices for data engineering standards
Required Qualifications
- 3+ years of data engineering experience
- Hands-on experience with Azure Databricks
- Strong SQL skills for querying, transformation, and data validation
- Experience with Python and PySpark for data processing
- Familiarity with Azure Data Factory for pipeline orchestration
- Experience working with data pipelines beyond simple ingestion (transformations, optimization, scaling)
- Ability to pass a hands-on coding assessment in Databricks / PySpark / SQL
T1459142-PHI_1779311295