To Apply for this Job Click Here
NOTE: THIS IS A 100% Remote role.
Senior Data Engineer
Must Haves
- Strong hands-on experience building and supporting production-grade data pipelines using PySpark and Apache Spark
- Extensive experience working within Databricks environments
- Experience with Databricks Unity Catalog migrations and modernization initiatives
- Experience with Databricks Unity Catalog governance and security models
- Strong understanding of ETL/ELT design, data modeling, and incremental data processing
- Experience modernizing legacy data platforms and data migration projects
- Experience working with large-scale datasets in cloud-based data platforms
- Experience with workflow orchestration, scheduling, and pipeline automation
- Knowledge of cloud data platforms such as Snowflake and/or Google Cloud
- Experience implementing data quality checks, monitoring, logging, and alerting
- Strong troubleshooting, collaboration, and problem-solving skills
Responsibilities
- Migrate existing data assets and pipelines to Databricks Unity Catalog
- Refactor and optimize existing pipelines using modern engineering standards and best practices
- Build and maintain scalable PySpark-based data pipelines
- Develop new source-of-truth datasets and modernize legacy data workflows
- Design and implement workflow orchestration using Databricks Jobs
- Deploy and manage pipelines using Databricks Asset Bundles
- Ensure data pipelines are reliable, observable, and production-ready
- Implement data quality controls, monitoring, and operational alerting
- Support platform migrations, testing, validation, and production cutovers
- Partner with engineering teams to improve platform stability, scalability, and data reliability
Nice to Have
- Familiarity with advertising, analytics, measurement, or large-scale data ecosystems
- Experience with CI/CD and DataOps best practices for data engineering environments
NS-DAD-NS_1780937688