To Apply for this Job Click Here
Title: Senior Data Engineer
Location: REMOTE
Type: Contract to Hire
Overview:
We are seeking a Senior Data Engineer. This role is responsible for designing and optimizing scalable data pipeline architectures, enabling data flow across the organization. The ideal candidate will collaborate with cross-functional teams including developers, database architects, analysts, and data scientists to ensure reliable data delivery systems. This position also involves mentoring junior engineers and contributing to the development of data-driven tools and analytics platforms.
Key Responsibilities
Data Engineering & Development
- Design, assemble, and maintain large, complex data sets that meet functional and non-functional business requirements.
- Build and optimize data pipelines using AWS services (Python, PySpark, EC2, S3, Glue, Lambda, and Delta Lake).
- Develop IaC using Terraform and contribute to CI/CD pipelines for automated deployment.
- Implement cloud-based data solutions and microservices architectures.
- Create analytics tools and dashboards that generate actionable business insights.
- Write and maintain clean, efficient, testable, and well-documented code.
- Collaborate on design and code reviews.
Production Support
- Perform regular system maintenance and upgrades.
- Troubleshoot production issues, including those involving third-party applications.
- Anticipate data and infrastructure challenges, proposing and implementing solutions.
Team Collaboration & Leadership
- Collaborate with stakeholders to understand data needs and translate them.
- Participate in Agile ceremonies and cross-functional meetings.
- Mentor junior team members.
Requirements
- Strong experience with Python and PySpark for data processing and automation.
- Proficient in Terraform and Git-based version control workflows.
- Hands-on experience building & maintaining CI/CD pipelines.
- Familiarity with message queuing, stream processing, and distributed data architectures.
- Strong understanding of DevOps and infrastructure automation practices.
- Ability to perform root cause analysis on data issues and identify improvement opportunities.
Education & Experience
- Bachelor’s degree in Computer Science, Information Systems, or a related field (preferred).
- 5+ years of experience in Python and building data pipelines.
- 3+ years working with AWS cloud data services and IaC.
- 2+ years of CI/CD pipeline development.
PBMAD0610_1749611209