Summary
Design and develop scalable data engineering solutions and distributed data processing pipelines to deliver near real-time insights for business stakeholders. Collaborate with product managers, analysts, data scientists and engineering teams while supporting production operations and driving engineering best practices.
Responsibilities
- Design and develop scalable data analytical solutions using cloud and modern technologies.
- Build and maintain distributed data processing pipelines in Big Data ecosystems.
- Implement data modeling, SQL solutions and optimize data performance.
- Participate in code reviews and uphold software engineering principles.
- Develop testing tools and CI CD pipelines to automate deployments.
- Provide on-going production support and participate in on-call rotations.
- Collaborate with stakeholders to deliver self-service BI and analytics solutions.
- Monitor cost savings, alerts, logging and observability dashboards.
Requirements
- 3+ years experience in a modern programming language such as Java or Python.
- 3+ years experience with databases, SQL, data modeling and automated engineering solutions.
- Experience with cloud platforms such as AWS or GCP.
- Experience with Kafka, Spark, Kubernetes and Airflow for near real-time processing.
- Familiarity with BigQuery and Presto is a plus.
- Bachelor's or Master's degree in Computer Science, Engineering or equivalent experience.
- Experience designing scalable distributed systems and event driven architectures.
We have summarized this job description for you, click apply to see more details from the employer.
