We have an exciting opportunity for a Data Engineer with good experience in PySpark, Airflow and proficiency in Unix. Docker and Kubernetes experience are also required. The Data Engineer will troubleshoot data pipelines and address issues with real-time, batch processing. Plus, develop an ETL process and data flows using PySpark and Airflow. Remote and a few days in the office initially
Skills / Experience : • PySpark & Airlow • Unix • Docker & Kubernetes CI / CD Tools: GitHub Actions • Maintenance & troubleshooting • Understanding & ensuring data quality, integrity, governance
On the other hand, the Data Engineer will troubleshoot data pipelines and address issues with real-time and batch processing. Plus, develop ETL process and data flows using PySpark and Airflow. Remote and a few days in the office initially