Data Engineer

Key Responsibilities

• Design, build, test and maintain scalable data pipelines and data products.
• Develop and optimise ETL/ELT processes for integrating data from multiple sources.
• Design and implement cloud-native data solutions, preferably within Google Cloud Platform (GCP).
• Support migration of data platforms, pipelines and services from on-premises environments to the cloud.
• Develop and maintain data models, data warehouses and data integration solutions.
• Implement CI/CD processes and automated deployment pipelines.
• Use workflow orchestration tools such as Airflow, Prefect, Dagster or Cloud Composer.
• Monitor, troubleshoot and optimise platform and pipeline performance.
• Collaborate with technical and non-technical stakeholders to translate business requirements into robust technical solutions.
• Contribute to code reviews, testing, technical documentation and software engineering best practices.
• Ensure compliance with data governance, security and regulatory requirements, including GDPR.

Essential Skills & Experience

• Strong programming skills in Python and SQL (R experience desirable).
• Experience with relational databases and data warehousing technologies.
• Proven experience building and supporting production-grade ETL/ELT pipelines.
• Strong understanding of data modelling principles, including conceptual, logical and physical models.
• Experience with Git-based version control systems such as GitHub, GitLab or Azure DevOps.
• Experience working in Agile delivery environments.
• Experience with cloud platforms, preferably Google Cloud Platform (GCP).
• Experience with workflow orchestration tools including Airflow, Prefect, Dagster or Cloud Composer.
• Knowledge of containerisation technologies such as Docker.
• Experience working with large-scale datasets and distributed processing technologies such as Spark, Dataproc and BigQuery.
• Understanding of DevOps and DataOps principles.
• Experience integrating systems through APIs and data services.
• Strong troubleshooting and performance optimisation skills.

Desirable Skills

• Experience with metadata management, data catalogues and lineage tools.
• Experience implementing data quality, profiling and validation frameworks.
• GCP Data Engineer and/or GCP Architect certification.
• Experience within highly regulated or data-driven environments.

Qualifications

• Degree in Computer Science, Data Engineering, Software Engineering or a related technical discipline, or equivalent commercial experience.