Principal Data Engineer - Contract

Principal Data Engineer (Contract) Hybrid (London-based) | £450–£650/day A leading scientific and cultural institution is seeking a Principal Data Engineer to design and scale cloud-native data infrastructure powering biodiversity research, digitisation, and AI workflows.

Key Responsibilities

• Architect and build scalable data pipelines for ingestion, transformation, enrichment, and publication.

• Integrate AI/ML workflows, including training data pipelines and inference outputs.

• Lead AWS-based data engineering using services like S3, Lambda, Glue, and Athena.

• Ensure interoperability with global research infrastructures (e.g. DiSSCo UK, RECODE, DToL).

• Define best practices in data quality, metadata, and reproducibility.

• Collaborate with cross-functional teams (science, software, product).

• Mentor engineers and contribute to institutional data strategy and standards.

Tech Requirements

• Strong Python skills; experience with Airflow, Luigi, or Spark.

• Proven track record building robust, scalable ETL/ELT systems.

• Experience with scientific, semi-structured, or biodiversity data.

• Familiarity with semantic web standards (e.g. Darwin Core, JSON-LD, Linked Data).

• Knowledge of NoSQL/graph databases (e.g. MongoDB, Neo4j, Neptune) a plus.

• Hands-on with cloud-native architectures, especially AWS.

• Understanding of data versioning, provenance, and governance.

Robert Walters Operations Limited is an employment business and employment agency and welcomes applications from all candidates