Lead Data Engineer

We’re thrilled to be supporting a well-known organisation in Liverpool that specialises in insurance services. As they continue to grow their technology function in line with meeting their increasing demands, they look to hire a Lead Data Engineer. Read on for more information.

Lead Data Engineer

Liverpool

£80,000

Lead Data Engineer duties:

• Lead the design development and optimisation of scalable data pipelines in a cloud-based environment.
• Implement and manage data warehouses on Azure Synapse, ensuring data integrity and security.
• Build, deploy, and manage ETL processes for both real-time and batch data processing using Azure tools, Databricks, PySpark, and SQL.
• Oversee data storage management across relational and non-relational databases to ensure efficient data retrieval.
• Design and implement robust data security protocols to protect sensitive information.
• Collaborate with DBA’s, cloud architects, and end users to ensure smooth integration between data platforms and business requirements.
• Manage the ingestion, transformation, and processing of large datasets using big data technologies.
• Optimise the performance of data pipelines and infrastructure with cloud services like Azure and AWS S3.
• Lead a team of data engineers, providing technical guidance and promoting a culture of continuous learning and improvement.

As Lead Data Engineer you will:

• Substantial experience within Data Engineering
• Expertise in Azure Data Warehouse (DWH) and (AWS Databricks).
• Strong programming skills in Python/PySpark or other relevant languages for data manipulation and ETL Processes.
• Proficient in SQL with experience in both relational databases and non relational databases.
• Experience working with AWS S3 and other AWS services for big data solutions.
• Excellent communication skills, with the ability to document processes and present technical concepts effectively.
• Relevant Azure or AWS cloud certifications are a plus.

In return you will receive:

• Competitive market pay.
• Flexible/Hybrid working.
• Amazing career progression.
• Sick pay, maternity/paternity paid leave.
• A wide range of discounted vouchers with selected retailers.

Robert Walters Operations Limited is an employment business and employment agency and welcomes applications from all candidates