Big Data Developer
Salary £50,000 - £57,000 per annum + 10% Bonus
Consultant Mark Pugh
Date posted 30 August 2018 2018-08-302018-10-29 it UK Robert Walters United Kingdom
A Global Financial Services Organisation currently has an exciting opportunity for an experienced Big Data Developer to join their established but rapidly expanding Technology Practice.
Overall purpose of role
- Be a member of a world class big data engineering team responsible for optimising business performance, sharing best practises, developing and connecting engineering teams
- Development of strategies and policies for specific complex business problems and use cases to improve business performance and delivery of business intelligence and insight that effectively communicates performance to stakeholders, as well as meeting internal & external Control and Governance standards
- Leverage existing and new analytical techniques to provide new insights into business performance issues and recommend changes to improve service/profitability.
- Building distributed and highly parallelized Big data processing pipeline which process massive amount of data (both structured and some unstructured data) in batch and streaming
- Taking part in migration of data from upstream sources into Hadoop, one-off and Change Data Capture
- Leverage Spark to enrich and transform corporate data to enable searching, data visualization, and advanced analytics
- Working closely with analysts and business stakeholders to develop analytics models
- Continuous delivery on Hadoop and other Big Data Platforms
- Automate processes where possible and are repeatable and reliable
- Work closely with release engineers and Product Management to deliver software in a continuous delivery environment
Essential Skills/Basic Qualifications
- Experience in Hadoop/Spark development
- Hands on knowledge of some of the following: Kafka, Flume, Hive, Spark, Impala, Sqoop, Oozie, Hbase, YARN, Zookeeper, Java, Scala, Pythonand HUE
- SQL knowledge and experience with a relational database and Hadoop Security: AD, Kerberos, Sentry
- Experience with Spark Streaming and Kafka in large scale enterprise environment is highly desired
Desirable skills/Preferred Qualifications
- Pre and post installation of Hadoop software.
- Familiar with Agile methodologies
- Experience of implementing data security capabilities such as tokenization, encryption, anonymisation etc.
**Should the above be of interest, please apply immediately to receive a more detailed job specification**