If you get a genuine kick out of developing and delivering complex data pipelines with big data/machine learning technologies as well as leading and defining such projects - this job is for you!
This role is a player/coach opportunity, meaning you will drive the team forward but be hands-on with development skills & also define best practice, standards and techniques for our new capability.
As this practice is brand new you will be instrumental in defining the practice and building of the team.
In this role you will creating cutting edge machine learning and big data pipelines to process a wide variety of data. You will be using deep learning, Spark, Hadoop, Scala, Python, HDFS, geospatial data heavily, so production experience of these would be highly beneficial.
We need you to build an industry leading data engineering team; so you'll be responsible for maintaining collaborations and relationships across government and wider, relating to Data Engineering capabilities.
You will work across the whole office in a matrix management approach under a smart working ethos sharing the working space with different teams on a daily basis. You will have line management responsibilities for senior data engineering staff.
A typical day may involve:
- Guide, develop and build production strength data pipelines and manage delivery through external companies as well.
- Develop and build the Data Engineering practice through collaborative working, coaching and shared projects.
- You will coordinate team's activities and set best practice and standards maintaining and building relationships relevant to data science and engineering across government.
Responsibilities will include:
- Productionising complex data pipelines, such as, implementing systems using cutting edge technologies like deep learning and geospatial big data
- Reviewing a wide variety of data problems, understand what type of problem you are looking at - data management/ETL/data science etc. Then design and build a solution if in the data engineering remit
- Building the capability at the organisation, including recruitment, procurement of additional support, developing existing talent and stakeholder management.
- Lead the data engineering work at UKHO, including best practice and tools required
- Working with business to prioritise work in line with organisational strategy
- Being the technical lead for data engineering
- Working with multi-disciplinary teams to deliver these systems
Proven track record of developing fit for purpose, production quality, resilient, scalable and future-proof data pipelines/systems to meet user needs - including with distributed systems. (higher weighting)
You will have a demonstrable understanding of data engineering tools and techniques, such as Hadoop, Spark, SQL, database types, data formats, data modelling, how to expose data from systems, distributed computing.
You will be able to design, write and iterate code from prototype to production-ready, understand security, accessibility and version control. Can use a range of coding tools and languages; including Spark, Scala and Python.
Experience of building, leading and managing successful technical teams (Collaborating and partnering)
Demonstrated evidence of an elevated level of emotional intelligence and behavioural understanding with the ability to bridge the gap between the business and technology teams (Leading and Communication)