• SKILL LEVEL:
  • Expert

Data engineer

Job profile:

Digital

Career zone:

The transformation driver


What is a data engineer and what do they do?

 

A data engineer is responsible for acquiring, structuring and leveraging data to support growth and innovation. They will need to have experience working with Teradata, Oracle or SQL Server. They will be an ‘extract, transform, load’ (ETL)/data warehousing (DWH) developer with strong knowledge in designing, developing and delivering end-to-end data integration processes for large-scale data warehouses. In this role, they also need to apply advanced knowledge of architecture, design and business processes. As such, certifications in Teradata, Oracle or SQL Server are an advantage.


Key responsibilities
 

Responsibilities will vary, but examples include: 

 

  • supporting the development of key strategic analytics projects and delivering continuous enhancements to the analytics pipeline (segmentation, propensity modelling and analytically driven process change)
  • leveraging exceptional attention to detail to deliver scalable datasets
  • building robust datasets to support insight generation and analytics
  • building and extending the customer analytics record, which is a foundational building block for analytics and predictive modelling
  • identifying and acquiring the relevant data to build out the engineering requirements behind customer journey analytics
  • working collaboratively with the data analysts and data scientists to analyse raw data sources and data transformation requirements
  • designing and developing code, scripts and data pipelines that leverage structured and unstructured data integrated from multiple sources
  • being the analytics engineering representative on big data projects.

Why are they important?

 

Data engineers can build reusable data assets and features for acquiring data, as well as using data to improve the performance of the organisation and the experience that customers receive. This improves profitability and enhances the reputation of a company when done well.


Person specification
 

In addition to the specific technology skills required in the role, a data engineer must be a strong collaborator and relationship builder, and previous consulting experience is an advantage. Strong leadership, communication and interpersonal skills are also essential.

Data engineers must be able to use agile development methodology and have knowledge of Cloud-based data and analytics tools. They will also bring their commercial acumen and understanding of strategy, governance and risk from their ACCA Qualification into the role.

Strong drive is essential. The data engineer has to be an individual who is excited to learn and test new toolsets, and someone who keeps in touch with emerging trends in data and analytics.

Careers insights from AB magazine

Top tips and advice on a wide range of career and workplace issues

Visit AB magazine