Banner Default Image
Back to job search

Data Engineer

We have a new role to join our BI and Data Engineering Team within our IT Department in Durham. Working as part of a growing Team, you will be tasked with overseeing the rebuild of our SQL Server data warehouse to AWS cloud.

This is a very exciting role in a business that is transitioning from a traditional on-premise data warehouse to AWS cloud. You would be responsible for the development of the end to end solution in AWS, ETL processes, data lake and performance administration, and dimensional design of the table structure.

The Data Engineer works closely with the Team Lead, mentors the Data Engineers and works with our BI Developers to power insight and avail meaningful data products for the business and enable consistently informed management decisions.

JOB

  • Design, build and operationalise large scale enterprise data solutions and applications using one or more of AWS data and analytics services.

  • Analyse, re-architect and re-platform on-premise data warehouses to data platforms on AWS cloud using AWS or 3rd party services.

  • Design and build production data pipelines from ingestion to consumption within a big data architecture, using Java, Python, Scala.

  • Design and implement data engineering, ingestion and curation functions on AWS cloud using AWS native or custom programming.

  • Work with the Delivery Lead to develop relationships with various Functions by producing analytics and reports.

  • Understand existing on-premise data warehouse and our ETL process and create an appropriate transition path to AWS cloud.

  • Work alongside the Delivery Lead, and coordinate with all other Departments and be involved in requirement gathering meetings, where appropriate.

  • Be responsible over the storage of data from a variety of sources, including dimensional, physical, and logical models.

  • Designing efficient data collection techniques specifically for the Company.

  • Contribute to the development of overall processes within the BI and Data Engineering Team.

  • Proactively seek opportunities from across the Organisation to improve the BI and Data Engineering Team.

SKILLS/EXPERIENCE

  • Has experience in any of the following AWS Athena and Glue Pyspark, EMR, DynamoDB, Redshift, Kinesis, Lambda, Snowflake

  • Proficient in AWS Redshift, S3, Glue, Athena, DynamoDB

  • Brilliant Communication skills.

  • Excellent team player.

  • Can follow direction.

  • Understand issues relating to data security and privacy.