Accessibility Links

Big Data Engineer - Spark, SQL, Hive, HDFS, Python, Oozie

  • Job reference: DO32674
  • Location: Brighton
  • Job type: Contract
  • Start date: Not specified
  • Contact: Not specified
  • Sector: BI, Data & Analytics
  • Salary: £300 - £350 per day

Sorry, this vacancy has now expired.

Please see our job results page to find our current open vacancies or take a look at our Specialisms below and go straight to your area.

BI, Data & Analytics, Change Management, Digital and Development, ERP and CRM Systems, Executive Leadership, Information and Cyber Security, Infrastructure & Cloud Services, Interim Management, Strategy & Architecture, or Work For La Fosse.




 Return to homepage.

Big Data Engineer - Spark, SQL, Hive, HDFS, Python, Oozie

We are looking for some incremental Big Data Engineers to support the CS work we are doing for Regulatory Reporting - Issues Management and KYCC.


The Engineer/s will be responsible for the designing and building of scalable data pipelines and processing applications on BIG Data Hadoop platform. This role will be involved in all phases of the software development life cycle including coding, testing and operational maintenance activities. The candidate must be passionate about data, innovative software development, cloud solutions and agile methodologies. They must be eager to advance their and the teams knowledge, and have a proven ability to deliver quality products.


  • Ownership and development of data pipelines including data ingestion, transformation and presentation following Agile methodology.
  • Development of the data platform to enable scalable pipelines including support and monitoring.
  • Hands on technical role contributing to all phases of the software development lifecycle including analysis, design, implementation, unit testing and QA.
  • Performance enhancements to code to improve speed and scalability.
  • Build internal knowledge base by documenting best practises, standards and governance.

Essential Knowledge and Skills:

The Engineer/s must be able to demonstrate advanced skills and knowledge at the depth and scope required for the position.

  • Bachelor's Degree in Computer Science, Engineering, or related discipline
  • Good experience in Hadoop technologies

Must haves

  • Experience in following software packages: - All of below skills must have
  • Spark, Spark SQL
  • Hive
  • Oozie or Similar
  • HDFS
  • Python
  • Advanced SQL

Basic knowledge must have

  • Unix shell scripting
  • Knowledge and experience of using Cloud based technology
  • Experience with Test Driven Development
  • Knowledge and understanding of Java programming language
  • Experience with source code control tools such as Git to manage production and development code
  • Agile Experience using Scrum or Kanban
  • Understanding of reporting tools such as Tableau
  • Excellent analytical and problem solving skills
  • Good prioritisation skills and be flexible enough to adapt
  • A focus on end goals and good team-playing mentality
  • An ability to work to tight deadlines and within constraints
  • Familiarity with the Atlassian and Rally suite
  • Ability to interact comfortably with the other members of a large, geographically dispersed team

Desirable Knowledge and Skills

  • Apache Flume
  • Experience with enterprise relational database systems like Oracle, DB2
  • Understanding of IBM Mainframe
  • Financial AML knowledge
  • Knowledge of secure software coding practices
  • Working experience of a defect tracking syste

If interested, please contact me directly or apply via the link!

Related jobs
Enterprise Data Solutions Architect - Data Warehouse
  • Contract
  • London
  • £0 per day + Rate negotiable depending on experience
  • Reference 34526
  • Enterprise Data Solutions Architect - Finance Data Warehouse - Design, Data Warehouse, IFRS 17, Actuarial. Enterprise Data Solutions Architect required for an initial 6-month contract for a financial services client in London. The role will join a large-scale finance transformation programme and lead the design of a...
Read more
Technical Data Architect - SQL , SSIS , Azure
  • Contract
  • Oxfordshire
  • £500 - £570 per day
  • Reference LFA - 34064
  • Technical Data Architect - SQL , SSIS , Azure Technical Data Architect with experience designing and developing DWH solutions on Azure is required for a client based in Oxford. Must have SME knowledge of ETL processes, SSIS, DWH architecture and Azure. Technology overview: SQL ServerSSISETL / DWH architectureAzure Data...
Read more
Tableau Developer
  • Contract
  • London
  • £450 - £500 per annum
  • Reference 34166
  • Tableau Developer with experience creating reports and dashboards in Tableau is required for a London based client. You will be required to convert requirements into visual dashboards using data from core transaction systems. Experience: * Tableau Server 9.3+ * SQL Server 2012 + * Experience creating dashboards *...
Read more