Accessibility Links

Big Data Engineer - Spark, SQL, Hive, HDFS, Python, Oozie

  • Job reference: DO32674
  • Location: Brighton
  • Job type: Contract
  • Start date: Not specified
  • Contact: Not specified
  • Sector: BI, Data & Analytics
  • Salary: £300 - £350 per day

Sorry, this vacancy has now expired.

Please see our job results page to find our current open vacancies or take a look at our Specialisms below and go straight to your area.

BI, Data & Analytics, Change Management, Digital and Development, ERP and CRM Systems, Executive Leadership, Information and Cyber Security, Infrastructure & Cloud Services, Interim Management, Strategy & Architecture, or Work For La Fosse.

 

 

 

 Return to homepage.

Big Data Engineer - Spark, SQL, Hive, HDFS, Python, Oozie

We are looking for some incremental Big Data Engineers to support the CS work we are doing for Regulatory Reporting - Issues Management and KYCC.

Profile

The Engineer/s will be responsible for the designing and building of scalable data pipelines and processing applications on BIG Data Hadoop platform. This role will be involved in all phases of the software development life cycle including coding, testing and operational maintenance activities. The candidate must be passionate about data, innovative software development, cloud solutions and agile methodologies. They must be eager to advance their and the teams knowledge, and have a proven ability to deliver quality products.

Requirements

  • Ownership and development of data pipelines including data ingestion, transformation and presentation following Agile methodology.
  • Development of the data platform to enable scalable pipelines including support and monitoring.
  • Hands on technical role contributing to all phases of the software development lifecycle including analysis, design, implementation, unit testing and QA.
  • Performance enhancements to code to improve speed and scalability.
  • Build internal knowledge base by documenting best practises, standards and governance.

Essential Knowledge and Skills:

The Engineer/s must be able to demonstrate advanced skills and knowledge at the depth and scope required for the position.

  • Bachelor's Degree in Computer Science, Engineering, or related discipline
  • Good experience in Hadoop technologies

Must haves

  • Experience in following software packages: - All of below skills must have
  • Spark, Spark SQL
  • Hive
  • Oozie or Similar
  • HDFS
  • Python
  • Advanced SQL

Basic knowledge must have

  • Unix shell scripting
  • Knowledge and experience of using Cloud based technology
  • Experience with Test Driven Development
  • Knowledge and understanding of Java programming language
  • Experience with source code control tools such as Git to manage production and development code
  • Agile Experience using Scrum or Kanban
  • Understanding of reporting tools such as Tableau
  • Excellent analytical and problem solving skills
  • Good prioritisation skills and be flexible enough to adapt
  • A focus on end goals and good team-playing mentality
  • An ability to work to tight deadlines and within constraints
  • Familiarity with the Atlassian and Rally suite
  • Ability to interact comfortably with the other members of a large, geographically dispersed team

Desirable Knowledge and Skills

  • Apache Flume
  • Experience with enterprise relational database systems like Oracle, DB2
  • Understanding of IBM Mainframe
  • Financial AML knowledge
  • Knowledge of secure software coding practices
  • Working experience of a defect tracking syste

If interested, please contact me directly or apply via the link!

Related jobs
Data Analyst
  • Permanent
  • London
  • £30000 - £35000 per annum
  • Reference 19722
  • A motivated and skilled Data Analyst to improve the handling and leveraging data for one of the fastest growing marketing companies. They are looking for: Proficiency in SQLExperience with visualisation tools such as PowerBI or Tableau Strong skills in Google AnalyticsStrong verbal and interpersonal skillsThis is an...
Read more
Senior Analyst
  • Permanent
  • London
  • £50000 - £60000 per annum
  • Reference 33369
  • A motivated and skilled Senior Analyst to improve the handling and leveraging data for one of the fastest growing marketing companies. They are looking for: Someone fluent in either French or GermanProficiency in SQLExperience with visualisation tools such as PowerBI or Tableau Strong skills in Google AnalyticsStrong...
Read more
Tech Lead/Data Architect
  • Permanent
  • London
  • £65000 - £70000 per annum + Car Allowance, Bonus, Holidays
  • Reference 33531
  • An assertive and skilled Tech Lead/Data Architect to be responsible for the development and implementation of enterprise data architecture and strategy for the UK's most prestigious real estate developer. They are looking for: Experience of Data ModellingSolid background in Data Warehouse design and BI reporting...
Read more
Latest articles