Senior Data Engineer (Databricks, Scala, Spark) – Contract

Location: Central London (2–3 days/week onsite)
Type: Contract – 6 months +

Outside IR35 

We’re looking for a Senior Data Engineer with a strong track record in building and maintaining data lakehouses, ideally within banking or capital markets domains to join our global financial services client on a contract basis. 

This role supports business users in the Capital Markets space, so knowledge of derivatives, risk, PnL, trade lifecycle, market data, and scenarios is highly desirable.

Key Responsibilities

• Own and drive components of the data platform through deep domain understanding.
• Design, develop, and maintain scalable, high-performance data pipelines.
• Ensure consistent and accurate integration of internal and external data sources.
• Follow best practices in ETL, data modelling (medallion architecture), and pipeline development.
• Collaborate with Traders and Quants to deliver data solutions aligned with business needs.
• Maintain data quality, troubleshoot issues, and optimize platform performance.
• Produce clear documentation for pipeline design and processes.

Required Skills & Experience

• Expertise around Databricks, including unity catalogue (preferably on Azure)
• Strong programming in Scala and Python
• Deep understanding of Spark architecture and performance tuning
• Proven experience with ETL design, data modelling, and database design
• Strong grasp on the full SDLC and Agile delivery
• Solid knowledge of trading, derivatives, and risk systems (nice to have)