Senior Data Engineer (Knowledge Systems)

This organisation is building a modern, cloud-native platform and evolving towards an AI-first engineering model, where intelligent systems rely on high-quality, well-structured knowledge to power automation, decision-making and software delivery.

They are looking for a Senior Data Engineer (Knowledge Systems) to design and own the knowledge platform that supports both AI applications and engineering teams. The role focuses on creating reliable, scalable knowledge systems that enable accurate retrieval and contextual understanding for production AI systems.

This is a senior engineering position where technical judgement is highly valued. Rather than building every component from scratch, the successful candidate will evaluate, integrate and extend best-in-class technologies to create a robust knowledge platform.

What you'll do

• Design, build and own the organisation's knowledge platform for AI applications and engineering teams.
• Develop scalable knowledge architectures that combine structured and unstructured information from multiple sources.
• Choose the most appropriate storage and retrieval approach for different use cases, including document stores, graph databases, vector databases and traditional data platforms.
• Model complex business domains to improve retrieval quality and contextual understanding.
• Design and optimise hybrid retrieval strategies using semantic search, keyword search, embeddings and knowledge graphs.
• Evaluate, integrate and extend best-of-breed tooling rather than building infrastructure unnecessarily.
• Measure, monitor and continuously improve retrieval quality, accuracy and relevance.
• Build and maintain data pipelines that keep knowledge systems accurate, current and production-ready.

Technology

The successful candidate will work with modern cloud platforms (AWS or similar), vector databases, knowledge graphs, retrieval systems, embeddings, RAG architectures, data pipelines, APIs and contemporary AI tooling.

What they're looking for

• Strong commercial experience in senior data engineering, knowledge engineering or a closely related field.
• Experience designing scalable knowledge systems for AI-powered applications.
• Strong understanding of domain modelling, ontology design and information architecture.
• Hands-on experience with hybrid retrieval techniques, semantic search, embeddings and vector databases.
• Experience building reliable data pipelines and integrating multiple data sources.
• Excellent judgement in selecting the right technologies and architectures rather than defaulting to a single solution.
• Strong software engineering and cloud platform fundamentals.

Nice to have

• Experience with production-scale RAG architectures and retrieval evaluation.
• Knowledge of emerging open knowledge standards and interoperable knowledge formats.
• Experience with context engineering for AI agents and LLM applications.
• Experience designing knowledge graphs and semantic data models.
• Familiarity with modern orchestration, workflow and data engineering platforms.
• Experience supporting AI, machine learning or autonomous systems with production data and knowledge infrastructure.

Oscar Associates (UK) Limited is acting as an Employment Agency in relation to this vacancy.

To understand more about what we do with your data please review our privacy policy in the privacy section of the Oscar website.