Tech Lead Data Retrieval / RAG – Python & Search

Project overview

Want to take on a new challenge in Data and AI? CELAD awaits you! 🚀

Are you passionate about information retrieval, data architectures, and generative AI? Do you want to work on large-scale volume, indexing, and relevance issues?

We are looking for a Tech Lead Data Retrieval / RAG to join a player in the legal publishing and Legaltech sector, based in La Défense.

Within a team specialized in AI services, you will contribute to the design of the data foundation that powers intelligent applications for legal professionals.

🎯 Your Role

You will be the technical point of reference for the collection, ingestion, transformation, and indexing of large documentary corpora. Your objective: to design reliable and scalable pipelines, while optimizing the relevance of results from hybrid search engines and RAG systems.

🚀 Your Main Missions

🛠️ Technical Environment

• Data ingestion: design pipelines capable of collecting and processing heterogeneous sources, ensuring parsing, cleaning, chunking, and normalization of documents.
• Indexing: industrialize indexing and embedding mechanisms on large corpora, ensuring the freshness and consistency of indexes.
• Information retrieval: design search strategies combining lexical search, semantic search, BM25, embeddings, and reranking.
• Relevance optimization: define metrics and test sets to measure and improve retrieval quality.
• RAG architecture: structure the retrieval of documents and context passed to LLMs, managing citations, deduplication, and context windows.
• Quality and traceability: implement dataset and index versioning, data lineage, and supervision mechanisms.
• Security: contribute to data protection, processing isolation, and compliance with regulatory requirements.
• Technical leadership: support teams, promote good development practices, and contribute to architecture choices.
• Languages: Python 3.11+, Go appreciated.
• Search and indexing: OpenSearch, Elasticsearch, pgvector, Weaviate, Qdrant, BM25, hybrid search.
• AI and NLP: embeddings, sentence-transformers, tokenization, document parsing, reranking, RAG, LangChain.
• LLM: OpenAI, Mistral, Bedrock, or equivalent technologies.
• Data engineering: PostgreSQL, Airflow, Spark, or equivalents.
• Infrastructure: Kafka, Redis, Docker, CI/CD, Kubernetes.
• Testing and evaluation: Pytest, precision, recall, nDCG, MRR.

Project details

• Start: flexible
• Remote: hybrid
• Location: 92800 Puteaux, France, FR
• Languages: French