Tech Lead Expert RAG - Freelance

Project overview

Context

Alongside a team of passionate experts, you will tackle the highest technical challenges for a company specializing in publishing and technology services with an international scope. As a Data & Retrieval Expert, you are the guardian of our AI's raw material: the collection, ingestion, indexing, and relevance search on massive volumes of specialized documents, the foundation that powers our language models and products. Your playground: transforming the way an entire professional sector works daily.

Your mission is not to write a simple script, but to design robust data pipelines capable of ingesting, cleaning, and indexing enormous textual corpora, then exposing fast and relevant search (semantic and hybrid) to serve several teams across Europe. You will ensure index freshness and retrieval quality while managing high-level technical complexity (volume, source heterogeneity, relevance evaluation, scalability).

Missions

Your main missions revolve around three axes: building, tooling, and transmitting.

Technical Stack

Working Conditions

• Collection, Ingestion & Indexing
• Design the collection and processing of heterogeneous sources (open data, specialized databases, business documents), parsing, cleaning, chunking, and normalization of documents.
• Choose and industrialize embedding models, populate and maintain vector databases, manage index freshness and reconstruction.
• Ensure the scalability of indexing on massive corpora and data traceability (lineage, index versioning).
• Search & Relevance (Retrieval)
• Combine lexical and semantic search (BM25 + embeddings) with reranking and metadata filtering to maximize relevance.
• Set up test sets and relevance metrics (recall, precision, nDCG, MRR) to measure and improve retrieval.
• Structure the retrieval phase that feeds LLMs (context, citations, de-duplication, context window management).
• Instrument pipelines (logs, metrics, quality traces) and ensure data security and compliance (country isolation, regulatory compliance).
• Technical Leadership & Governance
• Promote Clean Code, SOLID, and automated testing practices.
• Ensure constant monitoring and contribute to the group's technical influence (articles, meetups, open-source).
• Languages: Python 3.11+ (required mastery), Go
• Indexing: Vector databases (pgvector, Weaviate, Qdrant, OpenSearch), hybrid search / BM25
• Embeddings & NLP: sentence-transformers, embedding models, tokenization, document parsing
• RAG & LLM: LangChain, RAG, reranking, OpenAI, Mistral, Bedrock
• Data & Infra: PostgreSQL, Airflow / Spark (or equivalent), Kafka, Redis, Docker, CI/CD, Kubernetes
• Tests & Quality: Pytest, relevance evaluation (recall, nDCG, MRR)
• Sector: Specialized Publishing & Technology Services
• Scope: France (European scope)
• Contract Type: Freelance
• Domain: Artificial Intelligence & Data Engineering

Project details

• Start: flexible
• Remote: hybrid
• Location: Paris, France, FR
• Languages: French