Required skills
Languages: French
Job description
Join an existing generative AI project team as a senior expert to improve LLM reliability, enhance evaluation, and evolve the architecture based on measurable results.
Project overview
The mission
Join a team built on an existing generative AI project. Your role: provide senior expertise to ensure reliable LLM usage, strengthen evaluation, and evolve the architecture based on measurable results.
Your responsibilities
Profile sought
- Design the information extraction chain (function calling, structured outputs, verification, and correction)
- Measure to decide: build test cases, conduct experiments, and compare prompts, models, and methods based on quantified criteria
- Qualify LLM judges against human judgment
- Choose models by balancing quality, cost, and latency, within a sovereign framework (Mistral, AWS Bedrock EU region)
- Industrialize: tested code, observability, failure and drift management
- Transmit: explain your choices to the team and business stakeholders, and upskill them on evaluation
- 5 years minimum in development, including 2 years on LLM products in production
- Proven experience in LLM evaluation: test cases, metrics, LLM-as-judge and its biases, variance between executions
- Mastery of structured information extraction (structured output, function calling, schema validation)
- Advanced Python, tested code (mandatory)
- LLM / RAG (mandatory)
- Ability to make decisions based on measurements and explain them simply
Project details
- Start: 15/11/2026
- Duration: 12 mois renouvelable
- Remote: hybrid
- Location: Paris, France, FR
- Languages: French
Indexed by 9am's meta-search. You are a job provider? Claim or manage this listing
Found on free-work.com