Required skills
- Generative AI
- AI Agents
- LLM
- RAG
- Test Automation
- CI/CD
- Azure DevOps
- GitHub
- Quality Assurance
- Prompt Engineering
- Security Testing
- Performance Testing
Languages: French
Job description
Our client, a major audit and consulting firm, is looking to strengthen its capacity to industrialize AI-generated prototypes. They are seeking an AI QA specialist to design, develop, and improve AI agents for test design and scripting, and to define and implement the qualification strategy for AI solutions.
Project overview
Context
Our client is a major audit and consulting firm. It develops and operates about fifteen in-house application products in C# .NET / Vue.js on Microsoft Azure, intended for its Audit, Advisory, and support functions.
It is currently strengthening its capacity to industrialize POCs/POVs built rapidly with generative AI ("vibe coding"). These prototypes rely on cloud AI services (LLM, RAG, agents) and must become reliable, secure, and production-ready products.
Your missions
- Design, develop, and improve AI agents for test design and scripting: generation of test cases, scenarios, and automated scripts
- Define and implement the qualification strategy for AI solutions (LLM, RAG, agents)
- Implement evaluation methods adapted to generative AI:
- reference test sets
- quality metrics (relevance, fidelity, hallucinations)
- non-regression of prompts and models
- Secure the transition from POC to product: acceptance criteria, robustness, security (prompt injection, data leakage), performance, and cost control
- Integrate tests into CI/CD pipelines (Azure DevOps / GitHub)
- Support product teams on quality best practices applied to AI
Project details
- Start: flexible
- Remote: hybrid
- Location: Paris, France, FR
- Rate: 56-56 EUR/h
- Languages: French
Indexed by 9am's meta-search. You are a job provider? Claim or manage this listing
Found on collective.work