AI Developer

Project overview

Integrated into the team and its sprints, the consultant will work on the following topics:

Deliverables

Profile sought

Work environment

We are looking for an external AI Engineer to strengthen the team's capacity.

The AI Solutions team designs, optimizes, and operates the product range's AI products: AI Endpoints, which provides open-source models via API, and Machine Learning Services products (AI Notebooks, AI Training, AI Deploy).

• Integration and deployment of new open-source models on AI Endpoints (LLM, embeddings, speech-to-text, text-to-speech, ...)
• Inference optimization with vLLM and SGLang: quantization, batching, KV cache, prefix caching, speculative decoding
• Performance benchmarks on our current GPUs and upcoming new generations
• Improvement of autoscaling, observability, and reliability of production services
• Participate in product run
• New models deployed in production on AI Endpoints, with optimized and documented inference configuration
• Benchmark reports per model and per GPU, with recommendations
• Code delivered via reviewed and integrated pull requests (Python, Rust)
• Service monitoring dashboards and alerts
• Technical documentation and knowledge transfer to the team
• Proven experience deploying LLMs in production on GPUs
• Mastery of vLLM and/or SGLang
• Inference optimization: quantization (AWQ, GPTQ, FP8), KV cache, batching, speculative decoding
• Python and Rust
• NVIDIA GPU environments, Docker, Kubernetes
• Observability: Prometheus, Grafana, OpenTelemetry
• Git, CI/CD, code reviews, agile methods
• Proficient use of AI code assistants

Project details

• Start: ASAP
• Duration: 3 mois renouvelable
• Remote: hybrid
• Location: Paris, Île-de-France, FR
• Languages: French