Required skills
Languages: German
Job description
For a software manufacturer in a security-critical IT environment, an experienced LLM Platform Engineer is sought to transition an existing on-premise AI platform from evaluation to production. The role involves automated deployment of LLM inference services, providing an internal chat assistant, and connecting internal knowledge sources via RAG.
Project overview
For our client, a software manufacturer in a security-critical IT environment, we are looking for an experienced LLM Platform Engineer to transition an existing on-premise AI platform from evaluation to production.
Project Framework:
- Start: as soon as possible
- Duration: 6 months, extension possible
- Utilization: 40 hours/week
- Location: remote with occasional on-site appointments
- Project Language: German
Responsibilities
- Automated deployment of LLM inference services (vLLM / SGLang) via Helm on an existing Kubernetes environment with GPU scheduling
- Provisioning of an internal chat assistant (Open WebUI, LiteLLM as model router) for employees
- Connecting internal knowledge sources (documents, Confluence, Jira) via RAG, including mapping the existing authorization concept
- Deployment of open-weight and Mistral models, optimization of GPU utilization and performance
- Evaluation of the existing stack and recommendation of possible alternatives (e.g., Onyx)
- Transition to stable production operation, including monitoring and documentation
Requirements
Desirable:
If the project fits your profile, I look forward to receiving your CV, desired hourly rate, and earliest availability.
- Demonstrable experience with self-hosted LLMs in production on own GPU infrastructure (pure use of cloud APIs like Azure OpenAI is not sufficient)
- Sound practical experience with vLLM and/or SGLang
- Kubernetes and Helm in practice, ideally with GPU workloads
- Experience with RAG architectures on enterprise data, including authorization concepts
- Very good German language skills
- Open WebUI, LiteLLM, Onyx
- Self-deployment of Mistral models
- Vector databases (e.g., Qdrant, pgvector, Milvus)
- Experience in security-critical or regulated environments; willingness for security clearance is a plus
Project details
- Start: ab sofort
- Duration: 6 Monate
- Remote: remote
- Location: Nordrhein-Westfalen, Deutschland, DE
- Languages: German
Indexed by 9am's meta-search. You are a job provider? Claim or manage this listing
Found on FreelancerMap DE