Required skills
- Machine Learning
- AI
- GenAI
- Quality Assurance
- DevOps
- MLOps
- FastAPI
- Kubernetes
- OpenShift
- ArgoCD
- Jenkins
- LangChain
- LangGraph
- Vector Databases
- Milvus
- Data Privacy
- Agile
- SAFe
Languages: German
Job description
We are looking for an experienced Machine Learning Engineer for the development, quality assurance, and operational deployment of AI and GenAI solutions.
Project overview
For our client, we are currently looking for an experienced Machine Learning Engineer (m/f/d) for the development, quality assurance, and operational deployment of AI and GenAI solutions.
Responsibilities
Knowledge:
We look forward to receiving your CV, preferably with your hourly rate and possible start date.
We are of course available for any questions you may have.
- Development and further development of a standardized evaluation pipeline for AI and GenAI solutions
- Development and maintenance of versioned Golden Sets and Eval Sets, as well as ensuring the traceability and comparability of evaluation results
- Implementation of automated evaluation runs for systematic quality measurement and detection of quality regressions
- Development of an evaluation layer based on deterministic checks and LLM-Judge methods, including definition and documentation of evaluation criteria
- Development and maintenance of a metric catalog for evaluating retrieval, response, and scope quality, as well as relevant operational and resource metrics
- Development of repeatable and parameterized evaluation processes, including versioned result storage and end-to-end evaluation
- Development and operation of document processing for receiving multiple documents, extracting technical information, and grouping and aggregating results
- Integration of AI solutions into existing technical services via a Service Registry, as well as maintenance of associated service and field descriptions
- Development and continuous maintenance of a knowledge base, including indexing, chunking, context preparation, and updating of structured and unstructured data sources
- Deployment, operation, and monitoring of AI solutions in test, staging, and production environments
- Implementation of observability and logging solutions, as well as performing error analysis and troubleshooting
- Ensuring data protection-compliant processing of personal data within AI applications and logging processes
- Completed degree in Computer Science, Business Informatics, or a comparable qualification
- At least 3 years of professional experience as a Machine Learning Engineer with practical experience in the conception, development, implementation, and continuous optimization of scalable machine learning solutions
- At least 2 years of experience in project work within agile development teams
- German language skills at least at B2 level, spoken and written
- Experience in systematic quality measurement of GenAI solutions, particularly with versioned Eval Sets, automated evaluation runs, and detection of quality regressions
- Practical experience with automated evaluation of model outputs, especially LLM-Judge methods and deterministic checks with documented evaluation criteria
- Experience in building and maintaining vector stores, ideally with Milvus, including indexing, chunking, updating, and source-separated storage
- Practical experience in backend development with FastAPI
- Experience with LLM orchestration frameworks such as LangChain and LangGraph
- Experience in the operational deployment of AI systems (MLOps), especially with monitoring, structured logging, error analysis, deployment, and troubleshooting
- Practical experience in deploying distributed applications across multiple environments using OpenShift or Kubernetes, ArgoCD, and Jenkins
- Knowledge in data protection-compliant design of AI systems, especially in handling personal data in documents and logs
- Experience in agile delivery structures, ideally in the Scaled Agile Framework (SAFe), including participation in an Agile Release Train (ART) and PI Planning
- Ideally, project experience in public administration, regulated environments, or statutory social insurance carriers
Project details
16.11.2026
until 01.12.2027
90% remote, 10% on-site in Hanover, Hamburg, or Munich
Utilization: Full-time
Goal:
Development and further development of scalable AI and GenAI solutions with a special focus on automated quality measurement, evaluation pipelines, and operational deployment. This includes in particular the development of evaluation methods, the creation of a knowledge base, and the integration and provision of AI services.
- Start: 2026-11
- Duration: 12 Monate
- Remote: hybrid
- Location: Hannover, Deutschland, DE
- Languages: German
Indexed by 9am's meta-search. You are a job provider? Claim or manage this listing
Found on FreelancerMap DE