Required skills
- Prompt Engineering
- AI Agents
- LLM Applications
- System Instructions
- Context Management
- Few-Shot Prompting
- Tool Calling
- Retrieval
- JSON Schemas
- API Analysis
- Quality Assurance
- English
Languages: German
Job description
We are looking for an experienced external Prompt Engineer to develop and optimize AI agents for a dialog-based banking function within a customer app and for other use cases. The goal is to measurably improve the reliability and quality of responses through systematic prompt engineering, agent configuration, and evaluation.
Project overview
For an AI Platform, we are looking for an experienced external Prompt Engineer (m/f/x). The focus is on the further development of AI agents for a dialog-based banking function in the customer app as well as for other use cases. The goal is to measurably improve the reliability and quality of responses through systematic prompt engineering, agent configuration, and evaluation.
Responsibilities
Development and optimization of system prompts, agent instructions, and reusable prompt components
Configuration of tool usage, knowledge access, routing, and handovers
Definition of agent behavior in case of missing information and ambiguous requests
Creation of evaluation datasets, assessment criteria, and regression tests
Error analysis based on API requests, tool results, and conversation histories
Execution and documentation of reproducible tests as well as measurable before-and-after comparisons
Close collaboration with engineering, evaluation, product, and banking domain teams
Must-have Requirements
Practical experience in prompt engineering for LLM applications as well as in configuring AI agents
Experience with systematic evaluation beyond individual test cases
Knowledge of system instructions, context management, few-shot prompting, tool calling, and retrieval
Experience with structured outputs and JSON schemas
Ability to analyze technical workflows and document experiments comprehensibly
Structured approach to error diagnosis, optimization, and quality assurance
Professional English skills
Nice-to-have Requirements
Python scripting and Git
Automated evaluation frameworks and LLM judges
LangChain, LangGraph, or comparable agent frameworks
Experience with LLM applications in production
Experience in banking, regulated industries, or enterprise environments
Experience with testing multilingual applications
Application / Offer
Please submit CV, availability, potential workload, and conditions (travel expenses will not be reimbursed). In addition, concrete project examples with details on your role and measurable improvements are expected. Please clearly distinguish production experience from prototypes.
Project details
- Start: 2027-01
- Duration: 11 Monate
- Remote: hybrid
- Location: Wien, Österreich, AT
- Languages: German
Indexed by 9am's meta-search. You are a job provider? Claim or manage this listing
Found on FreelancerMap AT