Chief Gen AI Data Scientist (2 month contract)

Strategic Staffing Solutions customer is looking for a hands-on AI Engineer with a strong background in Data Science, Python, and LLM evaluation to help build and scale an enterprise AI Skills Registry. In this role, you will work with modern AI technologies, including AWS Bedrock, AWS AgentCore, Model Context Protocol (MCP), and Amazon Quick, focusing on developing reliable AI capabilities and evaluating the performance of agentic AI systems. Key Responsibilities

• Develop and integrate AI components for AWS AgentCore and Amazon Quick, supporting an MCP-based AI Skills Registry.
• Design and implement evaluation frameworks for LLM-powered agents, skills, and tools.
• Conduct structured error analysis to identify failure patterns, improve accuracy, and enhance system reliability.
• Develop evaluation methodologies, including rubric design, LLM-as-a-judge approaches, and calibration techniques.
• Define, monitor, and optimise AI quality, performance, and cost-efficiency metrics using tools such as Arize AX.
• Support the implementation of scalable evaluation and continuous improvement practices across AI solutions.
• Collaborate with technical teams to ensure AI systems are reliable, measurable, and production-ready.

Requirements

• Hands-on experience developing or evaluating ML/LLM-based systems in production, particularly agentic AI systems involving multi-step reasoning and tool usage.
• Strong Python programming and data analysis skills.
• Solid understanding of Data Science principles and ML/LLM evaluation methodologies.
• Experience with error analysis, evaluation metrics, rubric design, and LLM-as-a-judge frameworks.
• Experience working with cloud-based AI/ML infrastructure, such as AWS Bedrock or equivalent platforms.
• Ability to work independently, take ownership, and develop solutions in a fast-evolving environment.
• Strong analytical thinking and problem-solving skills.

Nice to Have

• Experience with AWS AgentCore, Strands Agents, or similar agent frameworks.
• Familiarity with Model Context Protocol (MCP).
• Experience with AI observability and evaluation tools such as Arize AX.
• Background in regulated industries, such as financial services or healthcare.
• Understanding of GDPR and data protection requirements.

Main information:

• Contract for 2 months with possibility to extend
• Hybrid work model
• B2B contract with pay rate up to 830 DKK/h

Interested? Apply now or get in touch to learn more!