AI Prompt Engineer & LLM Evaluator (Remote / Contract)

Are you obsessed with breaking, testing, and optimizing language models to produce high-quality, predictable outputs? We are building DataGenius—an advanced AI-driven B2B intelligence and synthetic data generation platform. We rely heavily on complex prompt chains and LLM interactions to generate accurate, unbiased, and structured data for enterprise workflows. We are looking for an AI Prompt Engineer & LLM Evaluator to join us remotely. In this role, you won’t just be writing prompts; you will be rigorously stress-testing our generative pipelines, identifying hallucinations, mitigating edge-case failures, and ensuring our AI models deliver consistent, enterprise-grade data structures every single time. Key Responsibilities

• Design & Optimize Prompt Chains: Build, test, and refine system prompts and multi-step agent workflows to improve the quality of synthetic data and market intelligence outputs.
• Establish Evaluation Benchmarks: Create automated and manual QA frameworks to evaluate model outputs for accuracy, formatting consistency (e.g., strict JSON outputs), and hallucination rates.
• Stress-Test Edge Cases: Push the AI workflows to their limits by introducing edge cases, ambiguous inputs, and large context constraints to see where the system breaks.
• Iterative Tuning: Collaborate directly with the development team to analyze output logs, adjust prompt parameters, and test different foundational models to find the optimal balance of speed, cost, and quality.

Qualifications & Experience

• Hands-on LLM Experience: Proven experience designing complex prompts for modern language models (Gemini, Claude, GPT-4, etc.) in a production or software development environment.
• Analytical Mindset: Strong understanding of how LLMs interpret context windows, zero-shot vs. few-shot prompting, and retrieval-augmented generation (RAG) mechanics.
• Attention to Detail: Ability to spot subtle formatting errors, statistical skew, or logic flaws in large datasets generated by AI.
• Agile & Autonomous: Comfortable working in a fast-paced, asynchronous, remote environment, taking initiative to find and fix prompt-related bugs before they hit production.

Before You Apply — Explore DataGenius We want to work with people who are curious about what we are building and understand the value of high-quality data generation. 👉 Visit datagenius.io to explore the platform. Take a look around to understand our mission. When we contact the right candidates, we expect you to have a basic understanding of what the platform does. How to Apply Submit your profile/resume directly here on LinkedIn. In your application note, briefly mention one potential challenge you anticipate when prompting an LLM to generate synthetic B2B data, and how you might approach solving it.