Odixcity Consulting is a Nigerian HR consulting and procurement firm that provides business solutions to companies, entrepreneurs, and SMEs. We offer a wide range of services including recruitment, performance management, training and development, compensation and benefits, payroll and benefits administration, and procurement of goods and services.
Job Summary
We are seeking an experienced LLM Evaluator to assess, annotate, and improve outputs from large language models (LLMs). The position focuses on reviewing AI-generated text for accuracy, reasoning quality, factual correctness, and alignment with task objectives.
Key Responsibilities
- Evaluate AI-generated outputs across multiple domains and use cases, including summarization, dialogue, reasoning, and content generation.
- Rank model responses and provide structured justification for evaluation decisions.
- Identify factual inaccuracies, logical inconsistencies, biased or inappropriate content, and formatting issues.
- Develop and maintain evaluation rubrics and internal guidelines for consistent assessment.
- Collaborate with AI trainers, data scientists, and ML engineers to refine model behavior.
- Participate in QA cycles to ensure consistency, depth, and quality of annotations.
- Provide detailed feedback to improve model outputs and training datasets.
Requirements
- Bachelor’s Degree in Computer Science or a related field.
- Minimum of 5 years relevant professional experience.
- Direct experience working with large language models (e.g., GPT, Claude, Gemini) or other generative AI systems.
- Familiarity with prompt engineering, annotation workflows, and AI training pipelines.
- Technical literacy in NLP or AI concepts.
- Experience in annotation leadership, QA, or model evaluation projects.
Method of Application
Meet the Qualifications? Email your CVs and cover letters to [email protected] using the Job Title as the subject.