Data Labeling Analyst III

Project overview

We are seeking detail-oriented and skilled contractors to support labeling and quality assurance for Large Language Model (LLM) training focused on Premier Partner Support. The successful candidates will play a critical role in training AI models to detect and understand policy guidelines, with a particular emphasis on account name changes. Your work will ensure high-quality labeling accuracy and consistency, directly impacting the effectiveness of our AI-driven protection systems.

Responsibilities

Perform detailed labeling and annotation of data related to Premier Partner Support, including account change approvals, denials, policy exceptions, and related abuse, to support LLM training

Ensure accuracy, consistency, and strict adherence to labeling guidelines and policies.

Collaborate with internal teams to create and maintain golden sets for quality control and model calibration.

Identify and report patterns or trends in support requests related to account changes that may indicate policy violations or require exceptions, to inform model improvements.

Participate in vendor site calibration and quality assurance processes as needed.

Provide actionable feedback to tooling and engineering teams to improve labeling workflows and annotation tools.

Communicate effectively with internal stakeholders to align on labeling standards and quality expectations.

Contribute to process improvements and documentation related to labeling workflows.

Stay up-to-date with policy changes, labeling guidelines, and industry best practices related to account changes.

Requirements

Experience in data labeling, quality assurance, or related fields (minimum 4+ years).

Direct experience working in operational support, combined with a strong understanding of policy guidelines is highly relevant.

Excellent communication skill and perfect English with writing and speaking.

Strong attention to detail and ability to maintain high accuracy in a fast-paced environment.

Ability to work independently and collaboratively within a team.

Familiarity with platforms, policy enforcement, or integrity problems is a plus.

Knowledge of social media platforms, account structures, and enforcement policies.

Experience with AI/ML model training or data annotation for machine learning.

Technical skills (e.g., experience with annotation tools, providing feedback to engineering teams, or basic scripting) to support tooling improvements.

Prior experience working with vendor management or quality calibration.

Certification or training in content moderation or data labeling.

Working Conditions:

This is a contractor role with an in office setting.

Must be able to work in a fast-paced environment with multiple priorities and deadlines.

Years of Experience: 3-4 years of experience as a data labeling, quality assurance, or related fields

Degrees/Certifications Required: Bachelor s degree nice to have

Strong attention to detail and ability to maintain high accuracy in a fast-paced environment

Direct experience working in operational support, combined with a strong understanding of policy guidelines is highly relevant

Analytical mindset with the ability to identify patterns and anomalies in complex data

Good to have skills:

Certification or training in content moderation or data. labeling

Interview: 1 round

Job Facts: Contract W2 | 6 Months | Remote | $28 - $30/hr

Skills: Documentation, Social Media, Training, Quality Assurance, Large Language Models (LLMs), Machine Learning (ML), Vendor Management, Process Improvement, Reporting, Scripting, Workflow, Communication, Quality Control, Analytical Skill, Attention To Detail

Reference: Dice Id: 10123373 | Position Id: 748596

Project details

• Start: flexible
• Duration: 6 Months
• Remote: remote
• Location: US