AI-Assisted, Human-Reviewed Polygon Segmentation
The variability that made this dataset valuable for AI training—multiple restaurant chains, diverse lighting conditions, regional menu differences—also made it exceptionally challenging to annotate. Our solution wasn't simply to apply more resources, but to develop a carefully designed workflow where technology, human expertise, and quality controls worked together to overcome these challenges. Each element of the workflow compensated for the limitations of the others, ensuring high-quality annotations and a reliable dataset for training AI models.
Here’s how we approached the solution:
AI-Assisted Polygon Segmentation using CVAT
To manage the large volume of images and ensure precise labeling, we utilized CVAT (Computer Vision Annotation Tool), which can be optimized for handling image annotation tasks at scale.
CVAT offers an AI-assisted polygon segmentation feature that we used to annotate irregularly shaped food items accurately. Our team also used CVAT's built-in brightness adjustment features to temporarily enhance visibility during annotation—without altering the original image data—ensuring accurate labeling even in challenging lighting scenarios while preserving data integrity for AI training.
Annotator Training by Subject Matter Experts
A specialized team of 10 annotators was assigned to the project, and a domain expert was aligned with them to ensure consistent handling of the high-volume dataset (20,000+ images). Each team member underwent comprehensive training on:
- Restaurant-specific terminology across multiple chains (McDonald's, KFC, etc.)
- Regional menu variations and packaging differences
- Contextual differentiation techniques for visually similar items
- Best practices for annotating crowded or partially obscured food items
Standardized Labeling Protocols and Classification Taxonomy
Given the diverse set of food items and packaging products, it was crucial to establish and follow uniform image annotation protocols. We developed a detailed labeling taxonomy for this client that accounted for the diverse food categories, packaging types, and regional variations present in the dataset, ensuring consistency across all annotators. This taxonomy included:
- Predefined categories for standard food items (burgers, fries, chicken, coffee, wraps, sauces, etc.) and standard packaging
- Clear guidelines for handling edge cases (obscured items, overlapping elements, poor lighting)
- Region-specific naming conventions to accommodate international menu differences
Multi-Level Quality Assurance Process
High-volume annotation projects present a fundamental tension: speed often compromises precision, while excessive quality controls can hinder throughput. Our image annotation solution reconciled this by distributing quality assurance across the workflow rather than concentrating it at the end.
- Peer Review: Cross-checking between annotators to catch initial errors
- Internal QA: Subject matter experts reviewed batches for consistency and adherence to the standard labeling taxonomy
- Client Collaboration: Regular feedback loops with the client's team to validate annotations against their AI training requirements and real-world performance expectations
Each image underwent a minimum of 2 review cycles before final approval. Quality checkpoints were conducted every 2,000 images to maintain consistency and check for any annotation drift (typical in an image annotation project at this scale). We also maintained an internal accuracy threshold of 98% before submitting labeled data to the client.
Raw image
Annotated image
