DoorDash described its production system for generating structured food metadata using LLM juries on Hacker News. The approach combines multiple models with voting and multimodal inputs to label menu items at scale.
How LLM Juries Generate Food Metadata
DoorDash routes each menu item through several LLMs that receive identical context windows and image crops of dish photos. Each model outputs structured fields such as cuisine type, ingredient list, dietary tags, and portion size. A jury layer aggregates outputs by majority vote and discards low-consensus results for human review.
The pipeline first retrieves similar past items via embedding search, then injects those examples into the prompt. Multimodal models receive both text and image tokens in the same forward pass.
Performance Metrics Reported
The post states the jury system processes thousands of items daily with measurable gains in precision over single-model baselines. Context optimization reduced token usage while maintaining accuracy on long-tail menu items. Multimodal fusion improved ingredient detection when text descriptions were incomplete.
HN discussion noted 41 points and 10 comments, with users asking about latency and failure modes on non-English menus.
Pros and Cons
- Jury voting cuts hallucinated ingredients compared with single-model runs.
- Context retrieval lowers prompt length and cost per item.
- Multimodal inputs handle image-only menu uploads without OCR.
- Requires multiple model calls, increasing total inference cost.
- Consensus thresholds still route 10-20% of items to human queues.
Comparison to Traditional Annotation Methods
| Method | Precision | Throughput | Cost per 1k items | Scalability |
|---|---|---|---|---|
| Human annotators | High | Low | High | Limited |
| Single LLM | Medium | High | Low | High |
| LLM jury + retrieval | High | High | Medium | High |
Single-model prompting lacks the error-correction step that juries provide. Pure human pipelines cannot match the daily volume DoorDash requires.
Who Should Use This
Teams managing large catalogs of user-generated or third-party content benefit when label consistency matters more than per-item latency. Companies already running multiple LLMs in parallel can add a lightweight jury layer without new infrastructure. Teams with fewer than 500 items per day or strict real-time constraints should skip the added complexity.
Verdict on Scalable Metadata Pipelines
LLM juries combined with retrieval and multimodal inputs give DoorDash a practical middle ground between fully automated and fully manual metadata production. The pattern is reproducible for any domain that needs structured labels from noisy text and images.
Top comments (0)