Quality control for AI-written product pages: a practical guide
Generated product descriptions can invent attributes, drift off brand and trigger search penalties. A review workflow with automated checks, human sampling and clear labelling.
KEY TAKEAWAYS Summary by the editors
- Amazon researchers found in 2024 that automated hallucination detection in LLM-enriched product listings works well for structured attributes with deterministic values but poorly for long free text, where precision was low and LLM validation added only marginal gains.
- Google's guidance warns that mass-producing pages with AI without adding value can breach its spam policy on scaled content abuse, and that generative models do not retrieve facts, so all output needs review before publishing.
- Google states that titles, meta descriptions, structured data and image alt text need the same review as body copy, and that Merchant Center expects AI-generated titles and descriptions to be provided separately and labelled as AI-generated.
- Stitch Fix describes a model fine-tuned on several hundred expert-written descriptions paired with attributes, with copywriters reviewing and editing every output, and regular quality checks feeding back into the model.
- The safest design generates text only from verified product attributes, runs automated checks against those attributes, and routes a risk-based sample plus every flagged page to a human editor.
Quality control for AI-written product pages means generating copy only from verified product data, checking the result automatically against that data, and having a person review a risk-based sample and every flagged page before publication. The aim is to catch invented attributes, claims you cannot support and generic text that adds nothing. Automated checks help, but published research shows they are weaker on free text than on structured fields.
What can go wrong with AI-written product pages?
- Invented or unsupported attributes: a model adds a lining, a fit or a use that is not in the product record.
- Wrong or conflicting facts: composition, care instructions, size information or dimensions that disagree with the data sheet.
- Unsubstantiated claims: sustainability, performance or origin statements with no evidence behind them.
- Brand and tone drift: language that sounds generic or contradicts your style guide.
- Search risk: large volumes of near-identical pages that add little value.
Amazon researchers define hallucination in this setting as text that is unfaithful to the provided source input, and distinguish intrinsic cases (the output contradicts the source) from extrinsic ones (the output cannot be verified against the source). Some unsupported values are still correct, for example a colour that matches the product image, while others are both unsupported and wrong. Your policy needs to decide whether unsupported but plausible statements are allowed. For most fashion retailers, the safe answer is no.
How do you generate copy that stays true to the data?
Start from a clean product record: attributes, composition, measurements, care and certifications in structured fields. Instruct the model to use only those fields and to leave a gap rather than guess. Stitch Fix describes fine-tuning a base model on several hundred descriptions written by human experts, each paired with its product attributes, after few-shot prompting produced generic, limited-quality text. It reports higher quality scores for the AI descriptions in a blind evaluation against human-written text, though it gives no numbers. The experts defined the target up front: originality, natural and compelling wording, truthful claims and brand alignment.
Which automated checks work best?
The Amazon paper proposes two phases. First, cheap lexical and semantic screening (token matching, n-gram metrics, embedding similarity and similar methods) flags candidates with high recall. Second, a large language model validates the flagged items to improve precision. On structured attributes with deterministic values such as colour the approach performed strongly, and the screening cut the entries needing validation to under 12.5%. On unstructured, long free text, recall was satisfactory but precision was low, and the LLM validation step gave only marginal gains. The authors conclude that the method works better on structured attributes.
| Check | Method | Best used for |
|---|---|---|
| Attribute match (colour, material, fit, length) | Rules and string or semantic matching against the product record | Every page, automatically |
| Banned and regulated claims list | Keyword and pattern rules | Every page, automatically |
| Numbers and units (sizes, percentages) | Pattern extraction compared with source | Every page, automatically |
| Tone and brand voice | Style rules plus human review | Sampled pages |
| Free text faithfulness | Model-assisted check plus human review | All flagged pages and a sample |
| Duplicate and near-duplicate content | Similarity scoring across the catalogue | Catalogue wide, weekly |
| Image and text consistency | Review against product images | High value or new styles |
What does Google say about AI-generated content?
Google's guidance on generative AI content says AI-assisted content must still meet Search Essentials and its spam policies, and that mass-producing pages without adding value for users can violate the spam policy on scaled content abuse. It explains that generative models predict likely word sequences rather than retrieving facts, so outputs can be wrong, and advises checking facts and reviewing all content before publishing. It also says titles, meta descriptions, structured data and image alt text need the same scrutiny, and that sites should explain to readers how automation was used. For ecommerce, it says AI-generated images need the IPTC TrainedAlgorithmicMedia digital source type metadata in Merchant Center, and that AI-generated product titles and descriptions must be provided separately and labelled as AI-generated.
How should human review be organised?
- Define the risk tiers: new categories, high value styles, regulated claims and anything flagged by automated checks go to full review; low risk repeat styles get a sample.
- Give editors a checklist: attributes against the data sheet, claims against evidence, tone against the style guide, uniqueness against the rest of the catalogue.
- Show reviewers the source attributes next to the generated text, so they can compare quickly.
- Record every edit and its reason. Stitch Fix describes regular quality checks feeding insights back into the model; the same loop lets you change prompts and rules where editors keep fixing the same problem.
- Track metrics: error rate per hundred pages by type, share of pages passing without edits, review time per page and return reasons coded as not as described.
Plan capacity honestly. Review time per page is the cost that decides whether AI saves money, so measure it from the first week. If editors spend as long correcting a draft as writing one, narrow the use to categories with rich structured data, or to short fields such as bullet points, where checks are more reliable.
What should you measure after publication?
Quality control continues after launch. Compare pages produced with and without AI for conversion, return rate by reason, customer service contacts that mention the description, and search performance. If pages written with AI show more returns coded as not as described, tighten the checks. Keep a rollback path: be able to revert a batch quickly when an error pattern appears, and keep an audit log of which pages used which prompt and model version.
Frequently asked questions
Does Google penalise AI-generated product descriptions?
Google does not penalise content only because AI was used, but its guidance says mass-producing pages without adding value can violate its spam policy on scaled content abuse. It advises reviewing AI content before publishing and, for Merchant Center, labelling AI-generated titles and descriptions.
How can I detect AI hallucinations in product copy?
Compare the generated text with the structured product record. Research from Amazon shows that screening plus LLM validation works well for structured attributes like colour but poorly for long free text, so combine automated checks with human review of flagged and sampled pages.
How much human review do AI-written product pages need?
Review all pages in high risk tiers, such as regulated claims, new categories or high value items, and every page flagged by automated checks. Sample the rest. Stitch Fix describes having copywriters review and edit generated descriptions rather than writing from scratch.
Should product pages say they were written with AI?
Google advises explaining to readers how automation was used, and Merchant Center requires AI-generated titles and descriptions to be provided separately and labelled as such. Check requirements for each channel and market.
One edition every weekday morning. Read in five minutes. Free for industry professionals.
