Story. 49.2 “Smart-Match A”: AI Image upload and Data matching

General info

We are exploring a radically simplified flow for registering bags using AI. The idea is to let users upload images first (instead of filling dropdowns manually) and receive suggested bag data based on the uploaded content.

Once images are uploaded, an AI process is triggered in the background. After approximately 10 seconds, the AI returns model suggestions, brand, size, material, and potentially serial number information. These suggestions are injected into our form fields as pre-filled values, which the user can then confirm or correct.

The AI request may fail or be incomplete based on image quality or missing views. In those cases, the user receives a simple message asking them to improve their upload. We do not flag individual empty fields — we simply recommend better image quality or more images.

The long-term goal is to reduce user friction, speed up submissions, and match bags accurately with our internal dropdown data. This story is focused on the AI-based upload flow and interaction, not on dropdown logic or data validation — that is handled in a separate story.

Estimated request time: ~10 seconds per set of images. Costs and tokens per request should be evaluated.

User story

As a user

I want to upload photos of my bag and get suggestions automatically

So that I don’t have to manually fill out long dropdown forms

Example User Images:

Example Prompt:

You are an expert in luxury handbags, especially Hermès, Chanel, Dior, and Bottega Veneta. You will be shown a set of 3–6 images of a single handbag.

Your task is to identify the handbag and return your best structured guess in the following JSON format:

{
  "brand": "",
  "brand_confidence": 0.0,
  "model": "",
  "model_confidence": 0.0,
  "size": "",
  "size_confidence": 0.0,
  "material": "",
  "material_confidence": 0.0,
  "hardware": "",
  "hardware_confidence": 0.0,
  "color": "",
  "color_confidence": 0.0,
  "serial_stamp_location": "",
  "stamp_confidence": 0.0,
  "notices": []
}

Confidence values must range from 0.0 to 1.0 and reflect your certainty in each prediction.

Only fill in values where you are reasonably confident (e.g. above 0.5). If unsure, leave the value blank but still include a confidence score (e.g. 0.2). Avoid guesses.

Model names should follow structured naming like:  
- `"Birkin 25"`  
- `"Picotin Lock 18"`  
- `"Classic Flap Medium"`  
- `"Saddle Bag"`

For Hermès, if the stamp is visible, describe its location (e.g. “Interior base”) and include that in `serial_stamp_location`.

The output is meant to pre-fill dropdown fields for a resale listing. It should be short, structured, and help reduce manual work.

If any part of the bag isn’t clearly visible (e.g. hardware is covered, interior is dark), include a note in `notices`.

Return only the JSON and nothing else.

Example AI output gpt 4o vision:

{
  "brand": "Hermès",
  "model": "Picotin Lock",
  "size": "18",
  "combined_model": "Picotin Lock 18",
  "material": "Clemence",
  "color": "Craie",
  "hardware": "Gold",
  "stamp_or_serial_hint": "Interior embossing visible (HERMÈS PARIS MADE IN FRANCE)",
  "confidence_scores": {
    "brand": 1.0,
    "model": 0.95,
    "size": 0.9,
    "material": 0.85,
    "color": 0.8,
    "hardware": 0.95
  },
  "feedback": "If any detail seems incorrect, you can update it manually before submission."

Visual design:

Desktop:

• Full-width upload box with drag-and-drop & preview

• Feedback animation and result step

Mobile:

• Large tap zone to add files

• Loader animation and compact result fields

Acceptance criteria

Notes for dev estimation

• AI response format should be JSON, containing keys for: brand, model, size, hardware, color, material, optionally serial_number, and a confidence_score

• Match to internal dropdown values must be exact or fuzzy (e.g. Birkin 30 vs. Birkin 30cm)

• No user-generated values here — the system always suggests known internal matches, or asks the user to improve image quality

• Expected model: GPT-4 Vision or similar (test in playground if needed)

• Cost & latency TBD (est. ~10 seconds per request, ~2000–3000 tokens, priced by OpenAI plan)

• Future exploration: combine vision with live web lookup or product matching — not yet in scope