How to fine-tune a Stable Diffusion model on your brand archive

Fashion brands are increasingly training custom versions of Stable Diffusion models on their design archives to generate images that match their aesthetic. In February 2026, ASOS upskilled more than 100 designers in generative AI and reported an average 75 to 80 per cent time saving across key design processes. Designer Norma Kamali trained a custom AI model on 57 years of fashion archives in collaboration with an AI studio. The process involves collecting training images, labelling them, choosing a base model, setting training parameters, validating outputs and integrating the trained model into design workflows. For most fashion applications, you will fine-tune an existing model using LoRA rather than training from scratch.

What images do you need to collect from your archive?

The quality and consistency of your training dataset determines output quality. For a face LoRA, you need 20 to 30 images at 1024 by 1024 pixels with varied angles and lighting; for a style LoRA, 15 to 40 images that share a consistent visual signal; for a concept or outfit LoRA, 10 to 25 images. Quality always matters more than quantity, with 20 excellent images outperforming 200 mediocre ones. Remove images with poor lighting, occlusions or inconsistent presentation. All images should be cropped to the same aspect ratio and resolution. Save images in PNG or JPEG format at the target training resolution.

How do you label training images?

Each training image requires a text caption stored in a separate text file with the same filename. Captions should describe visible elements without interpretation. For garment training, describe silhouette, fabric, colour, construction details and styling context. For style training, describe lighting, composition, colour palette and aesthetic qualities. Keep captions consistent in structure and vocabulary across the dataset. Use specific fashion terminology that the base model already understands rather than inventing new terms. Avoid subjective language and focus on observable attributes.

Which base model and training method should you choose?

LoRA fine-tuning combines Stable Diffusion with Low-Rank Adaptation, a method that trains only a small adapter rather than the full model. Stable Diffusion 1.5 and 2.x require 8GB to 12GB VRAM for LoRA fine-tuning and 16GB to 24GB for full training. SDXL requires 10GB to 12GB minimum for LoRA training and 24GB or more for full fine-tuning. For fashion applications, SDXL produces higher quality results at the cost of increased compute requirements. LoRA adapters are small files, typically under 200MB, that can be loaded on top of the base model at inference time. This approach allows you to train multiple adapters for different concepts without duplicating the base model.

What training parameters deliver reliable results?

Training parameters control how the model learns from your dataset. Start with these values and adjust based on validation results:

• Learning rate: 0.0001 to 0.0002 for subject training, lower for style training
• Network dimension (rank): 32 to 64 for most fashion applications, 128 for complex subjects
• Training steps: 1,500 to 4,000 for outfit or object LoRAs, 2,000 to 8,000 for style LoRAs
• Batch size: 1 to 2 for consumer GPUs with limited VRAM
• Save frequency: every 250 steps to allow comparison of checkpoints

To calculate training steps, divide the target step count by the number of training images to determine how many times each image will be used during training. Learning rate is where most training failures happen, with the right range depending entirely on LoRA type. One academic study fine-tuned Stable Diffusion on fashion data for 50 epochs using the AdamW optimiser with a learning rate of 0.00001, decreased by a factor of 0.1 every 15 epochs.

How do you validate outputs during training?

Generate test images at regular intervals during training using consistent prompts that were not in your training captions. Compare outputs across saved checkpoints to identify the optimal stopping point. Signs of undertrained models include weak adherence to your style or concept, high variation from the intended aesthetic and generic outputs. Signs of overtrained models include every generation looking identical, inability to respond to prompt variations and exact replication of training images. Stop training when test outputs match your intended style but still respond to prompt variations. Most successful training runs complete between 60 and 80 per cent of the planned steps.

What compute resources does training require?

Hardware requirements depend on base model choice and training method:

NVIDIA A100 GPUs with 80GB of memory are recommended for training, designed to handle large datasets and complex calculations typical in diffusion models. For SDXL LoRAs, a GPU with at least 12GB VRAM such as RTX 3060 12GB or RTX 4070 is sufficient, with training time ranging from 30 minutes to several hours. Budget-conscious users can use RTX A5000 with 24GB VRAM at USD 0.16 to USD 0.27 per hour for cost-effective SD 1.5 and LoRA training. Cloud GPU providers allow you to rent compute by the hour rather than purchasing hardware. Training on consumer GPUs with 8GB VRAM is possible for SD 1.5 with reduced batch sizes and resolution.

How do you integrate trained models into design workflows?

After training, test the LoRA at different weight values between 0.4 and 1.0 to find the optimal strength for your use case. Higher weights produce stronger adherence to your trained style but reduce flexibility in responding to prompts. Load the LoRA file into your inference pipeline alongside the base model. Generate initial concepts by combining text prompts with the trained LoRA, then refine outputs through iteration. ASOS embedded generative AI technology across design operations, enabling designers to turn sketches into photorealistic visuals in seconds and instantly explore different colours, fabrics and variations. Document successful prompt patterns and weight combinations for your team. Store multiple checkpoint versions to allow rollback if a later checkpoint shows degraded performance. Train separate LoRAs for distinct concepts rather than combining multiple subjects in one adapter.