DIVA: Deep Indic Virtual Apparel Try-On

1Indian Institute of Technology, Madras 2National Institute of Technology, Trichy 3Detect Technologies Pvt Ltd

Abstract

Virtual try-on technology has seen significant advancements in recent years, particularly in the domain of apparel visualization. However, a notable gap exists in high-resolution (720 × 540) datasets that adequately represent diverse poses and orientations for Indic clothing styles. To address this, we introduce a novel dataset, IndicViton, comprising multiple poses of the same garment around the wearer. This dataset fills a critical void as existing open-source collections lack such diversity. In addition to dataset creation, we propose an initial diffusion model (DIVA) tailored for Indic virtual try-on applications. Our model leverages a novel approach to handle multi-pose garment images, enabling realistic and accurate virtual fitting experiences. Central to our methodology is the utilization of diffusion models, which excel in capturing intricate details and variations in fabric textures.

To validate our approach, we conducted comprehensive experimental evaluations, comparing our results against established benchmarks. Results demonstrated superior visual fidelity and quantitative performance in Indic virtual try-on scenarios. By providing this dataset and model, we aim to spur further research and development in virtual try-on technologies tailored for Indic clothing styles. The link for the dataset is attached here.

Method

Method illustration
Generated results of DIVA Model trained on the IndicViton dataset.
Method illustration
Overview of the DIVA model’s training and inference processes integrating conditioning inputs:
  • Agnostic Map (AM)
  • AM Mask (AMM)
  • DensePose (DP)
  • Cloth Garment (CG)
  • Scribble Map (SM)
During training, DIVA refines noise x_t iteratively using these inputs to generate culturally authentic images. During inference, it leverages ESRGAN for enhanced resolution post-generation. The Scribble Map (SM) serves as a valuable prior, crucial for preserving traditional nuances in Indian clothing during virtual try-ons.

Dataset Analytics

IndicViton dataset examples
Diverse Indian clothing examples from the IndicViton dataset.

Women’s Clothing Analytics collected from Myntra

Embellishment/Design Element Cloth Type Total
Kurtas Lehengas Sarees
Sequins 332 684 325 1341
Zari Work 166 673 2269 3108
Bandhani 77 43 215 335
Leheriya 25 57 103 185
Gotta Patti 192 147 57 396
Mirror Work 165 116 54 335
Ethnic Motifs 1471 - 1455 2926
Beads & Stones 84 191 154 429
Floral Motifs 1387 - 1403 2790

Men's Clothing Analytics collected from Myntra

Pattern Count
Solid 572
Ethnic Motifs 370
Geometric 235
Striped 186
Floral 127
Woven Design 66
Abstract 62

Entire dataset used after cleaning and proper annotations

Women’s Ethnic Clothes Men’s Ethnic Clothes
Cloth Name Count Cloth Name Count
Tops 6167 Kurtas 1719
Sarees 6005 Sherwanis 1498
Kurtas 4549 Dhoti Pants 1494
Maxi Skirts 2481 Nehru Jackets 1748
Lehengas 3377
Total 22579 Total 6459

Results

Results for men’s clothing
Qualitative comparison between models for men’s clothing: Each row displays Agnostic Map and Target Cloth Ground Truth, followed by results generated by CATDM, Stable-VITON, and DIVA.

Results

Results for women’s clothing
Qualitative comparison between models for women’s clothing: Each row displays Agnostic Map and Target Cloth Ground Truth, followed by results generated by CATDM, Stable-VITON, and DIVA.

Results

Generated sarees from scribble maps
Generated sarees from various scribble maps, highlighting the influence of map variations.

BibTeX

K Sai Sri Teja, Hrishith Mitra, Girish Rongali, Kaushik Mitra, DIVA- Deep Indic Virtual Apparel Try-On.
      ECCV Workshop 2024.