ARShoe: Real-Time Augmented Reality Shoe Try-on System on Smartphones

• URL: http://arxiv.org/abs/2108.10515v1
• Date: Tue, 24 Aug 2021 03:54:45 GMT
• Title: ARShoe: Real-Time Augmented Reality Shoe Try-on System on Smartphones
• Authors: Shan An, Guangfu Che, Jinghao Guo, Haogang Zhu, Junjie Ye, Fangru Zhou, Zhaoqi Zhu, Dong Wei, Aishan Liu, Wei Zhang
• Abstract summary: This work proposes a real-time augmented reality virtual shoe try-on system for smartphones, namely ARShoe. ARShoe adopts a novel multi-branch network to realize pose estimation and segmentation simultaneously. For training and evaluation, we construct the very first large-scale foot benchmark with multiple virtual shoe try-on task-related labels.
• Score: 14.494454213703111
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Virtual try-on technology enables users to try various fashion items using augmented reality and provides a convenient online shopping experience. However, most previous works focus on the virtual try-on for clothes while neglecting that for shoes, which is also a promising task. To this concern, this work proposes a real-time augmented reality virtual shoe try-on system for smartphones, namely ARShoe. Specifically, ARShoe adopts a novel multi-branch network to realize pose estimation and segmentation simultaneously. A solution to generate realistic 3D shoe model occlusion during the try-on process is presented. To achieve a smooth and stable try-on effect, this work further develop a novel stabilization method. Moreover, for training and evaluation, we construct the very first large-scale foot benchmark with multiple virtual shoe try-on task-related labels annotated. Exhaustive experiments on our newly constructed benchmark demonstrate the satisfying performance of ARShoe. Practical tests on common smartphones validate the real-time performance and stabilization of the proposed approach.

Related papers

• GarmentLab: A Unified Simulation and Benchmark for Garment Manipulation [12.940189262612677]
  GarmentLab is a content-rich benchmark and realistic simulation designed for deformable object and garment manipulation. Our benchmark encompasses a diverse range of garment types, robotic systems and manipulators. We evaluate state-of-the-art vision methods, reinforcement learning, and imitation learning approaches on these tasks.
  arXiv  Detail & Related papers  (2024-11-02T10:09:08Z)
• ShoeModel: Learning to Wear on the User-specified Shoes via Diffusion Model [60.60623356092564]
  We propose a shoe-wearing system, called Shoe-Model, to generate plausible images of human legs interacting with the given shoes. Compared to baselines, our ShoeModel is shown to generalize better to different type of shoes and has ability of keeping the ID-consistency of the given shoes.
  arXiv  Detail & Related papers  (2024-04-07T06:56:51Z)
• DM-VTON: Distilled Mobile Real-time Virtual Try-On [16.35842298296878]
  Distilled Mobile Real-time Virtual Try-On (DM-VTON) is a novel virtual try-on framework designed to achieve simplicity and efficiency. We introduce an efficient Mobile Generative Module within the Student network, significantly reducing the runtime. Experimental results show that the proposed method can achieve 40 frames per second on a single Nvidia Tesla T4 GPU.
  arXiv  Detail & Related papers  (2023-08-26T07:46:27Z)
• Real-time Virtual-Try-On from a Single Example Image through Deep Inverse Graphics and Learned Differentiable Renderers [13.894134334543363]
  We propose a novel framework based on deep learning to build a real-time inverse graphics encoder. Our imitator is a generative network that learns to accurately reproduce the behavior of a given non-differentiable image. Our framework enables novel applications where consumers can virtually try-on a novel unknown product from an inspirational reference image.
  arXiv  Detail & Related papers  (2022-05-12T18:44:00Z)
• Evaluating Continual Learning Algorithms by Generating 3D Virtual Environments [66.83839051693695]
  Continual learning refers to the ability of humans and animals to incrementally learn over time in a given environment. We propose to leverage recent advances in 3D virtual environments in order to approach the automatic generation of potentially life-long dynamic scenes with photo-realistic appearance. A novel element of this paper is that scenes are described in a parametric way, thus allowing the user to fully control the visual complexity of the input stream the agent perceives.
  arXiv  Detail & Related papers  (2021-09-16T10:37:21Z)
• BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments [70.18430114842094]
  We introduce BEHAVIOR, a benchmark for embodied AI with 100 activities in simulation. These activities are designed to be realistic, diverse, and complex. We include 500 human demonstrations in virtual reality (VR) to serve as the human ground truth.
  arXiv  Detail & Related papers  (2021-08-06T23:36:23Z)
• Cloth Interactive Transformer for Virtual Try-On [106.21605249649957]
  We propose a novel two-stage cloth interactive transformer (CIT) method for the virtual try-on task. In the first stage, we design a CIT matching block, aiming to precisely capture the long-range correlations between the cloth-agnostic person information and the in-shop cloth information. In the second stage, we put forth a CIT reasoning block for establishing global mutual interactive dependencies among person representation, the warped clothing item, and the corresponding warped cloth mask.
  arXiv  Detail & Related papers  (2021-04-12T14:45:32Z)
• ShineOn: Illuminating Design Choices for Practical Video-based Virtual Clothing Try-on [8.909228149756993]
  We build a series of scientific experiments to isolate effective design choices in video synthesis for virtual clothing try-on. Specifically, we investigate the effect of different pose annotations, self-attention layer placement, and activation functions. GELU and ReLU activation functions are the most effective in our experiments despite the appeal of newer activations such as Swish and Sine.
  arXiv  Detail & Related papers  (2020-12-18T20:13:09Z)
• Reactive Long Horizon Task Execution via Visual Skill and Precondition Models [59.76233967614774]
  We describe an approach for sim-to-real training that can accomplish unseen robotic tasks using models learned in simulation to ground components of a simple task planner. We show an increase in success rate from 91.6% to 98% in simulation and from 10% to 80% success rate in the real-world as compared with naive baselines.
  arXiv  Detail & Related papers  (2020-11-17T15:24:01Z)
• Visual Imitation Made Easy [102.36509665008732]
  We present an alternate interface for imitation that simplifies the data collection process while allowing for easy transfer to robots. We use commercially available reacher-grabber assistive tools both as a data collection device and as the robot's end-effector. We experimentally evaluate on two challenging tasks: non-prehensile pushing and prehensile stacking, with 1000 diverse demonstrations for each task.
  arXiv  Detail & Related papers  (2020-08-11T17:58:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.