ViDiHand

The Surprising Effectiveness of Video Diffusion Models for Hand Motion Reconstruction

Yuxi Wang1, Chengkai Jin1, Yufei Liu2, Wenqi Ouyang1, Tianyi Wei1, Zhiwei Zeng1, Siyuan Huang2, Zhiqi Shen1, Xingang Pan1
1Nanyang Technological University   2Shanghai Jiao Tong University
Code ArXiv

Performance Overview

ViDiHand satisfies the target properties of occlusion robustness, accuracy, and temporal smoothness for 4D hand recovery.

Reconstruction Results

Click a case below, then switch between our results and method comparison.

Click ↑ a case to preview
Synchronized multi-view playback
Source
Joint Mesh
3D Mesh
Ego View
Front View
Master View

Method Overview

Top

The VACE branch is finetuned with hand-overlay rendering while the base DiT remains frozen, yielding a hand-aware video diffusion model.

Middle

A lightweight dual-branch decoder extracts MANO pose, 2D joints, and translation from a single intermediate VACE feature.

Bottom

At inference, the same feature is decoded in a single VACE pass.

ViDiHand method overview

Quantitative Results

Comparison on three egocentric benchmarks. ARCTIC and HOT3D are in-distribution; HOI4D is a held-out cross-dataset comparison fair to all methods.

Method Detection 3D Pose Orient. & Position Temporal
FAcc ↑Recall ↑F1 ↑ MPJPE-p ↓PA-MPJPE-p ↓ EPE-p ↓GO-p ↓CT-p ↓ Jitter ↓
ARCTIC InterWild0.8780.9430.95930.81715.95253.88825.3860.09746.577
HaMeR0.8750.9430.95729.19714.59665.28924.9070.09518.279
Hamba0.8330.9120.94131.23317.16887.04727.8220.11015.357
WildHands0.8790.9460.96025.70413.94150.51722.3200.05812.972
WiLoR0.9190.9510.97422.01211.87371.52717.3580.07524.091
Dyn-HaMR0.8420.9180.95127.90417.01785.72325.9510.12112.840
HaWoR0.7000.8170.89545.35726.375158.06243.3250.14919.789
OmniHands0.8660.9490.95429.67414.20351.50524.5800.08745.312
ViDiHand (Ours)0.9970.9990.99921.6689.82112.40714.6420.0473.183
HOT3D InterWild0.6690.8810.86877.16824.81171.48258.5010.213101.164
HaMeR0.6920.9040.88368.31421.45559.07749.6360.10223.632
Hamba0.6320.8280.85371.73229.620107.62556.5250.12818.507
WildHands0.6550.8630.84452.79128.946111.43853.9330.15722.885
WiLoR0.8270.8970.93730.96619.98072.97825.7460.09817.976
Dyn-HaMR0.6140.8110.80274.21438.201171.61743.8510.57144.942
HaWoR0.3480.4990.65471.39666.031327.29479.3500.26223.872
OmniHands0.6490.8950.86863.28122.68268.43749.1200.13369.510
ViDiHand (Ours)0.9480.9740.98321.51411.38314.95315.8290.0403.741
HOI4D InterWild0.7310.9220.86453.07222.90980.54941.7430.22898.866
HaMeR0.7310.9230.86444.48121.58079.49433.5570.18720.068
Hamba0.7100.8850.84947.16125.924115.79337.3900.20421.556
WildHands0.7300.9240.86445.62323.60182.24645.6540.15918.615
WiLoR0.9620.9660.97233.71014.90341.57925.5270.11517.449
Dyn-HaMR0.7500.8630.84545.09729.259144.64340.1760.25817.947
HaWoR0.8690.8640.91947.32928.851135.74843.0910.13928.376
OmniHands0.6550.9370.83444.25518.68970.66234.3920.10824.212
ViDiHand (Ours)0.9840.9910.99030.09013.96024.46023.4200.1174.010

Citation

If you find ViDiHand useful for your research, please consider citing our paper.

@misc{wang2026surprisingeffectivenessvideodiffusion,
      title={The Surprising Effectiveness of Video Diffusion Models for Hand Motion Reconstruction},
      author={Yuxi Wang and Chengkai Jin and Yufei Liu and Wenqi Ouyang and Tianyi Wei and Zhiwei Zeng and Siyuan Huang and Zhiqi Shen and Xingang Pan},
      year={2026},
      eprint={2606.30308},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2606.30308},
}