EgoSSA: Egocentric Stereo Structure-Aware 3D Hand Reconstruction for American Sign Language Gesture Modeling

Egocentric hand reconstruction is a fundamental technology for a range of applications, including augmented and virtual reality, human-computer interaction, and behavioral analysis. In this work, we focus on a particularly challenging and impactful use case: accurate reconstruction of American Sign Language (ASL) gestures from first-person dual-camera input. ASL involves rapid, fine-grained, and semantically rich two-handed articulations, which pose significant challenges for existing egocentric reconstruction methods due to motion blur, frequent self-occlusion, and limited temporal modeling. To address these limitations, we propose EgoSSA, a novel egocentric stereo hand reconstruction framework that integrates geometrically constrained epipolar-guided cross-view fusion (EpiCVF) and structure-aware latent modeling (SALM) for stable reconstruction under occlusion and fast motion. Specifically, EpiCVF integrates epipolar constraints and temporal coherence to enable robust cross-view feature fusion, while SALM encodes anatomical and temporal structure into a latent space to prevent physically implausible hand poses. To address the lack of egocentric ASL datasets, we introduce EgoStereoASL-10, a novel stereo egocentric dataset captured under realistic signing conditions. We further propose motion-aware metrics to more effectively assess reconstruction performance across varying gesture dynamics. Experiments on both the public benchmark HOT3D and our EgoStereoASL-10 dataset demonstrate that our method achieves state-of-the-art performance for egocentric 3D ASL hand reconstruction.