Authors: Lei Wang, Jun Liu, Liang Zheng, Tom Gedeon, Piotr Koniusz

Published on: February 07, 2024

Impact Score: 8.07

Arxiv code: Arxiv:2402.04599

Summary

• What is new: JEANIE presents a novel approach by incorporating both temporal and viewpoint alignments for comparing 3D skeleton sequences, advancing beyond Dynamic Time Warping (DTW) which only aligns temporally.
• Why this is important: Difficulty in comparing video sequences due to nuisance variations like speed, temporal locations, and poses, leading to temporal-viewpoint misalignment.
• What the research proposes: JEANIE aligns sequence pairs by jointly considering temporal blocks and camera viewpoints, using simulated views and matching support-query pairs with local warping.
• Results: JEANIE achieves state-of-the-art results on multiple datasets (NTU-60, NTU-120, Kinetics-skeleton, UWA3D Multiview Activity II) in both supervised and unsupervised Few-shot Action Recognition.

Technical Details

Technological frameworks used: 3D skeleton sequences manipulation, Dynamic Time Warping for alignment reference

Models used: Local temporal-viewpoint warping, unsupervised Few-shot Action Recognition algorithm

Data used: NTU-60, NTU-120, Kinetics-skeleton, and UWA3D Multiview Activity II datasets

Potential Impact

This research could significantly impact video analytics, security surveillance, sports analytics, and health monitoring industries by improving action recognition accuracy.

Want to implement this idea in a business?

We have generated a startup concept here: ViewFlex.