GitHub - YangDi666/SSTA-PRS: [WACV 2021] Selective Spatio-Temporal Aggregation based Pose Refinement System: Towards understanding human activities in real-world videos. (original) (raw)
Selective Spatio-Temporal Aggregation Based Pose Refinement System
Paper
Refined pose data
- Skeleton data of Smarthome:
SST-A toolbox
To obtain refined pose sequence, you need to:
- Extract 2D poses of input video from 3 expert estimators (e.g., LCRNet, OpenPose, AlphaPose, ...);
- Save 2D pose results into 'xxx-pose1.npz', 'xxx-pose2.npz', 'xxx-pose3.npz', .... Make sure '.npz' has the dimension of nb_frames * nb_joints * 2 and joints indexes are consistent;
- Run the script to get refined pose by SST-A. Ouput will be save as 'output.npz'.
python tools/ssta.py --pose1 --pose2 --pose3 --outname (--gt )
An example:
python tools/ssta.py --pose1 demo/WatchTV_p02_r03_v05_c05_Alphapose2d.npz --pose2 demo/WatchTV_p02_r03_v05_c05_LCRNet2d.npz --pose3 demo/WatchTV_p02_r03_v05_c05_OP2d.npz --outname demo/WatchTV_p02_r03_v05_c05_SSTA2d.npz
3D visualization
For 3D visualization, please use VideoPose3D.
Activity Recognition
For pose based activity recognition, please ref to UNIK / 2s-AGCN for smarthome.
Citation
If you find this code useful for your research, please consider citing our paper:
@InProceedings{Yang_2021_WACV, author = {Yang, Di and Dai, Rui and Wang, Yaohui and Mallick, Rupayan and Minciullo, Luca and Francesca, Gianpiero and Bremond, Francois}, title = {Selective Spatio-Temporal Aggregation Based Pose Refinement System: Towards Understanding Human Activities in Real-World Videos}, booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)}, month = {January}, year = {2021} }