GitHub - YangDi666/SSTA-PRS: [WACV 2021] Selective Spatio-Temporal Aggregation based Pose Refinement System: Towards understanding human activities in real-world videos. (original) (raw)

Selective Spatio-Temporal Aggregation Based Pose Refinement System

Paper

Refined pose data

ad

SST-A toolbox

To obtain refined pose sequence, you need to:

  1. Extract 2D poses of input video from 3 expert estimators (e.g., LCRNet, OpenPose, AlphaPose, ...);
  2. Save 2D pose results into 'xxx-pose1.npz', 'xxx-pose2.npz', 'xxx-pose3.npz', .... Make sure '.npz' has the dimension of nb_frames * nb_joints * 2 and joints indexes are consistent;
  3. Run the script to get refined pose by SST-A. Ouput will be save as 'output.npz'.

python tools/ssta.py --pose1 --pose2 --pose3 --outname (--gt )

An example:

python tools/ssta.py --pose1 demo/WatchTV_p02_r03_v05_c05_Alphapose2d.npz --pose2 demo/WatchTV_p02_r03_v05_c05_LCRNet2d.npz --pose3 demo/WatchTV_p02_r03_v05_c05_OP2d.npz --outname demo/WatchTV_p02_r03_v05_c05_SSTA2d.npz

3D visualization

For 3D visualization, please use VideoPose3D.

Activity Recognition

For pose based activity recognition, please ref to UNIK / 2s-AGCN for smarthome.

Citation

If you find this code useful for your research, please consider citing our paper:

@InProceedings{Yang_2021_WACV, author = {Yang, Di and Dai, Rui and Wang, Yaohui and Mallick, Rupayan and Minciullo, Luca and Francesca, Gianpiero and Bremond, Francois}, title = {Selective Spatio-Temporal Aggregation Based Pose Refinement System: Towards Understanding Human Activities in Real-World Videos}, booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)}, month = {January}, year = {2021} }