Skip to content

Latest commit

 

History

History
63 lines (42 loc) · 4.28 KB

README.md

File metadata and controls

63 lines (42 loc) · 4.28 KB
Raw

X3D

X3D: Expanding Architectures for Efficient Video Recognition

Abstract

This paper presents X3D, a family of efficient video networks that progressively expand a tiny 2D image classification architecture along multiple network axes, in space, time, width and depth. Inspired by feature selection methods in machine learning, a simple stepwise network expansion approach is employed that expands a single axis in each step, such that good accuracy to complexity trade-off is achieved. To expand X3D to a specific target complexity, we perform progressive forward expansion followed by backward contraction. X3D achieves state-of-the-art performance while requiring 4.8x and 5.5x fewer multiply-adds and parameters for similar accuracy as previous work. Our most surprising finding is that networks with high spatiotemporal resolution can perform well, while being extremely light in terms of network width and parameters. We report competitive accuracy at unprecedented efficiency on video classification and detection benchmarks.

Results and Models

Kinetics-400

frame sampling strategy resolution backbone top1 10-view top1 30-view reference top1 10-view reference top1 30-view config ckpt
13x6x1 160x160 X3D_S 73.2 73.3 73.1 [SlowFast] 73.5 [SlowFast] config ckpt[1]
16x5x1 224x224 X3D_M 75.2 76.4 75.1 [SlowFast] 76.2 [SlowFast] config ckpt[1]

[1] The models are ported from the repo SlowFast and tested on our data. Currently, we only support the testing of X3D models, training will be available soon.

  1. The values in columns named after "reference" are the results got by testing the checkpoint released on the original repo and codes, using the same dataset with ours.
  2. The validation set of Kinetics400 we used is same as the repo SlowFast, which is available here.

For more details on data preparation, you can refer to Kinetics400.

Test

You can use the following command to test a model.

python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [optional arguments]

Example: test X3D model on Kinetics-400 dataset and dump the result to a pkl file.

python tools/test.py configs/recognition/x3d/x3d_s_13x6x1_facebook-kinetics400-rgb.py \
    checkpoints/SOME_CHECKPOINT.pth --dump result.pkl

For more details, you can refer to the Test part in the Training and Test Tutorial.

Citation

@misc{feichtenhofer2020x3d,
      title={X3D: Expanding Architectures for Efficient Video Recognition},
      author={Christoph Feichtenhofer},
      year={2020},
      eprint={2004.04730},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}