GitHub - rozumden/fmo-deblurring-benchmark: [CVPR 2021] FMO Deblurring Benchmark (original) (raw)

Fast Moving Object (FMO) Deblurring Benchmark

Simple Python library to evaluate your FMO deblurring methods.

Datasets

All three datasets (TbD, TbD-3D, Falling Objects) can be downloaded by running (after modifying the data storage folder path):

bash download_datasets.sh

Usage

Implement a function that takes as input image I [w, h, 3], background B [w, h, 3], bounding box of approximation FMO location, the required number of generated sub-frames n (temporal super-resolution), and an approximate object size. Your method should output temporal super-resolution mini-video of size [w, h, 3, n]. Optionally, if you want to evaluate trajectory accuracy, output the sub-frame object trajectory of size [2, n] or None.

An example of a dummy algorithm that always outputs the input image and does not evaluate the trajectory accuracy:

def my_deblur(I,B,bbox,nsplits,radius): return np.repeat(I[:,:,:,None], nsplits, 3), None

Baselines

We provide several baseline and state-of-the-art methods.

Dummy baselines

Two baselines, one that always outputs the input image, and another that output the background image. Example is shown in example_dummy.py.

Deblatting

To evaluate this method, please check out the deblatting sub-module. We provide three versions of deblatting: classical deblatting with single appearance (TbD), deblatting with chanring appearance (TbD-3D), and deblatting with trajectory oracle (TbD-O). Examples are shown in example_deblatting.py.

DeFMO - current state-of-the-art

The easiest way to evaluate DeFMO is using Kornia (kornia.feature.DeFMO). Example is shown in example_defmo.py.

To evaluate this method using the original source coude, please download DeFMO. Example is shown in example_defmo_source.py.

Scores

TbD-3D-Oracle has access to the ground-truth trajectory. Therefore, it's not a competitive baseline and is provided just for the reference.

Falling Objects dataset

Arbitrary shaped and textured objects.

TbD-3D dataset

Mostly spherical but significantly textured objects moving in 3D.

TbD dataset

Mostly spherical and uniformly colored objects moving in a plane parallel to the camera plane.

Reference

If you use this repository, please cite the following publication:

@inproceedings{defmo, author = {Denys Rozumnyi and Martin R. Oswald and Vittorio Ferrari and Jiri Matas and Marc Pollefeys}, title = {DeFMO: Deblurring and Shape Recovery of Fast Moving Objects}, booktitle = {CVPR}, address = {Nashville, Tennessee, USA}, month = jun, year = {2021} }

The baseline TbD method:

@inproceedings{Kotera-et-al-ICCVW-2019, author = {Jan Kotera and Denys Rozumnyi and Filip Sroubek and Jiri Matas}, title = {Intra-frame Object Tracking by Deblatting}, booktitle = {Internatioal Conference on Computer Vision Workshop (ICCVW), Visual Object Tracking Challenge Workshop, 2019}, address = {Seoul, South Korea}, month = oct, year = {2019} }

The baseline TbD-3D or TbD-O method:

@inproceedings{Rozumnyi-et-al-CVPR-2020, author = {Denys Rozumnyi and Jan Kotera and Filip Sroubek and Jiri Matas}, title = {Sub-frame Appearance and 6D Pose Estimation of Fast Moving Objects}, booktitle = {CVPR}, address = {Seattle, Washington, USA}, month = jun, year = {2020} }

Some ideas are taken from:

@inproceedings{Rozumnyi-et-al-CVPR-2017, author = {Denys Rozumnyi and Jan Kotera and Filip Sroubek and Lukas Novotny and Jiri Matas}, title = {The World of Fast Moving Objects}, booktitle = {CVPR}, address = {Honolulu, Hawaii, USA}, month = jul, year = {2017} }