GitHub - WaldJohannaU/3RScan: 3RScan Toolkit (original) (raw)

3RScan is a large scale, real-world dataset which features 1482 3D reconstructions / snapshots of 478 naturally changing indoor environments, designed for benchmarking emerging tasks such as long-term SLAM, scene change detection and object instance re-localization

Each sequence comes with aligned semantically annotated 3D data and corresponding 2D frames, containing in detail:

• calibrated RGB-D sequences.
• textured 3D meshes.
• 6DoF camera poses and camera calibration parameters K.
• global alignment among scans from the same scene as a global transformation T.
• dense instance-level semantic segmentation where each instance has a fixed ID that is kept consistent across different sequences of the same environment.
• object alignment, i.e. a ground truth transformation for each changed object together with its symmetry property.

Paper

If you find the data useful please consider citing our paper:

@inproceedings{Wald2019RIO,
    title={RIO: 3D Object Instance Re-Localization in Changing Indoor Environments},
    author={Johanna Wald, Armen Avetisyan, Nassir Navab, Federico Tombari, Matthias Niessner},
    booktitle={Proceedings IEEE International Conference on Computer Vision (ICCV)},
    year = {2019}
}

Data Organization and Format

The data in 3RScan is organized by RGB-D sequence. Each sequence has a unique hash value to identify the scan. The RGB-sequences and 3D reconstructions are all stored together in a separate folder. The directory has the following structure:

<scanId>
|-- mesh.refined.v2.obj
    Reconstructed mesh
|-- mesh.refined.mtl
    Corresponding material file 
|-- mesh.refined_0.png
    Corresponding mesh texture
|-- sequence.zip
    Calibrated RGB-D sensor stream with color and depth frames, camera poses
|-- labels.instances.annotated.v2.ply
    Visualization of semantic segmentation
|-- mesh.refined.0.010000.segs.v2.json
    Over-segmentation of annotation mesh
|-- semseg.v2.json
    Instance segmentation of the mesh (contains the labels)

Data Formats

The following are overviews of the data formats used in 3RScan:

Reconstructed surface mesh file (*.obj): OBJ format mesh with +Z axis in upright orientation.

RGB-D sensor data (*.zip): ZIP-archive with per-frame color, depth, camera pose and camera intrinsics.

Instance segmentation of the mesh (semseg.v2.json):

{ "scan_id": "8eabc405-5af7-2f32-86d2-d757fa7b019d", ... "segGroups": [ { "id": 15, "objectId": 15, "label": "window", "segments": [ 21, 175, ... ], "obb": { "centroid": [ 2.15, 2.17, -1.18 ], "axesLengths": [ 4.57, 0.62, 4.14 ], ... }, ... }, { "objectId": 29, "label": "plant", ... }, { "id": 14, "objectId": 14, "label": "windowsill", ... }

meta data file (3RScan.json):

[ { "reference": "531cff08-0021-28f6-8e08-ba2eeb945e09", // id of the initial scan "type": "train"
"ambiguity": [ [ { "instance_source": 34, "instance_target": 35, "transform": [ ... ] // transformation of instance 34 to instance 35 (to resolve instance ambiguity) }, {...} ] ], "scans": [ // rescans { "reference": "531cff10-0021-28f6-8f94-80db8fdbbbee", // id of rescan "transform": [ ... ] // transformation to align rescan with reference "nonrigid": [ ... ], // list of instances with nonrigid changes "removed": [], // removed instances "rigid": [ // rigid changes { "instance_reference": 35, // instance ID in reference "instance_rescan": 35, // instance ID in rescan "symmetry": 0, // symmetry (0 = none) "transform": [ ... ] // transformation to align instance in the reference to the instance in the rescan }, { ... }
],

  },
  // metadata of other rescans
  {
    "reference": "19eda6f4-55aa-29a0-8893-8eac3a4d8193", ... 
  }, { ... }
]

} ]

3RScan Library and Examples

For examples on how to read these files or how to use 3RScan in general see our code samples and documentation. You have a question? Check out our FAQ.

Notes

• Training Data: You can find the train, test and validation splits here: [train, val, test]
• Segmentation format (surface mesh segmentation and aggregated semantic annotation) is similar to ScanNet
• Please see our project page for more information.