GitHub - jan-ackermann/cl-splats: Official code for "CL-Splats: Continual Learning of Gaussian Splatting with Local Optimization" ICCV 2025 (original) (raw)

ICCV 2025

Jan Ackermann Jonas Kulhanek Shengqu Cai Haofei Xu Marc Pollefeys Gordon Wetzstein Leonidas Guibas Songyou Peng

TL;DR: CL-Splats optimizes existing 3DGS scene representations with a small set of images showing the changed region.

Contents

• Install
• Dataset Format
• Usage
• Configuration
• Todos
• Citation

Install

Pre-requisites

While not strictly necessary for using our method, COLMAP is necessary to obtain camera poses for the initial reconstruction as well as to add new observations to existing models. Please follow the instructions on the COLMAP website to install COLMAP. If possible install it with CUDA support.

Environment

We tested our code on Ubuntu 24.04 with CUDA 12.8, and verified the full pipeline on Debian with CUDA 12.4 (A100, torch 2.6.0+cu124, Python 3.13). Install via pip (we recommend a conda/venv environment):

Create and activate environment

conda create -n cl-splats python=3.10 conda activate cl-splats

Install the package with everything needed for training:

[train] → transformers (Depth-Anything), torchmetrics, wandb

[gsplat] → gsplat rasterizer + ninja (for its CUDA JIT build)

pip install -e '.[train,gsplat]'

A bare pip install -e . installs only the core dependencies — enough for the unit tests, but not for training (cl-splats-train needs the train and gsplat extras).

**Note — CUDA versions:**Your PyTorch wheel's CUDA version must be supported by your NVIDIA driver, and nvcc (CUDA Development Kit) must be on PATH with a matching version — gsplat compiles its CUDA kernels with nvcc on first use (one-time, ~2 min; it also needs ninja, installed by the gsplat extra). For example, on a CUDA 12.4 driver install a cu124 wheel:

pip install torch torchvision --index-url https://download.pytorch.org/whl/cu124

A wheel built for a newer CUDA than the driver supports (e.g. cu13x on a 12.4 driver) makes torch.cuda.is_available() return False.

Data

The synthetic continual-learning benchmark (Blender Levels 1–3 with add/delete/move/multi changes) and the real-world scenes are hosted on HuggingFace at ackermannj/cl-splats-dataset:

Example: download the Level-1 base scene + 'add' change (skips .blend/checkpoint files)

python -c " from huggingface_hub import snapshot_download snapshot_download('ackermannj/cl-splats-dataset', repo_type='dataset', local_dir='data', allow_patterns=['Blender-Levels/Level-1/transforms_train.json', 'Blender-Levels/Level-1/train/', 'Blender-Levels/Level-1/add/']) "

cl-splats-train --data-path data/Blender-Levels/Level-1 --change-type add --eval --white-background

Dataset Format

CL-Splats supports two dataset formats, auto-detected at runtime.

COLMAP (real-world scenes)

Standard COLMAP workspace layout — what preprocessing.py produces:

path/to/dataset/
├── images/           # undistorted images
│   ├── frame_00001.jpg
│   └── ...
└── sparse/
    └── 0/
        ├── cameras.bin
        ├── images.bin
        └── points3D.bin

Run with:

cl-splats-train --data-path path/to/dataset

Computing Poses

For your convenience, we provide a preprocessing script that runs COLMAP automatically. It assumes raw images are organised as timestep folders:

path/to/your/input/
├── t0/   # images for timestep 0 (base scene)
│   ├── *.{png,jpeg,jpg}
│   └── ...
├── t1/   # images for timestep 1 (after changes)
│   └── ...
└── ...

Run:

python3 clsplats/utils/preprocessing.py --input_dir <path/to/your/input>

Note:
Our codebase currently only supports NeRF-Synthetic and COLMAP pose formats, and their naming scheme must be consistent with the output of our preprocessing script.

Blender / NeRF-Synthetic (CL benchmark scenes)

Used for the synthetic continual-learning benchmark dataset. Each timestep lives in its own subdirectory with a transforms_train.json file:

path/to/Level-1/
├── transforms_train.json   # base scene (t0)
├── images/
│   └── ...
└── add/                    # change subfolder named after the change type
    ├── transforms_train.json
    └── images/
        └── ...

Available change types: add, delete, move, multi.

Run with:

cl-splats-train --data-path path/to/Level-1 --change-type add

Usage

Basic Training

Real-world COLMAP scene (single timestep)

cl-splats-train --data-path path/to/dataset

Blender CL scene (base + one change timestep)

cl-splats-train --data-path path/to/Level-1 --change-type add

CLI Reference

Run cl-splats-train --help to see all options.

Overriding Config Values (Hydra)

Any configuration value can be overridden directly on the command line as positional arguments using Hydra dot-notation:

More iterations, lower position learning rate

cl-splats-train --data-path path/to/dataset
train.iters_per_timestep=500
train.position_lr_init=8e-5

Tighter change-detection threshold with dilation

cl-splats-train --data-path path/to/Level-1 --change-type add
change.threshold=0.7
change.dilate_mask=true
change.dilate_kernel_size=15

Enable live W&B syncing

cl-splats-train --data-path path/to/dataset wandb_mode=online

Output

Results are saved to outputs/<run-name>/ by default and include:

• gaussians_t<N>.ply — Gaussian splat checkpoint at each trained timestep,
  viewable in SuperSplat or Luma AI
• eval/t<N>/ — per-view rendered + GT images and console PSNR/SSIM when --eval is used
• W&B run logs (synced live, or stored offline for later upload)

Evaluating on a Test Split

Pass --eval to hold out a test set (every 8th camera LLFF-style for COLMAP; transforms_test.json for Blender) and compute PSNR and SSIM after training each timestep:

cl-splats-train --data-path path/to/dataset --eval

Per-view metrics are printed to stdout and (if W&B is active) logged under eval/t<timestep>/psnr and eval/t<timestep>/ssim. Rendered images are saved to outputs/eval/t<N>/.

The evaluation uses torchmetrics when installed (pip install 'torchmetrics[image]') and falls back to a pure-PyTorch SSIM/PSNR implementation otherwise.

Standalone Evaluation (cl-splats-eval)

To evaluate a previously saved .ply checkpoint without re-running training, use the dedicated cl-splats-eval command:

Evaluate a checkpoint on its dataset test split

cl-splats-eval
--ply outputs/gaussians_time_0001.ply
--data-path path/to/dataset

Blender CL scene — evaluate at t=1 (change timestep)

cl-splats-eval
--ply outputs/gaussians_time_0001.ply
--data-path path/to/Level-1
--change-type add
--timestep 1

Flags:

Rendered images are saved as <out_dir>/t<N>/<name>_render.png and <name>_gt.png. Metrics are printed to stdout.

Recovering Past Scene States (cl-splats-history)

CL-Splats only optimises the Gaussians inside the changed region; the rest of the scene is frozen by construction. This makes every past timestep exactly recoverable from the final model plus a small per-timestep delta (the pre-update state of the changed Gaussians) — no need to store a full checkpoint per timestep.

Train with history.log_history=true to save the deltas to outputs/history/, then reconstruct any earlier timestep:

cl-splats-history
--ply outputs/gaussians_time_0001.ply
--history-dir outputs/history
--time 0
--out outputs/gaussians_recovered_t0.ply

Flags:

The recovered checkpoint is bit-identical to the model as it existed at that timestep and can be evaluated with cl-splats-eval or opened in any 3DGS viewer.

Configuration

The default config lives in configs/cl-splats.yaml. A summary of the most useful keys:

train

change — DINOv2 change detection

lifter — Depth-Anything V2 3D lifting

model — Gaussian representation

constraints — Geometry constraints

Todos

I continue to release the missing modules required to replicate our method.

• Release initial codebase with framework skeleton.
• Release camera estimation script.
• Release fast change detection module.
• Release sampling module.
• Release pruning module.
• Release data.
• Release local-optimization CUDA kernels.
• Verify codebase.
• Release history recovery.

Disclaimer

Parts of this public reimplementation were developed and verified with the help of Claude (Anthropic's AI coding assistant). While the pipeline has been tested end to end on the released benchmark data, discrepancies with the original paper implementation may exist. If you notice any, please reach out via the email listed on my website.

Citation

@inproceedings{ackermann2025clsplats,
    author={Ackermann, Jan and Kulhanek, Jonas and Cai, Shengqu and Haofei, Xu and Pollefeys, Marc and Wetzstein, Gordon and Guibas, Leonidas and Peng, Songyou},
    title={CL-Splats: Continual Learning of Gaussian Splatting with Local Optimization},
    booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    year={2025}
}