3D4Science (CVPR'26) (original) (raw)

Description

The integration of AI and machine learning into scientific computing, particularly in 3D and 4D geometry generation, is transforming how complex, dynamic phenomena are understood and modeled. Traditional 2D approaches are proving inadequate in capturing the full scope of multi-scale, heterogeneous scientific data, limiting the accuracy and applicability of simulations across such diverse fields as climate science, urban planning, and biomedicine. The ability to reconstruct high-fidelity 3D geometries enables more precise analysis of natural environments and man-made systems, driving breakthroughs in predictive modeling and hypothesis generation. As scientific challenges become more intricate — ranging from glacier melt patterns to urban infrastructure planning under extreme weather — accurate, scalable 3D methods are increasingly essential.

To tackle these challenges, this workshop seeks not only to surface key research questions but also to build an active and interdisciplinary community at the intersection of computer vision and scientific domains. By bringing together researchers, practitioners, and domain experts, we aim to spark meaningful dialogue, initiate new collaborations, and create lasting connections that extend beyond the event itself. This raises a set of new questions:

Q1: How to develop scalable, high-fidelity 3D/4D methods that seamlessly integrate heterogeneous views to represent complex and multi-scale scientific data?
Q2: How to optimize 3D/4D vision models with an awareness of and for the purposes of addressing downstream scientific problems?
Q3: How to balance the computational efficiency and accuracy in real-time 3D/4D reconstructions for scientific problems?

Topics

In this workshop, we aim to bring together experts from computer vision and scientific problems, spur discussions, and foster collaborations on broad and transformative questions and challenges (include but not limited to):

Multi-scale Patterns: How can 3D/4D models effectively capture both fine-grained and large-scale details in complex scientific datasets, such as fluid and smoke?
Large-scale Scenes: What techniques can improve the scalability of 3D/4D reconstructions for large environments like cities, forests, or glaciers, without sacrificing accuracy or computational feasibility?
Heterogeneous Views: How can we effectively integrate data from multiple sources (e.g., satellite, LiDAR, drone, mobile devices) to produce accurate and seamless 3D models while minimizing noise and alignment issues?
Dynamic and Time-varying Views: What methods can improve temporal coherence in 4D reconstructions of dynamic scenes, such as fast-moving natural systems or urban traffic, while avoiding artifacts?
Complex and Unstructured Geometries: How can 3D/4D models better handle irregular, unstructured geometries found in natural environments like mountains or coral reefs, particularly in the presence of sharp features?
Occlusions and Missing Observations: What techniques can be developed to fill gaps in occluded or incomplete data in real-world scenarios, ensuring accurate reconstructions despite missing perspectives or environmental obstacles?
Computational Complexity: How can we reduce the computational cost of high-quality 3D/4D reconstructions, especially for real-time or large-scale applications that require high-resolution output?
Generalization and Scene Adaptability: What approaches can help 3D/4D models generalize to new environments without retraining, enabling wider applicability across different scientific domains?
Real-time Rendering for Dynamic Scenes: How can we achieve real-time rendering for dynamic 4D scenes in complex environments, such as simulating natural disasters or fast-moving ecosystems?
Lighting and Viewpoint Variations: What novel algorithms can improve the robustness of 3D reconstructions in variable lighting or challenging viewpoints (e.g., low-light conditions or extreme weather)?

Scientific Domains. We invite paper submissions from various scientific domains, including but not limited to: Fluid Dynamics, Climate and Glaciology, Biomedicine and Medical Research, Astronomy and Planetary Science, Material Science, Physics and High Energy Research, Astrophysics and Space Science, Computational Modeling and Forecasting, Earth Science, Chemistry and Small Molecules, Ecology and Environmental Studies, Geosciences and Geology, Urban Planning and Architecture. Applications-driven submissions focusing on 3D/4D reconstructions for scientific data are also highly encouraged.

Confirmed Speakers (A-Z by Last Name)

Chuang Gan

Assistant Professor, UMass Amherst

Research Manager, MIT-IBM Watson AI Lab

Eitan Grinspun

Ali Haddad

Danny Kaufman

JJ (Jeong Joon) Park

Tamar Shinar

Jiajun Wu

Assistant Professor, Stanford University

Xiaoyu Xiang

Xiaoxiang Zhu

Call for Papers

We provide more submission details: Guidance for 3D4Science CFP at CVPR 2026.
OpenReview submission portal: https://openreview.net/group?id=thecvf.com/CVPR/2026/Workshop/3D4S

Tentative important dates (deadline updated!) (AoE time):

Abstract Submission Deadline: April 5, 2026
Paper Submission Deadline: April 7, 2026
Review Bidding Period: April 5 - April 7, 2026
Review Deadline: April 25, 2026
Acceptance/Rejection Notification Date: April 27, 2026
Camera-Ready Submission: May 14 2026
Workshop Date: June 4, 2026

Schedule

All times are in Denver Time (GMT-6).

Denver Time (GMT-6)	Event
7:30-7:40	Opening Remarks
7:40-8:00	Oral Talks
8:00-8:30	Invited Talk 1: Xiaoxiang Zhu
8:30-9:00	Invited Talk 2: Tamar Shinar Title: "Reconstruction of implicit surfaces from fluid particles using convolutional neural networks" Abstract: We present a network-based method for reconstructing signed distance functions from fluid particles. Particles are mapped to a regular grid via a weighting kernel and processed by a CNN, with regression-based regularization that reduces surface noise while preserving high-curvature features. This yields superior spatial smoothness and temporal coherence, with robust behavior across varying sampling densities and thin features. We further accelerate the reconstruction scheme through algorithmic modifications, achieving 33x single-frame speedup with no perceptible quality loss, making the surface reconstruction fast enough for use within a simulation framework. We also present a benchmark suite for machine-learning surface tension computation, addressing the challenge of accurate interface curvature estimation in particle-based fluid simulations.
9:00-9:30	Invited Talk 3: Ali Haddad Title: "Physical AI in the Operating Room: Real-Time 3D/4D Scene Understanding for Surgical Intelligence" Abstract: "The operating room is one of the most demanding scene understanding problems in science: deformable and unstructured anatomy, heavy occlusion from instruments and blood, fluorescence and white-light imaging fused in the same field, viewpoint and lighting that change second-to-second, and a hard real-time budget on edge hardware. This talk presents how XRlabs is building intraoperative intelligence for this setting — combining real-time perception on a surgical exoscope with patient-specific digital twins at the edge of the surgical field. I'll walk through the geometry stack behind it: heterogeneous-view reconstruction across exoscope tool detection; and the use of synthetic surgical scenarios to train and validate perception and policy models before they ever touch a patient. I'll close with lessons from our first-in-human deployment of Physical AI in cranial neurosurgery, and the open 3D/4D research questions that, if solved, would meaningfully shift what is possible in surgery."
9:30-10:00	Invited Talk 4: Jiajun Wu Title: Seeing 4D Fluid Fields
10:00-10:30	Invited Talk 5: Jeong Joon Park Title: Beyond Neural Operators Towards Neural Solvers Abstract: ""Neural operators learn fast approximate mappings from PDE parameters to solutions. In this talk, I will talk about Error-Conditioned Neural Solvers (ENS), which internalize the correction loop within the network itself. Our neural solvers recurrently correct its predictions by computing the PDE residual field of the current prediction and passing it as an explicit input to the network, until the residual converges to a consistent floor. ENS achieves state-of-the-art prediction accuracy across PDE families while maintaining physical accuracy across diverse extrapolation settings, including zero-shot parameter shifts, super-resolution, and cross-equation transfer."
10:30-11:00	Invited Talk 6: Xiaoyu Xiang Title: "Toward Interactable 3D World Generation"
11:00-11:30	Invited Talk 7: Danny Kaufman Title: "Multiscale Adaptive Simulation for Predictive Soft-Body Modeling" Abstract: "Physics-based simulation methods are essential tools across applications ranging from scientific modeling and robotics to fashion and entertainment. A fundamental limitation to their application, however, is resolution. As we increase resolution, recently developed simulation methods like IPC can now can deliver unprecedented fidelity, expressiveness and accuracy -- opening the door to many new applications. However, increased resolution incurs significant and potentially prohibitive costs. To address these challenges a range of fast simulations methods have recently been developed that provide impressive performance acceleration at the cost of significantly degraded fidelity and reliability. Unfortunately, this makes them unsuitable for building models of our real world. So on the one hand we have fast performance with inaccuracies and instabilities and, on the other, slow performance with predictive and reliable output. Is there no free lunch? In this talk I'll cover our recent work on addressing this question via multiscale and adaptive strategies. I'll first cover our work on Progressive Simulation methods. Progressive Simulation is a multiscale framework for predictive, scalable simulation across a hierarchy of efficient solution approximations that preserve invariants and converge at finest levels. I'll then cover our companion works on In-Timestep Remeshing methods that temporally adapt resolution, in-application, to optimally move compute resources where they're most needed at each instant. I'll discuss applications were these methods provide efficient, scalable simulations and surrogates for digital reproduction and inverse modeling performed at orders of magnitude less cost than previously possible."
11:30-noon	Invited Talk 8: Chuang Gan Title: "Building AI Agents with Physical Common Sense"
noon-12:30pm	Poster