DEPARTMENT OF COMPUTATIONAL AND DATA SCIENCES
Ph.D. Thesis Colloquium
Speaker: Mr. Boya Sumanth Kumar
S.R. Number: 06-18-01-10-12-20-1-18435
Title: “Transformer Neural Operators for Learning Generalized Solutions of Partial Differential Equations and Data Assimilation”
Research Supervisor: Dr. Deepak Subramani
Date & Time : October 15, 2025 (Wednesday), 02:00 PM
Venue : #102, CDS Seminar Hall
ABSTRACT
Scientific machine learning is transforming the way we solve physical dynamics governed by partial differential equations (PDEs). Solving nonlinear PDEs across multiple initial and boundary conditions requires re-running traditional numerical solvers, and existing physics-informed neural networks require costly retraining for each new condition. Furthermore, with the availability of sparse observational data in the meteorological and ocean science domains, the necessity for data assimilation in forward simulations is crucial. The above challenges are addressed by proposing three neural operators with architectural innovations: (1) the Physics-Informed Transformer Neural Operator (PINTO) for simulation-free PDE solving that is trained using only physics loss and generalizes to unseen initial and boundary conditions, (2) the Physics-Guided Transformer Neural Operator (PGNTO) for finding generalizable PDE solutions from simulation data in the absence of governing equations, and (3) the Implicit Neural Transformer Operator (INTO) for the data assimilation task of reconstructing continuous fields from limited observational data. Collectively, the above contributions push the boundaries of scientific machine learning in solving PDEs and data assimilation.
The first part of this thesis introduces a novel Physics-Informed Transformer Neural Operator (PINTO). Current neural operator approaches, though capable of learning functional mappings between infinite-dimensional spaces, suffer from two critical limitations: dependence on substantial simulation data and poor generalization to unseen conditions. PINTO addresses these fundamental challenges by introducing a novel physics-informed framework that achieves efficient generalization through simulation-free, physics-only training. The core innovation of PINTO lies in the development of iterative kernel integral operator units that leverage cross-attention mechanisms to transform domain points into initial and boundary condition-aware representation vectors. This attention-based architecture enables context-aware learning that fundamentally differs from existing neural operators, allowing PINTO to learn mappings from input functions (initial/boundary conditions) to complete PDE solution spaces through a single forward pass. The architecture comprises three key stages: lifting layers for projecting the solution’s domain coordinates to a higher-dimensional representation space, iterative kernel integration layers for context-aware representation learning, and projection layers for mapping the learned representation to the solution space. We demonstrate PINTO’s superior performance across critical fluid mechanics and engineering applications, including linear advection, nonlinear Burgers equation, and steady and unsteady Navier-Stokes equations spanning multiple flow scenarios. Under challenging unseen conditions, PINTO achieves relative errors of merely 20 to 33% of those obtained by state-of-the-art physics-informed neural operator methods when compared with analytical and numerical solutions. Critically, PINTO exhibits temporal extrapolation capabilities absent in competing approaches, accurately solving the advection and Burger’s equations in time steps not present during training.
In the second part of the thesis, we extend our framework to physics-guided transformer neural operators (PGNTO), trained on simulation data rather than physics loss, to address scenarios where governing PDEs are unavailable and to eliminate the training instabilities that arise due to physics loss for high-dimensional PDEs. We introduce Gabor filters in the coordinate lifting layers to develop PGNTO, a version of our transformer neural operator that shows improved PDE solving with simulation data. Our PGNTO successfully solves standard PDE benchmarks that include the nonlinear Burgers, Darcy, 2D Navier Stokes, plasticity, elasticity, and also scales to complex 3D turbulent flows, specifically predicting wake dynamics behind wind turbines under varying inlet velocities. Across all test cases, PGNTO maintains superior accuracy with relative errors that are only one-third of competing neural operators.
Finally, in the third part of the thesis, we introduce the implicit neural transformer operator (INTO) for neural data assimilation, enabling continuous field reconstruction from sparse observational data. By modifying PINTO’s architecture with Gabor filters for coordinate lifting and Fourier filters for value representation, INTO outperforms existing implicit neural representation networks on reconstruction tasks. Uniquely, a single INTO model generalizes across varying sparsity levels, eliminating the need for training separate models for different data availabilities. The performance of INTO is demonstrated on global surface temperature and air temperature datasets, and compared with other leading deep neural models for physical field reconstruction.
Overall, this thesis establishes transformer-based neural operators as a unified framework for physics-informed PDE solving and neural data assimilation, achieving unprecedented generalization, computational efficiency, and accuracy while requiring minimal training data. Our suite of transformer neural operators opens new avenues for real-time prediction and data assimilation in physics and engineering applications.
ALL ARE WELCOME
